Pixelated thoughts on designing for the connected generation

User testing a dynamic form creator

Last year, we embarked upon a new project to create and deliver a usable, fast, and flexible generic application submission and review system.
The core component of this project, tentatively named EEE Scout, is the form creator. This component will be used to create forms from a variety of elements, including multiple choice questions, short/medium/long text responses, and file uploads.
We also decided early on that this form creator would be highly dynamic. Our course management system, EEE, has a form creator as part of its Quiz and Survey apps, but was designed a decade ago. Our goals were:
  • Speed - users should be able to quickly create forms/recreate their paper forms
  • Usability - somebody should be able to go in and create their form without having to contact us or refer to the help docs
  • Not be annoying - minimize page refreshes (none, if possible)
This presented an interesting problem for our usability tests. Most of our tools are highly state-driven, making paper prototypes simple to create and iterate. However, this new form creator would have dropdown menus, drag-and-drop, and inline editing, which meant that I had to come up with a new (for us) way of testing.

Before and after - what the user starts with, and where a successful path would take them.
I started off by creating the task I wanted the user to complete: "The existing paper form will be used to create the new online form. Your task: Recreate the paper form provided." To provide some context to my testers, I also provided them with their background: "You are a counselor in a student affairs office. Each year, your school offers a scholarship to students. Interested students are required to fill out a paper form and submit it to the student affairs office by a certain date. The forms are then copied and given to two faculty members, each of whom volunteer to review the submissions. The three of you then review the applications and choose a the best fit for the scholarship.".
After creating the tasks, I set about in Balsamiq creating mockups. The screen cap above shows the before and after, where the user would start, and where I'd like them to end up. To simulate the dynamic environment we were going for, I created a variety of widgets.


Widgets for usability test
Then, I took my trusty scissors, and cut them all out. For the test itself, whenever the user would do something (say, click on "Add Element"), I'd (quickly!) take the "Add Element" widget and put it out. Then, if they clicked on "Header", I'd put the header element on the table (on the blank form). In this way, they were able to create the form in whatever order they wanted, and, since everything was on its own strip of paper, reordering (and removing) was easy. I also asked a coworker to help take notes, since I was focused on simulating the environment.
What did I learn?
This test validated our approach - the dynamic form creator was apparently usable, people enjoyed being able to see their form built-up right in front of them, and commented on how much better it was than our existing form creation tools. I made a few tweaks (such as adding in an "Undo" for delete, and rearranging the icons so it was easier to edit (as well as noting a requirement to provide for inline editing when a user clicks on text, since they tried that a few times)).
Applying this to other projects
Usability testing is quick, effective, and actually fun (my office looked like an arts-and-crafts center during this particular test). Creating tasks doesn't take all that long, as long as you define your goal up front, maybe an hour or so. Throwing together the mockups in Balsamiq shouldn't take more than a couple of hours, assuming that you go through a few revisions like I did. The tests themselves took about 30 minutes (there were some other tasks targeting different functionality in the tool), and then another hour or so analyzing the results and making design tweaks.