(Originally published in 2009 and still highly relevant.)
Dana Chisnell presented an all-day session at the EDUI 2009 Conference held in Charlottesville September 21 & 22, 2009. She founded Usability Works and does usability research, user interface design, and technical communication consulting and development. She is co-author of the Handbook of Usability Testing. The focus of the workshop was to show how eminently doable user testing can be, if you let go of the idea that you have to do it “by the book.” Classic, by-the-book user testing can seem intimidating and scary. Instead, Dana spent the day showing how to identify and do the essentials and make it easier to test, and to do it on an on-going basis. Her approach takes into account that there usually isn’t a lot of time to test. The following is a list of all elements “by the book,” and I’ve noted which ones Dana says are essential:
- Develop a test plan ESSENTIAL
- Choose a testing environment–Don’t test in the lab—go to where the users are
- Find and select participants–ESSENTIAL
- Prepare test materials
- Conduct the sessions–ESSENTIAL
- Debrief with participants and observers–ESSENTIAL–Debrief only with observers
- Analyze data and observations
- Create findings and recommendations
User testing works best when you’re early in the design process (that’s where testing using paper prototypes comes in very handy). What you need is someone who will try the design; somewhere to test (preferably where the user typically would use your site, such as at home, the office, or in a dorm room); something to study. Neat and simple. And you’ll learn things about your website interface that you would never have seen on your own.
For example, in conducting user tests of the University of Pennsylvania website, Dana learned that users were looking for admissions “dates.” But they couldn’t find the information because the University used a different term: “timetable.” User testing provided the context in which they learned about this very simple labelling problem. Your site has to accommodate the way your users think and work, and you can’t be certain about that unless you actually watch them use your site. You must learn whether the user grasps the conceptual design of your site and if not, what the user’s concept is.
Pointers for moderating user tests
There’s tremendous value in sitting next to someone and watching them do something. Dana recommends saying to the tester things like “tell me what you’re doing–think aloud… tell me what you’re thinking…” When the tester is quiet, be careful about when you ask a question–sometimes they’re quiet because they’re working on a problem, and you may not want to interrupt that process. At the end of the test, review the experience with the tester. Walk through what happened, what happened next. Ask “how did that go? What was confusing or frustrating.” Dana conducted a demonstration, working with an attendee. I found the moderator questions she asked to be very helpful:
- How would you describe the information you’re looking for?
- How far do you think you are from getting it?
- Are you warm or cool?
- You were hovering there–tell me what you were thinking.
- What question do you have about the site right now?
- What do you think the site is about?
- What one main thing should be improved?
It’s essential for the moderator to talk with the tester and create a situation that feels natural to him or her. As moderator, you should be neutral and objective, but also friendly. Remember that this is not a psychology experiment, it’s a test of user-centered design.
NEXT: What Dana says about design.