Tips to Get the Most Out of Your Usability Study – Part 1
User testing should be an integral part of creating or redesigning any web presence. Unfortunately, many think of it as only a “nice to have”— something to include only if you havethe luxury of time and money. However, if you are a business owner ready to make changes in your online presence and are hesitant to include user testing because of the time and expense — think of it this way: what you will save in rework, time and customer satisfaction is priceless.
User testing doesn’t have to break the bank. Here are some suggestions to make your user testing project efficient and productive:
Limit your test subjects to five. Yes, you heard that right! Only five! According to usability guru Jakob Nielsen, this allows you to test almost as many usability problems as you’d find using many more participants. Why? Nielsen says that it doesn’t matter whether you’re testing websites, intranets, applications, or mobile apps, with five users you almost always get close to user testing’s maximum benefit cost ratio. Nielsen charted 83 usability consulting projects correlating how many users were tested vs. the number of insights reported. The chart shows that across this many projects, testing more users didn’t necessarily result in more insights. Depending upon the type of testing you’re doing, there are exceptions , so I encourage you read the entire article for more insights and useful information.
Screen intelligently. Create a recruiting screener that clearly defines all participant requirements. Decide which requirements are “make” or “break” requirements and put those at the beginning of the screener. You don’t want the recruiter to go through a whole line of questioning only to find out that the recruit didn’t meet the one critical requirement.
Schedule thoughtfully. Consider your target audience and the time of day they will be available. If you are targeting working professionals, evenings may be your best time for testing. If stay-at-home moms are your target, then daytime may be more convenient.
Allow plenty of time for recruiting. In general, it takes 50 names to get one recruit. This can take a couple of weeks or more to get through, with callbacks and leaving messages. You might also consider beefing up your participant incentive if you are getting close to your testing dates and haven’t filled all your seats.
Always recruit backups. If you are only including five participants, you don’t want to compromise the credibility of your study by having fewer than five if you have a no show. If you are doing a whole day of testing, I recommend recruiting two extra participants, and paying to have one present at all times, in case one of your scheduled participants doesn’t show up. It is well worth the additional incentive to pay recruits to “just sit there.”
Take care of observers. Watching study after study is interesting, but taxing. Most qualitative one-on-one studies last about an hour. This is a lot of information to take in. Make sure your observers are well fed, hydrated and comfortable. Make sure your participants are comfortable, also, so they are focused on what you are asking them about.
Limit the number of sessions. Don’t schedule more than 4 – 5 test sessions in one day. Facilitating a one hour test is highly taxing, and requires the facilitator to “think on his or her feet” based on each individual user’s actions. Make sure you allow for a break of at least a half hour between testing sessions.
Ensure consistency. Create a note taking guide that mirrors your facilitator’s guide and enables you to record each participant’s comments and actions at each step of the test. This will enable you to scan comments collectively and quickly see consistencies and inconsistencies among participant responses. Additionally, it will save you hours going back through recordings if you capture comments and quotes as the sessions are happening. If you are using a user testing software such as Morae, your notes can quickly point you to a quote you want to listen to again, vs. having to shuttle through entire recordings. Enlisting a good observer/note taker is also key in this process.
Control the environment. Make sure you choose a testing facility that can accommodate your particular needs, including type of room, equipment, times, recording, meals and snacks, as well as your recruiting needs. Recruiting is an art and it is worth it to put this in the hands of recruiting specialists.
So that gives you an idea of some of the logistics of conducting a user testing project. I my next post I will talk about the elements of an effective facilitator’s guide.
SOUND OFF: Have you done any usability testing? What did you get out of your investment?
