I have been to a few events such as RubyConf Australia and RoRo Sydney recently and have found myself explaining what I do to various people in the industry, such as developers, project managers, community managers, so on. One specific topic that kept coming up was usability testing. Most people have heard about it, or know exactly what it is and why it’s great, but are not sure about how the whole process works. First of all, I wholeheartedly recommend Steve Krug’s “Rocket Surgery Made Easy” book. The guy is a genius and gives an excellent explanation of the do’s and don’ts of usability testing. Secondly, I would like to raise a few points. There are various methods and types of usability testing: remote, on-site, moderated, unmoderated, individual, group. Each have their advantages and are used for obtaining different kinds of data. The one I do most of, by far, is the on-site, moderated usability test with individuals, so this is the one I’ll be focusing on.
What hardware do you need to conduct this kind of usability testing?
Usability labs have specialised software, eye-tracking and recording equipment that allows you to capture the participants’ eye movements on the screen as well as their facial expressions while interacting with the tested product. This, of course, is expensive. For low-budget options, any computer or mobile device will do, as long as it’s at a desk in a quiet room and has the screen size that your app was designed for. If you are testing more then one version, for instance, a mobile and a desktop view of the same app, I recommend doing two separate tests, with the right kind of device for each.
At what stage of the project should you test?
“Test early, test often” is the general mantra in the UX community. Contrary to popular belief, you don’t need a perfectly functional, polished product to conduct usability testing. You can test ideas and concepts created with programs such as Balsamiq, iRise, Axure, Pencil, or even Visio. It helps if they are interactive, but it’s not absolutely necessary for them to be polished-looking. Sometimes it is even better if they aren’t coloured and polished, because that way participants can focus on giving feedback on the information hierarchy and layout without being influenced by their preference or aversion to certain colours and styles. Working with rapid prototyping software allows you to do A/B tests as well, getting feedback on two variations of the same concept or two different concepts altogether.
How many people should you test with and where do you find them?
If you approach a UX research lab, they will use their database to find the ideal candidates depending on your target market. These people will be of the appropriate age range, occupation, income, gender, so on - whatever criteria you specify.
Short of this, you can always try to recruit friends and family, or even colleagues from within the business as long as they’ve never worked on the particular product you’re testing. I would say the number of test participants should be around 6-10, but this depends on how many people you can recruit and what exactly you’re testing.
What is the moderator’s role?
As mentioned before, I’m discussing moderated testing here, which implies the presence of… well, you guessed, a moderator. The moderator is crucial to making this type of test a successful one. It is their role to put the participant at ease at the beginning of the test: many people get nervous as soon as the word “test” is mentioned! They are also the one presenting the scenarios as per the usability test script, ensuring that all participants are walked through the same scenarios in the same way. This consistency is important to ensure that the data generated allows for comparison to generate meaningful insights. The moderator, ideally a trained and experienced UX professional, needs to be very careful not to ask leading questions and not influence the test outcome by giving hints or expressing their own preferences throughout the testing session. The moderator is there to facilitate the process, not to influence the outcome.
Okay, so what’s the “test script”?
A test script is a short document that contains a few questions and scenarios that is presented to the participant. They should be written to allow testing of key issues as realistically as possible. For instance if you are testing a rental car hire site, you would write something like: “Imagine you are flying into Adelaide and want to go on a Northern trip all the way to Darwin by car. You open this website… what do you think?” instead of “Where do you click to rent a car?”. Having scenarios helps the participants act as naturally as possible and will give you better insight than asking too specific questions.
What do you do with the data?
This type of test is used to gather qualitative data (answering the questions starting with “why…”), so it’s not easily quantifiable. A good way to do this is to add the data to a spreadsheet so that answers to each question can be seen side-by-side to a database and try to discover common issues / positive aspects. Once this is done, your team can use it to decide on what gets implemented and what doesn’t.
Usability tests are an extremely powerful tool which I recommend to anyone looking for direction when designing new software or trying to improve an existing application. It is best done often and done properly by a professional team, but it is worth attempting to run your own testing using the methods described above if your budget is limited. Relying on data rather than assumptions is the cornerstone of User-Centered Design (UCD) and the key to minimising the risk of failure with a new idea.
