Top Six Don’ts for Usability Testing

About a year ago, Patrick Sullivan, Jr. of Edit Weapon, decided to give UserTesting.com his own personal test and blog review. During that same post, Patrick mentioned that there were a few golden rules and guidelines to tasking users so that their test results would be optimally useful and easy to interpret, but that he’d have to cover these in a follow-up post. So here they are, the Top 6 User Testing Tips as disclosed by Edit Weapon:

1) Never ask, “What do you think about this?”
First of all, most people will simply give you a polite, rather than bluntly honest answer. Second of all, you’re not really interested in what they think of an interface /Web design /piece of software; you’re interested in how well and how easily they can USE it. That’s why it’s called usability testing. So you always want to put the question in the form of a goal /task. Tell the user what they want to do with the interface /software; give them an assigned scenario. This transforms the process into an objective exercise (rather than a subjective opinion) and allows you to watch how the testers go about using your tool. You can then get a much better idea of how easy or intuitive your interface is, where the friction occurs, etc.

2) Don’t feed the tester your question
As people learn new things they tend to be very literal - especially when it comes to tasking. For example, if you ask a tester to ‘Compose an email’ and the button for writing a new e-mail is actually labeled ‘compose email’, the tester will simply match the phrases up rather than thinking organically in terms of what they’re trying to accomplish and then figuring out the interface. This is leading the tester by feeding information with your questions. Ensure that you ask questions using terms that are not directly on the interface labels. Use synonyms. Going a step further, if most users won’t think of a task in terms of multiple steps, but your interface requires multiple steps, don’t break your tasking down into steps to match the interface. Write the question or task in the way that most users would think of it within a given scenario.

3) Don’t let users be the designers
When you get goal-oriented tasks, each user will have different levels at which they learn the interface and pick it up, and some users will do crazy things. So some users will offer suggestions. Don’t take those suggestions literally or at face value. You’re looking for what users DO more than what they say. This is similar to the rule against not asking users what they think of an interface.

4) Don’t let the statistics fool you
Let’s say you’ve done 20 tests in a row and 5 out of 20 were failures. But say you’ve been working on the interface and creating new iterations, and the last 5 tests went extremely smoothly. In that case you’ve got a good design. You need to think of the results in terms of being 5 for 5 rather than 15 for 20. This also applies to individual tasks within a test. If users find some minor tasks more difficult than operating commonly-used features, don’t let those usability problems count anywhere near as much as your successes with the main functions of your interface. Basically, not everything can be a big red button in the middle of the screen. You have to balance things out and sometimes a few items are a bit more difficult to find. There’s really no perfect solution for a multiple use interface.

5) Don’t get discouraged
Your expectations are usually going to be high prior to the first test. You’ll wonder how people aren’t seeing what they are supposed to see. User testing can be a humbling experience. That’s why it’s usually best to test and tweak your interface in iterations. You can’t design perfectly from the get-go because as the designer, you are too much inside the bottle. But as you alternate insight generated from testing with new and improved interface iterations, you’ll find the magic if you’re willing to hang in there.

6) Don’t try to test too much at once
You’ll get easier to analyze results if you limit your tasks to just 2-3. And at UserTesting.com’s prices, it’s not a big deal if you end up running additional tests instead of adding more tasks to the same test.

Associated Link: Post in full at the FutureNow blog

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.