Andrew’s Usability in the Real World: Predictive Usability

We usability researchers are curious by nature, perhaps even well-intentioned voyeurs at heart. Once, a couple of years ago, a neighbour caught me skulking around our local grocery store with a digital camera in hand taking pictures of a small colony of interactive kiosks that had popped up in the entryway. Red-faced, I explained that we were studying how people used these kiosks. I’m not sure she believed me, but in any case she didn’t report me to the authorities.

You may have seen the papers we delivered on this subject, which concluded that many kiosks were worse than useless, and in many cases effectively invisible – a surprising conclusion given that some of the kiosks were twice as tall as a very tall man and plunked down in the middle of busy shopping streets.

Our ‘kiosk paper’ talk included pictures of this grocery store, and showed an electronic photo booth (unused), a rewards points kiosks (unused), a city-sponsored internet access point (unused), and two coin-operated children rides (in use). We predicted a poor future for all but the children’s rides, and explained what kiosks would need to do succeed.

Well, imagine my sense of smug satisfaction when I went in to the grocery store just after the new year and discovered that all the devices except the Thomas the Tank Engine ride had been removed. We know that our methodologies work, but it’s nice to see the evidence so starkly.

* * *

Historically, usability research was focused on identifying obstacles to success, not on assessing how intensely people will desire a new product or service. The field of usability evolved studying how military personnel interacted with weapons systems and how employees used the software their bosses required of them. The goal was to increase efficiency and decrease errors, not necessarily to attract new customers.

A bog-standard textbook lab-based usability test on the kiosks mentioned above would have been able to identify classic usability issues, perhaps that the navigation was a bit awkward or that the buttons didn’t offer enough affordance; but without clever research design, they could not have predicted that hundreds of people would pass them by, not even noticing they existed.

In the real world, what is the more important finding? Isn’t it more important to note that a product will be a hulking expensive waste of space than to propose in detail what the size of buttons on the screen should be?

The brilliant teams who designed the new products that I have worked on have all been grateful for the findings that helped them fine-tune their products, but what they appreciated more than anything was a sense of whether people would use them to begin with.

The greatest savings from a usability study come when we can show that a product is not desired. The greatest benefit comes when we help identify who will be passionate about a new product, and what is required to unlock that passion.

* * *

Here are some practical considerations for designing usability research to address not only classic usability concerns, but also to assess whether people desire the products and services.
• LIMITS OF SELF-REPORTING. Users can be poor at predicting their own behaviour. We often see participants in usability studies claiming that they would love a new product only to see the product fail in the marketplace, or expressing disdain for a new product only to see it succeed. Take what users say with a grain of salt.
• ANALYSE BASED ON CONTEXT OF STUDY. Most lab-based usability studies give us the opportunity to see users’ first 60-90 minutes interacting with a new product or service. First reactions are important, but they are not everything. Poor initial reaction means only that – that users coming cold to the product are not receptive. In practice, it could be either that the product simply cannot succeed, or it could be that users need more background to make them appreciate the product’s value. (Think what a marketing team could do with information like that.)
• CONSIDER DIFFERENT KINDS OF STUDIES. Some products look remarkably different in the lab than in the real world. Some products require long start-up periods. For example, voice recognition products require an initial training period followed by a month or so in which the user intensively corrects the software. An hour in a test lab may tell you about the obstacles in the training process, but not much about the ultimate usefulness of such a product. Other products rely on ‘network effects’. For example, one person with a video-calling phone on his own isn’t going to find it very interesting; but a large group of sports fans may come to think of it as indispensable. Design your studies accordingly – longer-term studies for products that require it, and group studies for network effects.
• BEWARE PUPPY LOVE. Some features make users fall in love at first sight, but not every love lasts. Clippy, the infamous talking paper clip in Microsoft Office products (serving as a painfully eager help system), is typically adored by naïve users the first time they see it, and desperately despised within a week or two, once users become aware how intrusive it can be. If you had conducted a standard user test before Clippy shipped, it probably would have predicted great success; but now you can type "hate Clippy" in Google to see how a user interface can inspire genuine passion.

If you design your studies well and conduct your analysis thoughtfully, you can give the product teams fantastic insight into whether anyone wants to use their products.

I’d love to hear what you think. Send me an e-mail at aswartz@usability.serco.com.

Andrew Swartz
Managing Consultant

© 2005 Serco Limited, All Rights Reserved.

Associated Link: Serco Usability Services

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.