Andrew’s Usability in the Real World: Predictive Usability

We usability researchers are curious by nature, perhaps even well-intentioned voyeurs at heart. Once, a couple of years ago, a neighbour caught me skulking around our local grocery store with a digital camera in hand taking pictures of a small colony of interactive kiosks that had popped up in the entryway. Red-faced, I explained that we were studying how people used these kiosks. I’m not sure she believed me, but in any case she didn’t report me to the authorities.

You may have seen the papers we delivered on this subject, which concluded that many kiosks were worse than useless, and in many cases effectively invisible – a surprising conclusion given that some of the kiosks were twice as tall as a very tall man and plunked down in the middle of busy shopping streets.

Our ‘kiosk paper’ talk included pictures of this grocery store, and showed an electronic photo booth (unused), a rewards points kiosks (unused), a city-sponsored internet access point (unused), and two coin-operated children rides (in use). We predicted a poor future for all but the children’s rides, and explained what kiosks would need to do succeed.

Well, imagine my sense of smug satisfaction when I went in to the grocery store just after the new year and discovered that all the devices except the Thomas the Tank Engine ride had been removed. We know that our methodologies work, but it’s nice to see the evidence so starkly.

* * *

Historically, usability research was focused on identifying obstacles to success, not on assessing how intensely people will desire a new product or service. The field of usability evolved studying how military personnel interacted with weapons systems and how employees used the software their bosses required of them. The goal was to increase efficiency and decrease errors, not necessarily to attract new customers.

A bog-standard textbook lab-based usability test on the kiosks mentioned above would have been able to identify classic usability issues, perhaps that the navigation was a bit awkward or that the buttons didn’t offer enough affordance; but without clever research design, they could not have predicted that hundreds of people would pass them by, not even noticing they existed.

In the real world, what is the more important finding? Isn’t it more important to note that a product will be a hulking expensive waste of space than to propose in detail what the size of buttons on the screen should be?

The brilliant teams who designed the new products that I have worked on have all been grateful for the findings that helped them fine-tune their products, but what they appreciated more than anything was a sense of whether people would use them to begin with.

The greatest savings from a usability study come when we can show that a product is not desired. The greatest benefit comes when we help identify who will be passionate about a new product, and what is required to unlock that passion.

* * *

Here are some practical considerations for designing usability research to address not only classic usability concerns, but also to assess whether people desire the products and services.
• LIMITS OF SELF-REPORTING. Users can be poor at predicting their own behaviour. We often see participants in usability studies claiming that they would love a new product only to see the product fail in the marketplace, or expressing disdain for a new product only to see it succeed. Take what users say with a grain of salt.
• ANALYSE BASED ON CONTEXT OF STUDY. Most lab-based usability studies give us the opportunity to see users’ first 60-90 minutes interacting with a new product or service. First reactions are important, but they are not everything. Poor initial reaction means only that – that users coming cold to the product are not receptive. In practice, it could be either that the product simply cannot succeed, or it could be that users need more background to make them appreciate the product’s value. (Think what a marketing team could do with information like that.)
• CONSIDER DIFFERENT KINDS OF STUDIES. Some products look remarkably different in the lab than in the real world. Some products require long start-up periods. For example, voice recognition products require an initial training period followed by a month or so in which the user intensively corrects the software. An hour in a test lab may tell you about the obstacles in the training process, but not much about the ultimate usefulness of such a product. Other products rely on ‘network effects’. For example, one person with a video-calling phone on his own isn’t going to find it very interesting; but a large group of sports fans may come to think of it as indispensable. Design your studies accordingly – longer-term studies for products that require it, and group studies for network effects.
• BEWARE PUPPY LOVE. Some features make users fall in love at first sight, but not every love lasts. Clippy, the infamous talking paper clip in Microsoft Office products (serving as a painfully eager help system), is typically adored by naďve users the first time they see it, and desperately despised within a week or two, once users become aware how intrusive it can be. If you had conducted a standard user test before Clippy shipped, it probably would have predicted great success; but now you can type "hate Clippy" in Google to see how a user interface can inspire genuine passion.

If you design your studies well and conduct your analysis thoughtfully, you can give the product teams fantastic insight into whether anyone wants to use their products.

I’d love to hear what you think. Send me an e-mail at aswartz@usability.serco.com.

Andrew Swartz
Managing Consultant

© 2005 Serco Limited, All Rights Reserved.

Associated Link: Serco Usability Services

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.