Andrew's Usability in the Real World: Hard Nose in a Soft World

Every profession has its unexpected side effects – both good and bad. For example, just about the best biceps I've ever seen belonged to the middle-aged women who ran a superb ice cream parlour called The Banana Tree. All that scooping did wonders for the muscle tone. But then again, it also wrecked their wrists.

In the usability world we sometimes suffer unexpected side effects too, but it is our emotional 'musculature' that tends to get overused rather than (alas) our biceps. After spending all day in user labs asking what people think of this or that interface, we seem to spend our evenings worrying about what our colleagues and clients think of our profession. It's like we have hypertrophic empathy organs.

How many books and papers have been published on cost-justifying usability?

If you go to Google and type in "cost justifying accounting" (with the quotes) you get zero hits. If you type "cost justifying management consultants" it’s also zero. Same with estate agents. Same with CEOs, telephone sanitizers, human resources, and quality assurance. Even if you try for 'cost justifying Enron management', you get zero hits.

But if you search for "cost justifying usability", you get over 1,700 hits. Does it start to look like we might be overly obsessed with this question? Could it be that we are the only profession that requires constant questioning of our own worth? I think not.

The only other profession I could think of that yielded any hits at all in my Google test was marketing, and even that gave just one single solitary hit. Marketing/advertising makes for an interesting comparison with usability: both are considered soft sciences, and both are often questioned about the value they add to the bottom line.

There’s a famous quote about advertising, one that advertising professionals tell about themselves: "Half the money I spend on advertising is wasted; the trouble is I don't know which half." This quote, attributed sometimes to John Wanamaker and sometimes to Hesketh Lever, comes close to capturing the frustration that usability people feel about making financial arguments to defend their position. On almost every project we do, we see how our usability techniques uncover critical design issues, help prevent fundamental errors in propositions, and bring a unified vision to fractious teams, but still we feel we must justify ourselves with a definitive cost-benefit calculation.

This flailing for justification can lead to some pretty outrageous claims. Daniel Rosenberg, a vice president at Oracle, gives a talk on "The Seven Myths of Usability ROI" (see the UN story on the Seven Myths) that summarises some of the misguided ways that people justify usability, and that inadvertently undermine the profession's credibility. He points out that many of the studies are based on poor reasoning – he makes special fun of Nielsen's analysis of abandoned shopping carts on the Web – or fail to take account of confounding variables, such as studies that claim increased revenue is due to usability improvements when the same version may have many other confounding factors. He also points out that the same few studies tend to be trotted out time and again, amplifying the problem.

Rosenberg's analysis is useful as a tonic for flushing away poorly conceived justifications. But what would be more useful still is to re-focus our arguments away from vast generalisations, and toward discussing specific research for specific goals.

To see how absurd the current argument is, consider that we are arguing the benefits of toasters in the same poorly considered manner. Proponents might point out that toasters are not only good for browning bread at breakfast time, but also for heating up bagels, crisping pop tarts, and warming up your hands if the central heating stops working. Everyone, these toaster fanatics say, would be happier if only they used their toasters more, and the more toasters people have, the happier a place the world would be. Meanwhile, toaster sceptics would point out that toasters are no good for boiling tea, reading in the bathtub, or keeping beer cool. Don't bother with toasters, they say. They're just a waste of money.

Meanwhile, the level-headed toaster crowd knows that while toasters might not solve all the problems of the world, it has earned its place in the kitchen, and it might have more purpose than generally given credit for.

So should it be with usability. We don't need to argue for the cost-justification of usability in general, but for specific projects. When you do this, the arguments diminish greatly, and a space for commonsense opens up.

Here's one example: back in the 1980s when many people didn’t know how to use a mouse, Apple shipped an award-winning online tutorial that showed people how to click and double-click. This tutorial was shipped on a floppy disk. After a while, there was concern that the floppies were too expensive, and a product manager asked if we couldn’t just put the tutorial on the hard disk instead. The tutorial designer complained: how could users launch the mouse tutorial on the hard disk if they didn't know how to use the mouse yet? A reasonable and logical objection, and one that was easily settled by a usability test. As it happened, usability testing demonstrated a way to make the tutorial work without shipping the floppy, and the savings over the years have been in the millions.

There are countless other examples. Usability research helped reduce the time per call in a call centre; it ensured the take up of a new generation of digital TV box would avoid the customer churn previous changeovers had done; it prevented development of a publishing product it turned out that no one wanted.

If you are looking for proof that every kind of usability research everywhere is cost effective, such proof doesn't exist, and it will never exist. Some usability research is meaningless, unfocused, and a waste of money. This tends to happen when it is planned without taking into account business goals or organisational objectives.

But this is the exception rather than the rule. When you target your research at specific problems in the real world, it can hardly fail to win real benefits, and if the research (and the insight behind it) are well conceived, the returns can be many times the investment.

Andrew Swartz
Principal Consultant
Serco Usability Services
www.usability.serco.com
020.7421.6499

This article would not have been possible without the insights of my colleagues Owen Daly-Jones and Simon Herd, for which I am grateful.

© 2004 Serco Limited, All Rights Reserved.

Associated Link: Serco Usability Services

User Interviews - Analysis Simplified
Source: Webcredible, 6 January 2009
 
You’ve conducted your user interviews, but now you need to make sense of all that information you’ve gathered.

Why Products Fail
Source: ComputerWorld, 5 January 2009
 
Most gadget and software makers don't understand what users want most: control.

How to Design Websites for Mobile Phones
Source: stemkoski.com, 3 January 2009
 
Tips from Ryan Stemkoski's web design blog.

Pioneer of Cyberspace honoured
Source: BBC, 2 January 2009
 
A professor who invented a forerunner of the world wide web has been made a dame in the New Year Honours.

2008 in Review: Developments that rocked the world of User Experience
Source: Catalyst Resources, 1 January 2009
 
A look back at 2008 highlights some of the key developments that rocked the world of user experience.

2008: The Year Online
Source: MIT Technology Review, 31 December 2008
 
The business of social networking, cloud computing, and a flaw in the fabric of the Internet top the most notable stories of 2008.

Shoveling through the Spamalanche
Source: UN, 30 December 2008
 
A ‘Spamalanche’ of 3,000 emails will be waiting in your inbox by the time you get back to work. What can you do besides 'delete all'?

New guidelines boost Web Access
Source: BBC, 29 December 2008
 
The World Wide Web Consortium (W3C) has announced a new standard to make sites more accessible to older and disabled people.

Measuring Emotional Engagement
Source: UN, 27 December 2008
 
New applications of electroencephalography (EEG) provide the means to gather detailed information on a user’s emotional relationship with a brand or service.

UXalliance thrives on Research strengths
Source: UN, 24 December 2008
 
The UXalliance provides clients with access to over 220 user experience professionals and 40 test labs worldwide.