CHI2003: Benchmark Yourself Against the Experts

Many usability professionals have acquired considerable experience through interviews, observation and user tests. They often find, however, that their enthusiasm, experience and in-depth knowledge of usability tools and techniques is not enough to ensure success in the usability area. To ensure success, solid knowledge about specific, practical and efficient actions that can be used to promote usability in a commercial or public organisation is needed, says Rolf Molich in his rationale for offering a chance for experts to self-assess themselves with their peers at CHI panel, "The Politics of Usability".

'It is our experience that many usability professionals think they have this knowledge while actual testing shows that they haven't. Discussions we have had at usability meetings and conferences also indicate that new and experienced professionals alike are looking for insights about how well they are doing, and what they can do to improve.'

Molich and co-presenters Ron Perkins of Design Perspectives, Deborah J. Mayhew of Deborah J. Mayhew & Associates, and Kara Pernice Coyne at Nielsen Norman Group used a CHI panel as a chance to help people understand their limitations by setting up this self-assessment. In doing so, they hoped to support people as they appraised and developed their knowledge about a particular usability topic, thinking through the concepts and deciding whether to pursue the matter further. They used a usability business case, giving a hypothetical but realistic scenario that serves to illustrate points of importance to usability professionals.

'Solutions to usability issues can be applied correctly or misguided in different contexts; technical, organizational, political, business among others. It is this multidimensional nature along with concrete examples that makes the case study a good vehicle for teaching, exploring, and measuring ability,' argues Molich. 'If this model is successful, it could be an effective means to evaluate, teach, and share ideas among usability professionals.'

The panel based the case studies on Harvard Business case studies they looked at to include: presenting a real-life scenario for a usability professional; including various problems that the reader has to recognise, and eventually solve; and finally, allowing the reader to do most of the thinking and work.

The goals for the panel were to:
* show a practical example of a usability business case.
* demonstrate how a usability business case can be used for self-assessment.
* discuss the politics of usability - specific, practical actions that any usability champion should be aware of to promote usability in a commercial or public organization.
* allow participants to assess their strengths and weaknesses in the area of usability politics.

At the door as the panel started, student volunteers gave each member of the audience a self-instructing paper copy of the business case so they could start studying the usability business case immediately. After introductions to the panel and the way of working, the audience was given time to consider their suggestions and discuss them with neighbours.

Then each panellist presented part of their suggested solution, allowing other panellists and members of the audience to interject when they seriously disagreed. During this discussion period, student volunteers distributed paper copies of the suggested solution. Each member of the audience could then check both the suggestions that they had found, the suggestions they hadn't found but agreed with, and the suggestions they disagreed with.

Following discussion, there was a voluntary survey asking how many suggestions each member of the audience found. The exercise had been previously undertaken by members of usability discussion groups, allowing the panellists to prepare a survey of the results to indicate the current knowledge level in the area and let people compare themselves with the survey results.

The Business Case Scenario used:

Mary Smith has been working as a usability manager at the headquarters for MassLink for about nine months now.

MassLink is an Internet and wireless service provider that mainly serves the North-Eastern US states. The company headquarters are in Cambridge, a suburb to Boston, Massachusetts, USA. For historical reasons about half of the development staff is located in Buffalo (New York State), about 500 miles west of Boston. MassLink employs approximately 1,000 people of which 150 are developers.

Mary has a BS in Computer Science from the University of Illinois and an MBA from the University of Chicago. Before joining MassLink Mary worked for four years as a software engineer and project manager for a medical company.

During her time at MassLink Mary has carried out a number of usability inspections and a "think aloud" usability test. Mary feels that the usability inspections have been somewhat successful. Some of the teams have followed her advice carefully and a few have asked her to come back and provide further usability advice. Some teams have invited her to drop in at informal team meetings and talk about usability for 20-40 minutes.

About three months ago, a product manager on the Boston-based MassMail product development team, a highly respected team responsible for MassLinks key e-mail product, asked Mary to do a usability test of their product. Mary conducted the think aloud usability test herself, shortly before the product was released. Since MassLink has no usability lab, the test was conducted in a rented lab about 20 miles from the MassLink Boston office. No team members observed because they were too busy developing the next release of MassMail.

Mary finished the report about a week after the testing. It highlighted a number of usability issues that Mary thought were important enough to justify changes. One recommendation, about the address book feature, recommended a complete redesign because 8 of 10 study participants could not find an addressee that actually existed in the book. The team was hostile to the usability report. A few people sent sarcastic emails to Mary and no changes were made to the interface.

Apart from this setback, Mary feels reasonably successful. During recent months the requests for her services have increased, but they continue to come from only from four or five of the approximately twenty project teams.

The marketing people that Mary have spoken to say that MassLink's biggest competitor is the somewhat larger MaineLink. Mary has taken a look at MaineLink's offerings and she was not impressed by their usability.

Sheila, the CEO of MassLink, is very interested in usability. Sheila has taken some personal interest in Mary and her work. Three months ago Mary was allowed to hire a colleague.

Tomorrow Mary will have her monthly two-hour talk with Gus, the VP of Research and Development. Gus is skeptical towards usability, but not hostile. Sheila and Gus get along very well and have done so for many years despite (or, perhaps, because of) their different backgrounds. Mary has learned that Gus, who has an engineering background, can be influenced by indisputable facts and logic reasoning.

Mary feels that she is repeating herself too often with basic usability advice. She feels that the product development teams that she serves know too little about usability. She therefore wants to educate her colleagues and at the same time put more focus on usability in the development phase.

Mary therefore intends to propose to Gus a compulsory one-day seminar for all programmers, designers and marketing people at the company, a total of about 250 people. People from the Buffalo office will be flown into Boston for the seminar.

Mary envisages the following contents of the seminar:
* Introduction to usability. The human sensory system (one hour).
* Examples of good and bad usability taken from Don Norman's book "The design of everyday things" (two hours).
* Guidelines for usable design (one hour).
* A usability inspection of an e-commerce website unrelated to MassLink based on the guidelines (two hours).
* A brief overview of other usability techniques, such as "think-aloud" testing, interviews and field studies (half hour).

In order not to take too much time from her other activities, Mary is planning that the seminar will be taught by a Ph.D. student from a local university, who is currently working on a usability research project at MassLink. Mary knows that this student is a reasonable teacher. Mary will supervise the development of the seminar and sit in on the first seminar to hear how her "users" are responding.

The Assignment:
For her meeting with the VP tomorrow, Mary wants you to review her plans for the seminar. She's also interested in some frank, general advice on her situation in general.
* Comment on Mary's plans for the seminar.
* Provide five to fifteen prioritised suggestions that could help Mary improve the seminar.
* Point out any suggestions from Mary that you like.
* Comment on Mary's situation in general.
* Provide the five most important suggestions that could help Mary improve her situation.
* Assume that you are unable to get answers from Mary to any questions you might have before tomorrow. You must provide your advice based solely on the scenario, but you are welcome to include the questions you would have liked to ask Mary.
* Be brief, specific and constructive.
* Although this business case is inspired by a number of real-world cases, many details have been altered so any similarity to real companies, people and events is purely coincidental.

Why not test yourself by following the instructions given above and next week, UN will post the original suggestions made by the panellists at CHI.

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.