CHI2003: Benchmark Yourself Against the Experts

Many usability professionals have acquired considerable experience through interviews, observation and user tests. They often find, however, that their enthusiasm, experience and in-depth knowledge of usability tools and techniques is not enough to ensure success in the usability area. To ensure success, solid knowledge about specific, practical and efficient actions that can be used to promote usability in a commercial or public organisation is needed, says Rolf Molich in his rationale for offering a chance for experts to self-assess themselves with their peers at CHI panel, "The Politics of Usability".

'It is our experience that many usability professionals think they have this knowledge while actual testing shows that they haven't. Discussions we have had at usability meetings and conferences also indicate that new and experienced professionals alike are looking for insights about how well they are doing, and what they can do to improve.'

Molich and co-presenters Ron Perkins of Design Perspectives, Deborah J. Mayhew of Deborah J. Mayhew & Associates, and Kara Pernice Coyne at Nielsen Norman Group used a CHI panel as a chance to help people understand their limitations by setting up this self-assessment. In doing so, they hoped to support people as they appraised and developed their knowledge about a particular usability topic, thinking through the concepts and deciding whether to pursue the matter further. They used a usability business case, giving a hypothetical but realistic scenario that serves to illustrate points of importance to usability professionals.

'Solutions to usability issues can be applied correctly or misguided in different contexts; technical, organizational, political, business among others. It is this multidimensional nature along with concrete examples that makes the case study a good vehicle for teaching, exploring, and measuring ability,' argues Molich. 'If this model is successful, it could be an effective means to evaluate, teach, and share ideas among usability professionals.'

The panel based the case studies on Harvard Business case studies they looked at to include: presenting a real-life scenario for a usability professional; including various problems that the reader has to recognise, and eventually solve; and finally, allowing the reader to do most of the thinking and work.

The goals for the panel were to:
* show a practical example of a usability business case.
* demonstrate how a usability business case can be used for self-assessment.
* discuss the politics of usability - specific, practical actions that any usability champion should be aware of to promote usability in a commercial or public organization.
* allow participants to assess their strengths and weaknesses in the area of usability politics.

At the door as the panel started, student volunteers gave each member of the audience a self-instructing paper copy of the business case so they could start studying the usability business case immediately. After introductions to the panel and the way of working, the audience was given time to consider their suggestions and discuss them with neighbours.

Then each panellist presented part of their suggested solution, allowing other panellists and members of the audience to interject when they seriously disagreed. During this discussion period, student volunteers distributed paper copies of the suggested solution. Each member of the audience could then check both the suggestions that they had found, the suggestions they hadn't found but agreed with, and the suggestions they disagreed with.

Following discussion, there was a voluntary survey asking how many suggestions each member of the audience found. The exercise had been previously undertaken by members of usability discussion groups, allowing the panellists to prepare a survey of the results to indicate the current knowledge level in the area and let people compare themselves with the survey results.

The Business Case Scenario used:

Mary Smith has been working as a usability manager at the headquarters for MassLink for about nine months now.

MassLink is an Internet and wireless service provider that mainly serves the North-Eastern US states. The company headquarters are in Cambridge, a suburb to Boston, Massachusetts, USA. For historical reasons about half of the development staff is located in Buffalo (New York State), about 500 miles west of Boston. MassLink employs approximately 1,000 people of which 150 are developers.

Mary has a BS in Computer Science from the University of Illinois and an MBA from the University of Chicago. Before joining MassLink Mary worked for four years as a software engineer and project manager for a medical company.

During her time at MassLink Mary has carried out a number of usability inspections and a "think aloud" usability test. Mary feels that the usability inspections have been somewhat successful. Some of the teams have followed her advice carefully and a few have asked her to come back and provide further usability advice. Some teams have invited her to drop in at informal team meetings and talk about usability for 20-40 minutes.

About three months ago, a product manager on the Boston-based MassMail product development team, a highly respected team responsible for MassLinks key e-mail product, asked Mary to do a usability test of their product. Mary conducted the think aloud usability test herself, shortly before the product was released. Since MassLink has no usability lab, the test was conducted in a rented lab about 20 miles from the MassLink Boston office. No team members observed because they were too busy developing the next release of MassMail.

Mary finished the report about a week after the testing. It highlighted a number of usability issues that Mary thought were important enough to justify changes. One recommendation, about the address book feature, recommended a complete redesign because 8 of 10 study participants could not find an addressee that actually existed in the book. The team was hostile to the usability report. A few people sent sarcastic emails to Mary and no changes were made to the interface.

Apart from this setback, Mary feels reasonably successful. During recent months the requests for her services have increased, but they continue to come from only from four or five of the approximately twenty project teams.

The marketing people that Mary have spoken to say that MassLink's biggest competitor is the somewhat larger MaineLink. Mary has taken a look at MaineLink's offerings and she was not impressed by their usability.

Sheila, the CEO of MassLink, is very interested in usability. Sheila has taken some personal interest in Mary and her work. Three months ago Mary was allowed to hire a colleague.

Tomorrow Mary will have her monthly two-hour talk with Gus, the VP of Research and Development. Gus is skeptical towards usability, but not hostile. Sheila and Gus get along very well and have done so for many years despite (or, perhaps, because of) their different backgrounds. Mary has learned that Gus, who has an engineering background, can be influenced by indisputable facts and logic reasoning.

Mary feels that she is repeating herself too often with basic usability advice. She feels that the product development teams that she serves know too little about usability. She therefore wants to educate her colleagues and at the same time put more focus on usability in the development phase.

Mary therefore intends to propose to Gus a compulsory one-day seminar for all programmers, designers and marketing people at the company, a total of about 250 people. People from the Buffalo office will be flown into Boston for the seminar.

Mary envisages the following contents of the seminar:
* Introduction to usability. The human sensory system (one hour).
* Examples of good and bad usability taken from Don Norman's book "The design of everyday things" (two hours).
* Guidelines for usable design (one hour).
* A usability inspection of an e-commerce website unrelated to MassLink based on the guidelines (two hours).
* A brief overview of other usability techniques, such as "think-aloud" testing, interviews and field studies (half hour).

In order not to take too much time from her other activities, Mary is planning that the seminar will be taught by a Ph.D. student from a local university, who is currently working on a usability research project at MassLink. Mary knows that this student is a reasonable teacher. Mary will supervise the development of the seminar and sit in on the first seminar to hear how her "users" are responding.

The Assignment:
For her meeting with the VP tomorrow, Mary wants you to review her plans for the seminar. She's also interested in some frank, general advice on her situation in general.
* Comment on Mary's plans for the seminar.
* Provide five to fifteen prioritised suggestions that could help Mary improve the seminar.
* Point out any suggestions from Mary that you like.
* Comment on Mary's situation in general.
* Provide the five most important suggestions that could help Mary improve her situation.
* Assume that you are unable to get answers from Mary to any questions you might have before tomorrow. You must provide your advice based solely on the scenario, but you are welcome to include the questions you would have liked to ask Mary.
* Be brief, specific and constructive.
* Although this business case is inspired by a number of real-world cases, many details have been altered so any similarity to real companies, people and events is purely coincidental.

Why not test yourself by following the instructions given above and next week, UN will post the original suggestions made by the panellists at CHI.

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.