CHI2003: Benchmark Yourself Against the Experts

Many usability professionals have acquired considerable experience through interviews, observation and user tests. They often find, however, that their enthusiasm, experience and in-depth knowledge of usability tools and techniques is not enough to ensure success in the usability area. To ensure success, solid knowledge about specific, practical and efficient actions that can be used to promote usability in a commercial or public organisation is needed, says Rolf Molich in his rationale for offering a chance for experts to self-assess themselves with their peers at CHI panel, "The Politics of Usability".

'It is our experience that many usability professionals think they have this knowledge while actual testing shows that they haven't. Discussions we have had at usability meetings and conferences also indicate that new and experienced professionals alike are looking for insights about how well they are doing, and what they can do to improve.'

Molich and co-presenters Ron Perkins of Design Perspectives, Deborah J. Mayhew of Deborah J. Mayhew & Associates, and Kara Pernice Coyne at Nielsen Norman Group used a CHI panel as a chance to help people understand their limitations by setting up this self-assessment. In doing so, they hoped to support people as they appraised and developed their knowledge about a particular usability topic, thinking through the concepts and deciding whether to pursue the matter further. They used a usability business case, giving a hypothetical but realistic scenario that serves to illustrate points of importance to usability professionals.

'Solutions to usability issues can be applied correctly or misguided in different contexts; technical, organizational, political, business among others. It is this multidimensional nature along with concrete examples that makes the case study a good vehicle for teaching, exploring, and measuring ability,' argues Molich. 'If this model is successful, it could be an effective means to evaluate, teach, and share ideas among usability professionals.'

The panel based the case studies on Harvard Business case studies they looked at to include: presenting a real-life scenario for a usability professional; including various problems that the reader has to recognise, and eventually solve; and finally, allowing the reader to do most of the thinking and work.

The goals for the panel were to:
* show a practical example of a usability business case.
* demonstrate how a usability business case can be used for self-assessment.
* discuss the politics of usability - specific, practical actions that any usability champion should be aware of to promote usability in a commercial or public organization.
* allow participants to assess their strengths and weaknesses in the area of usability politics.

At the door as the panel started, student volunteers gave each member of the audience a self-instructing paper copy of the business case so they could start studying the usability business case immediately. After introductions to the panel and the way of working, the audience was given time to consider their suggestions and discuss them with neighbours.

Then each panellist presented part of their suggested solution, allowing other panellists and members of the audience to interject when they seriously disagreed. During this discussion period, student volunteers distributed paper copies of the suggested solution. Each member of the audience could then check both the suggestions that they had found, the suggestions they hadn't found but agreed with, and the suggestions they disagreed with.

Following discussion, there was a voluntary survey asking how many suggestions each member of the audience found. The exercise had been previously undertaken by members of usability discussion groups, allowing the panellists to prepare a survey of the results to indicate the current knowledge level in the area and let people compare themselves with the survey results.

The Business Case Scenario used:

Mary Smith has been working as a usability manager at the headquarters for MassLink for about nine months now.

MassLink is an Internet and wireless service provider that mainly serves the North-Eastern US states. The company headquarters are in Cambridge, a suburb to Boston, Massachusetts, USA. For historical reasons about half of the development staff is located in Buffalo (New York State), about 500 miles west of Boston. MassLink employs approximately 1,000 people of which 150 are developers.

Mary has a BS in Computer Science from the University of Illinois and an MBA from the University of Chicago. Before joining MassLink Mary worked for four years as a software engineer and project manager for a medical company.

During her time at MassLink Mary has carried out a number of usability inspections and a "think aloud" usability test. Mary feels that the usability inspections have been somewhat successful. Some of the teams have followed her advice carefully and a few have asked her to come back and provide further usability advice. Some teams have invited her to drop in at informal team meetings and talk about usability for 20-40 minutes.

About three months ago, a product manager on the Boston-based MassMail product development team, a highly respected team responsible for MassLinks key e-mail product, asked Mary to do a usability test of their product. Mary conducted the think aloud usability test herself, shortly before the product was released. Since MassLink has no usability lab, the test was conducted in a rented lab about 20 miles from the MassLink Boston office. No team members observed because they were too busy developing the next release of MassMail.

Mary finished the report about a week after the testing. It highlighted a number of usability issues that Mary thought were important enough to justify changes. One recommendation, about the address book feature, recommended a complete redesign because 8 of 10 study participants could not find an addressee that actually existed in the book. The team was hostile to the usability report. A few people sent sarcastic emails to Mary and no changes were made to the interface.

Apart from this setback, Mary feels reasonably successful. During recent months the requests for her services have increased, but they continue to come from only from four or five of the approximately twenty project teams.

The marketing people that Mary have spoken to say that MassLink's biggest competitor is the somewhat larger MaineLink. Mary has taken a look at MaineLink's offerings and she was not impressed by their usability.

Sheila, the CEO of MassLink, is very interested in usability. Sheila has taken some personal interest in Mary and her work. Three months ago Mary was allowed to hire a colleague.

Tomorrow Mary will have her monthly two-hour talk with Gus, the VP of Research and Development. Gus is skeptical towards usability, but not hostile. Sheila and Gus get along very well and have done so for many years despite (or, perhaps, because of) their different backgrounds. Mary has learned that Gus, who has an engineering background, can be influenced by indisputable facts and logic reasoning.

Mary feels that she is repeating herself too often with basic usability advice. She feels that the product development teams that she serves know too little about usability. She therefore wants to educate her colleagues and at the same time put more focus on usability in the development phase.

Mary therefore intends to propose to Gus a compulsory one-day seminar for all programmers, designers and marketing people at the company, a total of about 250 people. People from the Buffalo office will be flown into Boston for the seminar.

Mary envisages the following contents of the seminar:
* Introduction to usability. The human sensory system (one hour).
* Examples of good and bad usability taken from Don Norman's book "The design of everyday things" (two hours).
* Guidelines for usable design (one hour).
* A usability inspection of an e-commerce website unrelated to MassLink based on the guidelines (two hours).
* A brief overview of other usability techniques, such as "think-aloud" testing, interviews and field studies (half hour).

In order not to take too much time from her other activities, Mary is planning that the seminar will be taught by a Ph.D. student from a local university, who is currently working on a usability research project at MassLink. Mary knows that this student is a reasonable teacher. Mary will supervise the development of the seminar and sit in on the first seminar to hear how her "users" are responding.

The Assignment:
For her meeting with the VP tomorrow, Mary wants you to review her plans for the seminar. She's also interested in some frank, general advice on her situation in general.
* Comment on Mary's plans for the seminar.
* Provide five to fifteen prioritised suggestions that could help Mary improve the seminar.
* Point out any suggestions from Mary that you like.
* Comment on Mary's situation in general.
* Provide the five most important suggestions that could help Mary improve her situation.
* Assume that you are unable to get answers from Mary to any questions you might have before tomorrow. You must provide your advice based solely on the scenario, but you are welcome to include the questions you would have liked to ask Mary.
* Be brief, specific and constructive.
* Although this business case is inspired by a number of real-world cases, many details have been altered so any similarity to real companies, people and events is purely coincidental.

Why not test yourself by following the instructions given above and next week, UN will post the original suggestions made by the panellists at CHI.

The Hygiene Factor of Usability
Source: inspireUX, 15 March 2010
 
Is it true that usability can no longer take us 'beyond lack of dissatisfaction'?

The Business Benefits of building Accessible websites
Source: Econsultancy, 13 March 2010
 
There’s a good business case for making your website more accessible to the UK’s disabled community.

Internet access is 'a fundamental right'
Source: BBC, 12 March 2010
 
Almost four in five people around the world believe that access to the internet is a fundamental right, a poll for the BBC World Service suggests.

The Net generation, Unplugged
Source: The Economist, 11 March 2010
 
Is it really helpful to talk about a new generation of "digital natives" who have grown up with the internet?

Rewriting the Human-Computer interaction Handbook
Source: CIOL, 10 March 2010
 
Indrani Medhi of Microsoft Research India has developed text-free user interfaces (UIs) to allow any illiterate or semi-literate person on first contact with a computer, to proceed with minimal or no assistance.

Lip reading Mobile promises End to noisy phone calls
Source: BBC, 9 March 2010
 
A prototype device shown off at CeBIT could allow people to conduct silent phone conversations.

Games User Researchers band together
Source: UN, 8 March 2010
 
The number of UX professionals in gaming has reached critical mass.

Quince Pro enables privately-held UX Design Libraries
Source: Infragistics, 6 March 2010
 
Infragistics has launched Quince Pro, a private, secure and organized way to collaborate, communicate and cultivate private UX design libraries to ensure consistent user experiences across teams, departments and companies.

Announcing a new issue of the Journal of Usability Studies
Source: UPA, 5 March 2010
 
UPA is happy to announce the publication of the second issue of volume 5, the Journal of Usability Studies.

Impatient versus Bored
Source: Gerry McGovern, 4 March 2010
 
Customers are much more likely to get impatient with your website than they are to be bored with it.