CHI 2003: Don't Go to a Panel Like This Again!

User testing is a very narrow approach and it is becoming increasingly irrelevant to industry, said Dennis Wixon, usability manager at MSN, Microsoft. He was speaking on the CHI 2003 panel: "The Magic Number 5: Is it Enough for Web Testing?". He criticised the method for lack of ecological validity and welcomed alternatives like ethnography. 'Stop looking at statistics and examine the relationship of lab to field - that is a useful discussion to have,' he told the audience. 'And don't go to a panel like this one again!'

The panel was discussing the relevance and accuracy of Nielsen and Landauer's old formula that five users is enough to find a majority of usability problems with a product. Wixon was not alone in viewing this as the wrong discussion to be having. (Read a more detailed account of the debate from Larry Constantine, also on UN today at Testing... 1 2 3 4 5 ... Testing....) He implied that rather than focussing on identifying problems, a solution centred approach would be more useful in improving products.

Other areas that speakers highlighted as in need of consideration included:
* frequency of testing: few often is better than few users once;
* inter-facilitator reliability;
* frequency and severity of problems;
* the underlying statistical flaw that accompanies the methodological flaw in the five user study;
* the benefit of letting users choose their own tasks in attempting to find all problems/types of problem;
* the limits of automated testing.

Despite Wixon's admonitions, members of the audience commented that it was the best panel of this kind they had experienced.

Other speakers were Nigel Bevan of Serco (chair), Carol Barnum of Southern Polytechnic State University, Gilbert Cockton from University of Sunderland, Rolf Molich of DialogDesign, Jared Spool of User Interface Engineering, and Dennis Wixon from Microsoft. Jakob Nielsen, though billed, was not present to defend his five user assertion, having had to pull out of CHI due to a health problem.

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.