Caroline's Corner: Usability Test Reports - your Chance to influence a New Standard

What do you think about international standards such as ISO 13407 or ISO 9000? Necessary but dull? Expensive but important? Irrelevant to daily life?

I'm not going to pretend that these documents are ever going to compete with Harry Potter or the Da Vinci Code for the public's attention as light reading. But a standard can be really helpful in guiding us towards appropriate professional practice (that's 'doing things properly' in plain English). I've also found that certain clients with engineering or bureaucratic tendencies respond really well to the idea that 'we're following a standard'.

CIF FOR FORMATIVE

So I'm delighted to report on progress that's being made on a standard for usability test reports, to be precise 'a common industry format for reports of formative usability tests'.

Let's deconstruct that snappy little title.

'Common industry format': the idea is that we'll have a format that is accepted and used widely by usability practitioners.

'Reports': there are lots of ways of communicating the results of testing within teams and to clients. Many of us find that getting people along to view the test is one of the most effective methods, with video highlights a close runner up. But the written report is still crucial: for communicating results, for creating a record, and for demonstrating value in what we do. And, let's be honest: many of us get paid when we've delivered the report.

'Formative usability tests': why put in the word 'formative'? Because there is already a common industry format for 'summative' usability tests. Not sure of the difference between 'formative' and 'summative'? I think of them like this: the primary purpose of a 'summative' test is to find out how usable something is. Whereas the primary purpose of a 'formative' test is to create ideas for how to make it more usable. You might actually do similar things for either type of test but the purpose is different. (Arguments on this, as with anything else I write, are always welcome: email address below).

PROGRESS TOWARDS A STANDARD

I've always been a bit wary of the common industry format for summative usability tests (CIF). It's a good format, to be sure, but my work is almost always intended to help create change, preferably in the direction of better usability. My experience has been that the clients are mostly interested in what we found and what to do about it, whereas the CIF majors on what we did and doesn't have space for recommendations.

So I was pleased when, thanks to a bit of juggling of commitments in the USA, I was able to get to workshop in Boston that's started the work on the 'CIF for formative'. Over 30 highly opinionated usability specialists succeeded in working together and achieving consensus on things like a first version of a shopping list of items that could be included in the report, possible metrics, and some rules or suggestions for how you choose which things to put in the report.

WHAT HAPPENS NEXT

Now we get to the 'your chance to influence' bit. The workshop ended with a call for volunteers who will take the work so far, formalise it, and generally beat it into shape. I'm sure that some testing of the proposed format will be in there somewhere. So if you're interested in helping, now is your chance. Write to Mary Theofanos at the National Institute for Standards and Technology: mary.theofanos@nist.gov explaining why you'd like to help, what you can do and your background in the field. Let's give her plenty of options to choose from the best in the profession.

REFERENCES

Description of ISO 13407 http://www.usabilitynet.org/tools/13407stds.htm

Purchase ISO 13407 or ISO 9000 in UK through BSI www.bsi-global.com or internationally through ISO http://www.iso.org

More information on the common industry format for summative testing http://zing.ncsl.nist.gov/iusr/

If you have any comments or suggestions about this article then please contact Caroline at:

Caroline.Jarrett@Effortmark.co.uk

Caroline Jarrett is a usability consultant specialising in forms, questionnaires and data capture.

© 2004 Caroline Jarrett, all rights reserved.

Associated Link: Effortmark

User Interviews - Analysis Simplified
Source: Webcredible, 6 January 2009
 
You’ve conducted your user interviews, but now you need to make sense of all that information you’ve gathered.

Why Products Fail
Source: ComputerWorld, 5 January 2009
 
Most gadget and software makers don't understand what users want most: control.

How to Design Websites for Mobile Phones
Source: stemkoski.com, 3 January 2009
 
Tips from Ryan Stemkoski's web design blog.

Pioneer of Cyberspace honoured
Source: BBC, 2 January 2009
 
A professor who invented a forerunner of the world wide web has been made a dame in the New Year Honours.

2008 in Review: Developments that rocked the world of User Experience
Source: Catalyst Resources, 1 January 2009
 
A look back at 2008 highlights some of the key developments that rocked the world of user experience.

2008: The Year Online
Source: MIT Technology Review, 31 December 2008
 
The business of social networking, cloud computing, and a flaw in the fabric of the Internet top the most notable stories of 2008.

Shoveling through the Spamalanche
Source: UN, 30 December 2008
 
A ‘Spamalanche’ of 3,000 emails will be waiting in your inbox by the time you get back to work. What can you do besides 'delete all'?

New guidelines boost Web Access
Source: BBC, 29 December 2008
 
The World Wide Web Consortium (W3C) has announced a new standard to make sites more accessible to older and disabled people.

Measuring Emotional Engagement
Source: UN, 27 December 2008
 
New applications of electroencephalography (EEG) provide the means to gather detailed information on a user’s emotional relationship with a brand or service.

UXalliance thrives on Research strengths
Source: UN, 24 December 2008
 
The UXalliance provides clients with access to over 220 user experience professionals and 40 test labs worldwide.