Caroline's Corner: Estimating of Testing the Plausible Way

How to estimate the time needed for a usability test? One of my favourite lists was discussing this recently. Formulae were proposed, variables discussed, and weighting factors considered.

I was torn somewhere between a wry smile and an attack of nostalgia. The nostalgia? Because it took me back to my days as a project manager searching for better ways of estimating. Chronic underestimating by the engineers and continual shortening of timescales by management led to constant disappointments as projects overran. Surely there had to be some tools out there that would provide reliable estimates, "independent" from the usual politics?

Two of the models I found were COCOMO (now COCOMO II) and function point analysis. Great, now all I had to do was a bit of counting, hit the button and out would pop my estimate. Only of course it didn't quite work like that. The mechanical estimates weren't any better. Chatting about this at a conference, a researcher in the field of software estimating said to me 'Oh sure, a blind average of all your previous projects is more reliable as a predictor of your next project than any of these tools'. And she was right. Generally, when we do the same thing twice it takes about the same time both times. Even though we try to persuade ourselves that writing the report will surely be quicker this time...

So my first rule of estimating is not: how long should it take but how long did it take the last n times we did it? Looking forward by looking backward. Take the unadjusted average time for that task and use it again for your estimate for next time. Looks too long? Are you sure that the next time really will be easier? Why? Have you allowed for the unexpected being different this time?

And the wry smile? Well, a current project is typical. It has its plan and if we want to "do usability" we've got to figure out how to infiltrate usability activities into what else is going on. For example, testing the prototype. No question on this project of a couple of weeks for planning, running a test and writing a report. We were told 'you'll get the prototype on day x, and we want to know if it's OK by the end of the same day'. So we had to:

* accept that the development team would demonstrate the prototype to us, rather than getting raw user reactions to it sight unseen. We added realism to the demonstration by getting them to use real tasks during it.
* test with the people we could get, users' managers, rather than the users we wanted. We got them to try doing real work on the system rather than trying to recall how their staff worked.
* create a list of issues before writing the report. Actually, this made it easier and quicker to write the report because we'd already thought the issues through.

Which brings me to my second rule of estimating: work inside out from the time available to deciding on the tasks. Can you add a bit of a usability perspective to something that will happen anyway? Can you pick a usability activity that will get you the information you need in the time you have? Better to get any input than miss the chance of having usability activities on the project because we're being too purist about the methods we're using.

Caroline welcomes comments at: caroline.jarrett@effortmark.co.uk

Caroline Jarrett
Independent usability consultant specialising in forms, questionnaires and data capture.

Effortmark Ltd
w. www.effortmark.co.uk
e. caroline.jarrett@effortmark.co.uk
p. +44 (0)1525 370379

Associated Link: Effortmark

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.