Caroline's Corner: Playing Piggy in the Middle

Questionnaires often ask us to rate something or other. Recently, I've been asked about:

- my satisfaction with a huge website

- the effectiveness of a selection of ways to maintain or increase charge-out rates

- the cleanliness of a hotel room.

My problem? I didn't know, didn't care or didn't remember. Sometimes all three. So I skipped the question or junked the questionnaire.

There's a lesson here for choosing the 'correct' number of points in a rating scale. (I'm assuming that you need to offer a number of discrete points rather than offering some widget that allows a continuously variable scale).

Some people take the view that you should have an even number of points. This encourages your respondent to move off the fence and plump for a positive or negative rating.

I disagree. When I've done usability tests of questionnaires, I've found that users have many reasons for choosing the middle point. As well as 'don't know', 'don't care' and 'don't remember', users have told me that they chose the middle point because:

- they didn't understand the question

- they sometimes felt positive and sometimes felt negative

- they mostly felt quite positive but occasionally felt very negative (or vice versa)

- they felt positive about some aspect but negative about another

- or, (and could this one be particularly British) they felt that it wasn't up to them to express an opinion on this item.

So I take the view that you can't force someone into a positive or negative opinion simply by taking away their middle point. They still have their opinion, and will express it in one of these ways:

- skip the question

- scrap the questionnaire

- split their votes: choose either mildly positive or mildly negative, at random, and make it stand for a middle point.

In effect, the 'split' tactic means that scales with an even number of points lose two of those points to the middle.

You can sometimes ease respondents away from clinging to the middle point if you offer a 'don't know' or 'not applicable' option. The problem with these extra options is: where to put them? If they are close to the main rating points then they look like extra points in the scale and get confused with it. If they are separated from them, they get missed.

So, it's an odd number of points for me. I've met some user resistance to 3-point scales as they want to express milder and stronger shades of opinion, so I generally opt for 5-point scales. But when I'm testing the questionnaire I hear things like "oh, I'd never give a 5" (when 5 is the best mark) or "I like to be generous so I'll tick 5" - contrasting strategies even though the expressed level of satisfaction is the same. If interrogated closely, I'll admit that when analysing data for myself I sometimes, bizarrely and irrationally, treat the points as if they expressed concrete difference and try to do manipulations like working out the average score out of 5. A temptation that I resist when working with client data. Then, I do the sensible thing and group the two positive points together for analysis and the two negative points similarly. I've noticed that market researchers often do the same, and will report 'top two boxes' together. They're not being over-optimistic, simply reflecting the lack of difference in the opinions expressed in the two boxes.

I've never tried 7- or 9-point scales myself. I've seen them used in questionnaires on psychological topics where, presumably, the experimenter has decided that the subjects really do want to express fine shades of meaning. In the more mundane world of the everyday satisfaction questionnaire, respondents' time is too valuable and any theoretical benefit from the extra points is certainly offset by the possibility that respondents might be put off by the extra work entailed and by the challenge of deciding how to group the results.

As in most things in usability "your mileage may vary". Let me know if you've got views that differ from mine.

As usual, Caroline welcomes any comments or suggestions about this article.

Caroline Jarrett
Independent usability consultant specialising in forms, questionnaires and data capture.

Effortmark Ltd
w. www.effortmark.co.uk
e. caroline.jarrett@effortmark.co.uk
p. +44 (0)1525 370379

Associated Link: Effortmark

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.