Honest Appraisal shows that asking Users may give Misleading Results

The recent Lyra research into using digital video recorders (DVR) reveals the shortcomings of questionnaire methodology upfront. (See UN story: Ad-Skipping is Commonplace using Digital Video Recorder, finds Survey for their findings on user behaviour.)

Quoting from the report, where the Lyra authors discuss what happened when respondents were asked if their DVRs were capable of recording HDTV (high-definition telly): 'First, 22 percent of standalone TiVo users said they had an HD-capable DVR, despite the fact that at the time this survey was conducted, licensed TiVo manufacturers had not yet shipped any such devices. (HD-capable DirecTV satellite units with built-in TiVo had just begun to ship during the survey period.)

'A few of these respondents may have owned multiple DVRs, although the survey directed such respondents to answer all the questions regarding the primary unit they use. Therefore, we must conclude that many of the respondents answered the HD-capable DVR question inaccurately.

'This conclusion leads us to question whether the responses to the HDTV question also may have been inaccurate, although the smaller proportion of "Don’t know" responses makes that seem less likely.

'Second, 47 percent of all the female respondents compared with only 20 percent of all male respondents said they did not know if their DVR was HD-capable. Therefore, either the men were more likely to know, or they were less likely to admit they did not know.

'Third, of those who said they owned an HDTV set, only 15 percent said they did not know if their DVR was HD-capable, compared with 38 percent of the non- HDTV users. Most likely, those who had an HDTV were more sensitive to whether or not their DVR was HD-capable.'

It's good to see some honest analysis. But it does reinforce one's doubts about self-administered questionnaires - users do not necessarily know what they are talking about.

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.