EUPA/HCI2002: Remote Evaluation gets Evaluated

Andy Smith of the University of Luton and director of optimum.web gives his perspective on the "Remote Evaluation" track at the HCI2002/EUPA conference:

The session focusing on Remote Evaluation was chaired by Martin Maguire as part of the EUPA track. There were two speakers: Ron Perkins had 30 minutes to address 'Remote usability evaluations using the Internet' whereas Gaby Wenneker had the luxury of a whole hour to talk about 'improving efficiency: conducting walkthroughs remotely'.

At the outset Ron Perkins from Design Perspectives in the US claimed not to be an expert in the subject of remote evaluation. By the end of his presentation I think it was fair to say that Ron was being rather modest. He provided a classification of remote evaluation methods (real time attended vs. instrumental vs. automated) and by discussing some examples of remote evaluation projects identified the pros and cons' of the various techniques. However I felt that there was limited comparative evaluation of the techniques. Which methods are best for which type of project and why?

Gaby Wenneker is based with Zuhlke Engineering in Germany. It was during her session that we found out the difference between HCI and EUPA sessions. Just as we were settling down about 20 minutes into her talk concerning a methodology for remote evaluation that could be implemented over the telephone, we suddenly found out that we had to do some work! The concept of the audience actually doing some evaluation work was certainly novel. In an HCI event, Gaby was being participative and user centred - bizarre or what?

In groups of two we were presented screen dumps from a web site and copies of an evaluation instrument. Sitting back to back (to mirror the remote concept) one member of the pair played the user role and the other the evaluator. For me the whole concept of working during a presentation was so alien that it took most of the available time to sort out what I was meant to do. However I just had time (thanks to the patience of my co-worker) to sort out the details of the approach.

Overall the session provided a fairly rich cultural mix - the session was held in the UK with speakers from USA and Germany. Given this, and the fact that the session following this one was provided by Aaron Marcus, I felt that a key issue that needed further exploration was the suitability of different remote evaluation methods for different countries / cultures. Gaby's method seemed to me to be highly structured, which may be OK for German usability professionals and their users, but how would it work in the UK? Remote evaluation methods are particularly necessary for international systems and the suitability of methods to address cross-cultural differences is important.

Overall I felt the sessions to be informative (particularly for usability folk new to the concept of remote evaluation) rather than innovative (in that I did not feel that key novel techniques were being disseminated). However, it was good fun!

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.