Evaluating Design Effectiveness sparks Debate

Some of the recent raft of design awards appear to reward websites that exhibit serious design concerns, according to AIGA organiser Nico Macdonald. Habitat.net, which has recently been awarded a BAFTA Interactive Entertainment Award, is billed by BAFTA as 'a true replacement for the showroom', but in fact is frustrating to use, he says.

So how are we to evaluate designs? How can we talk about why a website works and why it doesn't? Can we talk about what's good and what's bad? And how does a designer convince someone to hire them based on their history of design effectiveness?

Macdonald drew together a wide range of viewpoints to discuss these issues and others at the Design Council, with a designer, a usability academic and writer, a lecturer in graphic communication and a design consultancy practitioner on the panel.

Awards such as the BAFTA award to the Habitat site were unhelpful and even counterproductive for those teaching design, according to Gerry Leonidas, a lecturer in graphic communication at Reading University.

'These kind of awards teach students that design is something other than what it is. Then we have to undo that message,' argued Leonidas.

He argued that one of the problems facing designers and their clients is nomenclature: there are no clear demarcations of expertise, with self-described design consultants, graphic designers and interaction designers all swimming around in the same design pond. To make matters worse, designers do not themselves use a common language, and concepts are not uniformly understood, so there is little basis for expressing evaluations. He said that graduates from his courses sometimes had a hard time explaining to clients what exactly it is they do and the value being added.

In contrast, designer Jack Schulze claimed that we don't really need tools to tell us if design is any good. But one problem Schulze pointed to was the practice of design agencies 'borrowing the language of design to support business decisions'. The graphic language of the Bauhaus is bandied about without any of the original principles being applied, for example.

People will use things despite poor design, Schulze argued. 'Often things that work on the web are not "designed" at all,' claimed Schulze, pointing to the success of eBay, Hotmail and Amazon. Schulze perceived poor usability to be an engineering issue, not a design problem. Although he conceded that both functions were on occasion the responsibility of the designer: 'I would like to believe that as a designer that is not my role'.

Perhaps this perspective arose out of a narrow view of the meaning of usability, a merely mechanical application of deterministic criteria. The apex of usability, according to Schulze, would be that 'the same product would emerge from two agencies' given the same brief, whereas an indication of good design is that 'you can see something of the designer in the product'.

Schulze saw little sign of consumers picking up the usability agenda. Continued poor usability had not halted the sales of that most notorious of badly designed consumer goods, the video recorder, he argued, with most consumers continuing to choose recorders based on functions offered. The grounds for improving such designs are not in place if consumers continue to buy these products, he said.

Suddenly finding herself described as involved in engineering, Usability News editor Ann Light was quick to return the goalposts to their former position: 'I've come to talk about Design, and what Jack's been talking about is Art,' said Light, who is a judge in the Brighton and Hove Virtual Web Awards as well as being an academic researcher.

While evaluation of websites is very important to their owners, Light believes that it is more important that designers evaluate their own work than for outsiders like herself to do so.

Light argued that breaking down design into its component parts is essential if we are to learn from experience. 'The moment you want to repeat the success of a design, then you need to know which parts of the design contribute to or achieve that and which are just there for the ride.'

She also described the role of the researchers – who unlike designers has the time to consider these issues – as key in the development of tools to evaluate effectiveness: 'But they need direction, and there needs to be more co-operation between designers and researchers.' Furthermore, better paths are needed to deliver the results of research back to the world of practice, enabling designers to pick up on and apply the findings.

Following up Light's point about the importance of designers evaluating their own work, Meriel Lenfestey, founder and MD of Flow Interactive argued for internal evaluation based on a consideration of the quality of the design process.

'We need to define what we mean by design: the verb design versus the noun with a capital D,' she argued. For the former, there are usability methodologies and benchmarking against set criteria. The latter can be evaluated with design awards, and achievements against stated objectives within the team.

Lenfestey also contrasted the positions of the designer per se, and the design or development team, incorporating a range of job roles. Being a good designer involves being able to produce ideas, facilitate and problem solve, she said; the team process is more concerned with communication. But nobody is evaluating the quality of the team process, she said, echoing a point made by Schulze that the dialogue around the product is more important than the product itself.

A point made by several speakers was that design needs to be evaluated against the brief. Light accepted that external evaluators would not get the design brief or sales figures, but regarded it as inevitable that in evaluating a website, the evaluator would be considering what the producer had intended or what the evaluator thinks the producer had intended and what the user wanted to do. Echoing this, Lenfestey argued that design succeeds in relation to criteria set by the sponsor, whether this is to make money, to entertain or to inform.

Money talks and the real measures that we should be using to evaluate are sales and financial returns, argued Schulze. But Phil Barrett of Flow Interactive questioned the ability of such measures to assess effectiveness of experience design involved in such products as pop videos, which perhaps – according to Light – might need to be judged as a thing of beauty rather than products requiring a direct financial return.

As Macdonald pointed out in his introduction, the debate was lacking a client perspective – unfortunately, none that were approached had stepped forward to provide their views. And, ultimately, it's the client with his or her chequebook that ultimately stands in judgement of the success or failure of design.

'Internet addiction' linked to Depression
Source: BBC, 9 February 2010
 
There is a strong link between heavy internet use and depression, UK psychologists have said.

Could *You* be more Usable?
Source: UN, 8 February 2010
 
Bet you could.

Stowe Boyd on 'Steampunk' thinking about the Future of Computing
Source: Stowe Boyd's blog via Experientia, 6 February 2010
 
Are established metaphors of user experience holding us back from new ways of structuring our interaction through computers?

Nokia's User Experience Programme
Source: UN, 5 February 2010
 
Nokia has put together a rich and informative website covering the key elements of user experience.

Interfaces magazine: latest issue available now
Source: HCI News Service, 4 February 2010
 
The latest issue of Interfaces is now available in pdf format, free from the Interaction Website.

A Lighter Brigade of Chargers
Source: UN, 3 February 2010
 
Lots of gadgets, one charger. At last.

Mobile Touch Screens could soon Feel the Pressure
Source: MIT Technology Review, 2 February 2010
 
A quantum switch could add pressure sensing to mobile screens.

Usability, Usability, Usability: why the iPad will Succeed
Source: Econsultancy, 1 February 2010
 
The tech critics love it, hate it, love it again, shrug it off. What do usability experts say?

British Airways - at last some good news
Source: Loop11, 30 January 2010
 
In a recent website usability study for the world's leading airlines, the British Airways website proved to be the most user friendly, with Malaysia Airlines and Virgin Atlantic having the lowest user experience rating.

Computation of Emotions in Man and Machine
Source: Royal Society, 29 January 2010
 
Advances in computer technology now allow machines to recognise and express emotions, paving the way for improved human-computer and human-human communications.