Feature: Accessibility Checklists get a Ticking Off

Accessibility checklists did not fare too well in the DRC's formal investigation of UK Web accessibility conducted by City University. The report even managed to elicit a defensive response from the W3C's Web Accessibility Initiative (see UN story: W3C argues with DRC Web Accessibility Findings).

In the 1,000 sites that were tested against the WAI’s Web Content Accessibility guidelines with an automatic testing tool…
• There was no correlation between the number of WAI checkpoint violations and the results of the user tests.
• 69% of the warnings raised by automated testing needed to be manually checked, but only 5% of those resulted in violations (so 95% of the warnings that required manual checking were false positives)
These are pretty discouraging results, though automated testing does have some compensating strong points:
• The tests are quick and inexpensive to run.
• They are (or can easily be) incorporated in development tools.
• If developers are given the responsibility of ensuring that all their pages pass accessibility tests, they will make it their business to ensure that any manual checking needed is minimal.

100 of the sites were usability tested. The tests found 585 issues. The majority of these (425) were attributed to just 10 problems as shown in the table below.

Problem Count (%)
1. 101 (17.3%) Unclear and confusing layout of pages

2. 96 (16.4%) Confusing and disorienting navigation mechanisms

3. 59 (10.1%) Inappropriate use of colours and poor contrast between content and background

4. 45 (7.7%) Incompatibility between accessibility software and web pages

5. 44 (7.5%) Graphics and text size too small

6. 24 (4.1%) Incorrect or non-existent labelling of links, form elements and frames

7. 23 (3.9%) Cluttered and complex page structures

8. 16 (2.7%) ALT tags on images non-existent or unhelpful

9. 10 (1.7%) Lack of alternative media for audio-based information and complex terms/language

10. 7 (1.2%) Complicated language or terminology

Other: 160 (27.4%)

Total: 585 (100.0%)

Here is where the controversy begins. City University says in its report that 45% of these problems cannot be attributed to WAI guidelines. The WAI in its defence claims that '95% of the barriers reported are indeed covered by existing checkpoints in WAI Guidelines'. Personally, I agree that problems one and two identified in the table are not directly covered by WAI guidelines. They account for about a third of the total (33.7%). It may be that some of the problems grouped in 'other' are also not addressed by the guidelines, but problems three through ten certainly are.

So who is right? The correct answer is that it does not matter. What distinguishes problems one and two from most of the others is that they are fundamentally usability problems. We have become used to thinking of accessibility as an isolated issue, when, of course, it is simply a prerequisite to usability.

Consider an example from the real world: Houses need firm foundations. You wouldn't spend a lot of time and money building a house on poor foundations, but the same time it would be pointless to commit 90% of your resources to the foundations alone. The problem of accessibility guidelines is very similar. If you think you're done just because you've been able to put a tick against each of the checkpoints, you've largely missed the point. Disabled users don't need accessible systems, they need usable systems. But it's very difficult, if not impossible, to get usability through rigid adherence to guidelines.

Accessibility guidelines are a good starting point for developers but the whole issue of accessibility as a separate topic is something of a distraction from the real need for inclusive design. We have to consider usability across a much broader range of the population, including people with disabilities and of advancing years. Happily, some of the effort and attention that accessibility attracts will bring us a step closer to inclusive design.

William Hudson
email: whudson@syntagm.co.uk
Syntagm

William Hudson writes, teaches and consults in the fields of user centred design, user interface design and usability. As head of expert evaluation for the Intranet Benchmarking Forum, he has assessed the usability and accessibility of some of the UK's largest intranets.


References
1. DRC Web Accessibility Report Produced by City University:
http://www-hcid.soi.city.ac.uk/rhDrc.html
2. WAI Response to UK Web Accessibility Report:
http://www.w3.org/2004/04/wai-drc-statement.html

Associated Link: Syntagm

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.