Feature: Accessibility Checklists get a Ticking Off

Accessibility checklists did not fare too well in the DRC's formal investigation of UK Web accessibility conducted by City University. The report even managed to elicit a defensive response from the W3C's Web Accessibility Initiative (see UN story: W3C argues with DRC Web Accessibility Findings).

In the 1,000 sites that were tested against the WAI’s Web Content Accessibility guidelines with an automatic testing tool…
• There was no correlation between the number of WAI checkpoint violations and the results of the user tests.
• 69% of the warnings raised by automated testing needed to be manually checked, but only 5% of those resulted in violations (so 95% of the warnings that required manual checking were false positives)
These are pretty discouraging results, though automated testing does have some compensating strong points:
• The tests are quick and inexpensive to run.
• They are (or can easily be) incorporated in development tools.
• If developers are given the responsibility of ensuring that all their pages pass accessibility tests, they will make it their business to ensure that any manual checking needed is minimal.

100 of the sites were usability tested. The tests found 585 issues. The majority of these (425) were attributed to just 10 problems as shown in the table below.

Problem Count (%)
1. 101 (17.3%) Unclear and confusing layout of pages

2. 96 (16.4%) Confusing and disorienting navigation mechanisms

3. 59 (10.1%) Inappropriate use of colours and poor contrast between content and background

4. 45 (7.7%) Incompatibility between accessibility software and web pages

5. 44 (7.5%) Graphics and text size too small

6. 24 (4.1%) Incorrect or non-existent labelling of links, form elements and frames

7. 23 (3.9%) Cluttered and complex page structures

8. 16 (2.7%) ALT tags on images non-existent or unhelpful

9. 10 (1.7%) Lack of alternative media for audio-based information and complex terms/language

10. 7 (1.2%) Complicated language or terminology

Other: 160 (27.4%)

Total: 585 (100.0%)

Here is where the controversy begins. City University says in its report that 45% of these problems cannot be attributed to WAI guidelines. The WAI in its defence claims that '95% of the barriers reported are indeed covered by existing checkpoints in WAI Guidelines'. Personally, I agree that problems one and two identified in the table are not directly covered by WAI guidelines. They account for about a third of the total (33.7%). It may be that some of the problems grouped in 'other' are also not addressed by the guidelines, but problems three through ten certainly are.

So who is right? The correct answer is that it does not matter. What distinguishes problems one and two from most of the others is that they are fundamentally usability problems. We have become used to thinking of accessibility as an isolated issue, when, of course, it is simply a prerequisite to usability.

Consider an example from the real world: Houses need firm foundations. You wouldn't spend a lot of time and money building a house on poor foundations, but the same time it would be pointless to commit 90% of your resources to the foundations alone. The problem of accessibility guidelines is very similar. If you think you're done just because you've been able to put a tick against each of the checkpoints, you've largely missed the point. Disabled users don't need accessible systems, they need usable systems. But it's very difficult, if not impossible, to get usability through rigid adherence to guidelines.

Accessibility guidelines are a good starting point for developers but the whole issue of accessibility as a separate topic is something of a distraction from the real need for inclusive design. We have to consider usability across a much broader range of the population, including people with disabilities and of advancing years. Happily, some of the effort and attention that accessibility attracts will bring us a step closer to inclusive design.

William Hudson
email: whudson@syntagm.co.uk
Syntagm

William Hudson writes, teaches and consults in the fields of user centred design, user interface design and usability. As head of expert evaluation for the Intranet Benchmarking Forum, he has assessed the usability and accessibility of some of the UK's largest intranets.


References
1. DRC Web Accessibility Report Produced by City University:
http://www-hcid.soi.city.ac.uk/rhDrc.html
2. WAI Response to UK Web Accessibility Report:
http://www.w3.org/2004/04/wai-drc-statement.html

Associated Link: Syntagm

User Interviews - Analysis Simplified
Source: Webcredible, 6 January 2009
 
You’ve conducted your user interviews, but now you need to make sense of all that information you’ve gathered.

Why Products Fail
Source: ComputerWorld, 5 January 2009
 
Most gadget and software makers don't understand what users want most: control.

How to Design Websites for Mobile Phones
Source: stemkoski.com, 3 January 2009
 
Tips from Ryan Stemkoski's web design blog.

Pioneer of Cyberspace honoured
Source: BBC, 2 January 2009
 
A professor who invented a forerunner of the world wide web has been made a dame in the New Year Honours.

2008 in Review: Developments that rocked the world of User Experience
Source: Catalyst Resources, 1 January 2009
 
A look back at 2008 highlights some of the key developments that rocked the world of user experience.

2008: The Year Online
Source: MIT Technology Review, 31 December 2008
 
The business of social networking, cloud computing, and a flaw in the fabric of the Internet top the most notable stories of 2008.

Shoveling through the Spamalanche
Source: UN, 30 December 2008
 
A ‘Spamalanche’ of 3,000 emails will be waiting in your inbox by the time you get back to work. What can you do besides 'delete all'?

New guidelines boost Web Access
Source: BBC, 29 December 2008
 
The World Wide Web Consortium (W3C) has announced a new standard to make sites more accessible to older and disabled people.

Measuring Emotional Engagement
Source: UN, 27 December 2008
 
New applications of electroencephalography (EEG) provide the means to gather detailed information on a user’s emotional relationship with a brand or service.

UXalliance thrives on Research strengths
Source: UN, 24 December 2008
 
The UXalliance provides clients with access to over 220 user experience professionals and 40 test labs worldwide.