Six Metrics for Managing UI Design

I head up Product Design at my company, and as part of a recent management summit, we were asked to fill out an RMPT matrix for our departments (an RMPT matrix consists of (R)esponsibilities, (M)etrics, (P)rocesses, and (T)ools. I have been intending to develop better metrics for both measuring and guiding our design efforts, and this exercise served as a catalyst to get me started. Bear in mind that metrics help you focus your efforts and measure your progress, but you are also held accountable to them.

For (R)esponsibilities I specified the following:
1) Improve our products & innovate
2) Provide the UI design for new features /functions /products
3) Approve any UI design work done outside of Product Design
4) Validate our UI designs and explore user needs through user testing

Given those responsibilities (and that's important because your metrics are linked to them) I then came up with the following metrics and met with my team who helped to refine them:

For a given period (eg a business quarter):
1) Number of layouts delivered
2) Number of interactive prototypes created
3) Percentage of product design requests completed by commit date
4) Number of users tested
5) Number of product improvements made
6) Number of products insights documented

Discussion:

1) We struggled with what to call 'layouts' and how to word it specifically. We started with 'Number of designs delivered', moved to 'Number of screens delivered' and settled on 'Number of layouts delivered'. We chose layouts because designs seemed a little too broad and didn't really give any indication of what we would actually be delivering.
2) We prototype when feasible and want to push this effort as well as measure it. Again we struggled with the wording (ie 'prototypes' vs. 'designs', etc). We wanted to make it clear that we are measuring the number of interactive, non-static prototypes.
3) We recently implemented a Product Design Request System where we track the growing volume of design requests and deliverables to development and product management. We are holding ourselves accountable to deliver on these requests by our committed due date.
4) We test designs on users as much as possible and want to both measure this and push ourselves to do this as much as possible.
5) Here's where it gets a little interesting. This one is admittedly qualitative and may need further refinement. With that said, it is easy for us to remain reactive and work day to day by responding to requests from product management and development without ever taking the initiative to improve our products by combining our research with our unique expertise in usability and design. This metric is meant to drive us to be proactive and take initiative to improve our products and to track those improvements over time to further illustrate our value. To make this metric more objective I plan to tie in efficiency measurements and satisfaction scores (SUS).
6) With our considerable user 'face time' we are in a unique position to collect insights into product shortcomings, needs, and opportunities. We often take copious notes when doing user studies only to leave the notes sleeping silently in our Moleskines. We meet regularly with SME's (subject matter experts), developers, users, and various other stakeholders. This metric is intended to remind us to document, publish, and take action on the insights that we collect.

My reason for publishing these metrics is to get feedback and to learn what metrics other designers work with. Let me know if you think an important metric is missing, or if you disagree with these metrics and why.

Associated Link: User Interface Design

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.