Six Metrics for Managing UI Design

I head up Product Design at my company, and as part of a recent management summit, we were asked to fill out an RMPT matrix for our departments (an RMPT matrix consists of (R)esponsibilities, (M)etrics, (P)rocesses, and (T)ools. I have been intending to develop better metrics for both measuring and guiding our design efforts, and this exercise served as a catalyst to get me started. Bear in mind that metrics help you focus your efforts and measure your progress, but you are also held accountable to them.

For (R)esponsibilities I specified the following:
1) Improve our products & innovate
2) Provide the UI design for new features /functions /products
3) Approve any UI design work done outside of Product Design
4) Validate our UI designs and explore user needs through user testing

Given those responsibilities (and that's important because your metrics are linked to them) I then came up with the following metrics and met with my team who helped to refine them:

For a given period (eg a business quarter):
1) Number of layouts delivered
2) Number of interactive prototypes created
3) Percentage of product design requests completed by commit date
4) Number of users tested
5) Number of product improvements made
6) Number of products insights documented

Discussion:

1) We struggled with what to call 'layouts' and how to word it specifically. We started with 'Number of designs delivered', moved to 'Number of screens delivered' and settled on 'Number of layouts delivered'. We chose layouts because designs seemed a little too broad and didn't really give any indication of what we would actually be delivering.
2) We prototype when feasible and want to push this effort as well as measure it. Again we struggled with the wording (ie 'prototypes' vs. 'designs', etc). We wanted to make it clear that we are measuring the number of interactive, non-static prototypes.
3) We recently implemented a Product Design Request System where we track the growing volume of design requests and deliverables to development and product management. We are holding ourselves accountable to deliver on these requests by our committed due date.
4) We test designs on users as much as possible and want to both measure this and push ourselves to do this as much as possible.
5) Here's where it gets a little interesting. This one is admittedly qualitative and may need further refinement. With that said, it is easy for us to remain reactive and work day to day by responding to requests from product management and development without ever taking the initiative to improve our products by combining our research with our unique expertise in usability and design. This metric is meant to drive us to be proactive and take initiative to improve our products and to track those improvements over time to further illustrate our value. To make this metric more objective I plan to tie in efficiency measurements and satisfaction scores (SUS).
6) With our considerable user 'face time' we are in a unique position to collect insights into product shortcomings, needs, and opportunities. We often take copious notes when doing user studies only to leave the notes sleeping silently in our Moleskines. We meet regularly with SME's (subject matter experts), developers, users, and various other stakeholders. This metric is intended to remind us to document, publish, and take action on the insights that we collect.

My reason for publishing these metrics is to get feedback and to learn what metrics other designers work with. Let me know if you think an important metric is missing, or if you disagree with these metrics and why.

Associated Link: User Interface Design

Poor Shopping Basket Usability will hinder Christmas Rush
Source: UN, 20 November 2008
 
A new study from eDigitalResearch identifies customer service and online shopping basket usability as the two key areas where online retailers need to improve.

Future Phones to Read Your Voice, Gestures
Source: Wired, 19 November 2008
 
Five years from now, it is likely that the mobile phone you will be holding will be a smooth, sleek brick — a piece of metal and plastic with a few grooves in it and little more.

Why Digital Research is important in tough Financial Times
Source: Financial Times, 18 November 2008
 
With the banking sector moving towards consolidation, it is crucial that customers are understood, reacted to and rewarded for their loyalty.

Get Ready for 'Ergobamanomics'
Source: AssemblyBlog via ergonomics in the news , 17 November 2008
 
Is President Elect, Barack Obama, a secret ergonomics and usability enthusiast?

The Most Dreaded Keyword Phrase
Source: SearchEngineLand, 15 November 2008
 
In website usability, one of the hardest obstacles to overcome is the mentality of “This is what I would do.”

User Experience Standards Missing from Web 2.0 Designs
Source: UN, 14 November 2008
 
Usability professionals should be more involved with the development of Web 2.0 tools.

New technology showcased at World Usability Day
Source: User Vision, 13 November 2008
 
Today is World Usability Day!

Review: 16 User Interface Prototyping Tools
Source: Dexo Design, 12 November 2008
 
People constantly ask me what the best prototyping or mockup tool is, so I decided to do a review of all the tools I’m aware of.

Radio 4's PM show announces Winner of 'Show Us a Better Way'
Source: BBC, 11 November 2008
 
Ever been frustrated that you can't find out something that ought to be easy to find? Ever been baffled by league tables or 'performance indicators'?

Cisco Systems extends Product Usability Research
Source: Catalyst Resources, 10 November 2008
 
Cisco is conducting extensive usability research with Catalyst Resources, into complex new technologies.