Testing the Testing Software: Morae 1.3 is put through its paces

Morae is an indispensable tool for anyone who conducts PC-based usability studies. It's indispensable for two reasons. Firstly, it’s good. Secondly, it doesn't have any competitors. There just isn't anything out there that enables you to do live logging and easily generate picture-in-picture full resolution highlight videos from your user studies.

If you don't already have it, I recommend contacting Techsmith and requesting a copy of the trial version. It's fully functional for 15 days so you'll have enough time to run at least one study. The interface is quite simple and doesn’t have an overwhelming set of features, almost anyone who has done a little video editing before should be able to just pick it up and start making highlight videos. However, it doesn’t really tell you how to structure and run a user study, so if you are a new-comer to user research, you will benefit from getting yourself a usability textbook like Kunivasky (2003), Rubin (1994), or one of the many others that are available.

Morae also offers click logging, keystroke logging and screen text logging, which they refer to as "Rich Recording Technology" (RRT). Some practitioners I’ve spoken to have been a little disappointed about this, as it seems to promise a little more than it delivers, in that it captures the data for you, but it’s up to you to export this data from Morae and process it yourself (e.g. in Excel or SPSS). It also doesn’t directly capture measures like task time and task success, because Morae doesn’t directly support the idea of your test being separated into a series of discrete tasks. You can log events on the timeline as they happen, and have Morae automatically chunk up the video based according to these events afterwards, but this isn’t quite the same as configuring the tasks in advance and having it lead your participant through them in sequence. Similarly, Morae does not provide the facilities to integrate the capture of questionnaire data. For example, if you want to use the System Usability Scale (SUS) in your study, you’ll have to do it separately, using another package. This contrasts with the approach typically employed by remote user testing packages (e.g. Vividence or UserZoom); although Techsmith do have plans to integrate questionnaire data capture and study structuring into a future release.

Nevertheless, if you tend to run studies that utilise click and keystroke logging, you will obviously find Morae’s RRT capture facilities useful. If you tend towards qualitative analysis and highlight videos, this logged data will still be useful to you as it is searchable, and thus you can quickly cut though large tracts of video footage. For example, if you want to find the point in a video at which the participant visits a particular web-page, then you can just search for the page filename, and it will be flagged up on the timeline. Morae’s RRT even records the text of the clock in the task-bar, which is useful if you have an untrained observer who wants to contribute (e.g. a client), since they can just scribble down the clock time with their comment, and later on you can find that point in the video by doing a search for that clock time.

If you're upgrading from 1.1, you'll also notice a much-needed embellishment of the storyboard tool that was introduced in version 1.2. The storyboard tool is the area in which you create your highlight video. Where previously in 1.1 you could only ever have one storyboard in a project, in 1.2 and 1.3 you can now have as many as you like, and, most importantly, you can save and retrieve them. Another crucial improvement from 1.1 to 1.2 was the ability to batch export your highlight videos- previously you had to do them manually one-by-one, which was utterly tedious. In fact, if you are a 1.1 user, this alone should encourage you to upgrade to 1.3.

If you’re upgrading from 1.2, you’ll notice only one major feature update - the addition of a zooming timeline. Previously, it was very hard to pinpoint an exact frame in the video. Now, the zooming timeline allowing your edits to be spot on. One minor limitation with this zooming timeline is that the scale labels can seem rather arbitrary, so for example you can find yourself looking at a scale that shows “00:00.00, 01:34.43, 03:08.88, etc.” and it would be clearer if it jumped through sensible increments, but this is a very minor issue.

One of my personal gripes with Morae is the logging tool provided in the remote viewer software. The concept of the tool is fantastic – while moderating the session or watching it remotely from another location, you can type a comment or press a key combination and have it time-stamped and associated on the video timeline. However, if you’re like me and you like to type a continuous transcript of spoken words and notes, this isn’t really supported. For example, if you log a large number of comments, you cannot later view these comments as a textual document. Perhaps this is an improvement we will see in a later version.

Interestingly, Techsmith has released an API that allows 3rd party developers to build software that can talk to Morae. This means that usability companies will be able to customise Morae to their own needs, which is great because we all like to do things differently, and this will help maintain diversity in the industry. So perhaps I should dust down my VB skills and write that logging tool I’m dreaming of instead of waiting for someone else to do it?


Conclusions

To boil the decision down to a nutshell, you’re going to be coming from one of four scenarios: never-used-it, used-it-but-don’t-own-it, considering an upgrade from 1.1, or considering an upgrade from 1.2. If you’re in the first two of those you’re going to be faced with a big bill: $1298. Still, if you do more than a little usability work I don’t see how you can live without it. If you’re considering an upgrade from 1.1, it’s well worth it since you’ll get the compounded improvements of 1.2 and 1.3 together for just $249. If you’re considering an upgrade from 1.2, the benefits aren’t so huge, as you’re basically paying for a zooming timeline and a few other embellishments, but $249 is still not a big price to pay. Plus, if you’re in an educational institution, you’ll also qualify for a small discount.

If you run user studies regularly and you’re somehow still not sure that you need Morae, I strongly suggest that you get the trial version, run a quick hall study, generate some highlight videos and present them to the product managers. Morae highlight videos always get great feedback – they are just so clear, so convincing, and they give a human touch to your findings. Show people numbers, even if they’re statistically significant, and people just don’t buy into it emotionally. Show them a highlight video of a problem happening to a few of their customers, and the reactions are immediate.

Associated Link: Harry's Personal Website

All change at the top for System Concepts
Source: System Concepts Ltd, 3 July 2009
 
Leslie Fountain has been promoted to joint Managing Director of leading usability consultancy System Concepts.

Life in UCD immortalised in fiction: you couldn't make it up
Source: UN, 2 July 2009
 
Sarah Herman's fictitious book on life in a user-centred design company has hit the shelves and The Guardian's book pages...

Interfaces Magazine - Issue 79: The Education Issue
Source: Interaction Group, 1 July 2009
 
The latest issue of Interfaces is now available as a free download from the Interaction Website.

Two new Behavioural research Tools from Noldus
Source: UN, 30 June 2009
 
Tool updates make on-site behavioural data collection easier.

Cell Phones that Listen and Learn
Source: MIT Technology Review, 29 June 2009
 
New software tracks a user's behavior by monitoring everyday sounds.

Top Six Don’ts for Usability Testing
Source: FutureNow Inc., 27 June 2009
 
Six tips for creating quality usability tests to ensure useful feedback from testers.

Usability: ‘Lovely software. But I can’t work it’
Source: FT.com, 26 June 2009
 
In a recent survey by Global Graphics, 77 per cent of office workers estimate they lose up to one hour a week because business software is difficult to use.

And what do you do?
Source: Dexo Design, 25 June 2009
 
How do you describe your job role? Here are the results of a recent 'Preferred UX/UI Title' Poll.

Most Doctors cite Usability as critical to Electronic Health Record Adoption
Source: TMCNet, 24 June 2009
 
It's all about 'meaningful use'.

Glossy monitors look good but can hurt
Source: QUT, 23 June 2009
 
A new advisory cites research which suggests high gloss monitors make users sit awkwardly.