The topic for the presentation was “The power of live viewer on the decision making process”. As one of WUP’s main goals is to turn research into action, consultants at WUP have found that live viewing (of where the user is looking on a page) using eye tracking, accompanied by the attitudes and behaviours of the user from the other side of the glass, helps stakeholders, designers, web managers (anyone who is involved in decision making for the changes) get a collective view of the usability issues. This in turns helps them make decisions during a facilitated discussion on the same day.

The value of using the eye tracker in this way was discussed by the conference participants – about 100 practitioners and academics. Tasin Reza, the WUP user experience who presented the findings, said:

“It was brilliant to hear responses from some of the attendees that they completely agree with what we have presented. The way live viewer is helping our clients to make collective decisions about the issues is invaluable to turn research into action so promptly!”

• The number of people using their mobile device to access news and information on the Internet more than doubled from January 2008 to January 2009.
• Of the 63.2 million people who accessed news and information on their mobile devices in January 2009, 22.4 million (35 percent) did so daily; more than double the size of the audience last year.
• Social networking and blogging are the next most popular daily uses of the mobile Web and these activities are growing rapidly
• Maps are the most popular downloaded application with 8.2 million users

More about this survey

What our research shows is that users can often have very different goals that they wish to achieve when using their mobiles than on a desk or lap top. Virtually nobody is going to use a mobile to look at a web site if they have a larger machine handy. So you use your mobile when you are out and about and you use it for very different things. We did some work testing a financial services site on mobiles and unsurprisingly no one wanted to use their mobiles to research these products, but they were keen to check the value of their portfolios when on the move. Often mobile users will want an organisation’s contact details which can take ages to find even when a site has been designed for mobile use.

Our belief is that organisations looking to build their first mobile web site should identify a very small number of user goals – possibly only one – that the mobile site is to support and concentrate on doing this well. For some sites, like the ones mentioned above, the goals of the user on the move are fairly obvious, for most sites they are not. In which case, research with the target audience to work out the goals to be supported is essential.

• The client, or WUP, produces wireframes/page designs for user testing
• On the morning of day one we test the wireframes/page designs with 3 users. This testing is observed by the client development team
• In the afternoon of day one WUP facilitates a discussion with the development team to identify issues with the prototypes and possible solutions
• On day 2 the client, or WUP, develops new wireframes/page designs based on the previous day’s conclusions. These are subject to an expert review by a WUP usability consultant
• On the morning of day 3 we re-test the revised prototypes with a further 3 users. This testing is observed by the client development team
• During the afternoon of day 3 WUP facilitates a discussion with the development team to identify any remaining issues with the prototypes and possible solutions

This approach means that the client can undertake two rounds of testing in just 3 days (more rounds can be undertaken if required). As the client team are active participants in the development process they take ownership of any usability issues and the solutions developed. Also, because of their involvement, the solutions identified also fit with the client’s technical, political and financial constraints.

WUP can offer this service starting from £6,950 plus VAT.

If you want further information on this service contact Julie Marshall on 01249 444757 or email: julie@wupltd.co.uk

“10.4 Until user agents handle empty controls correctly, include default, place-holding characters in edit boxes and text areas.”

The keywords here being ‘Until user agents handle empty controls correctly’. They now do, so this checkpoint is obsolete - it has been removed from WCAG v2.0. However, what is necessary is provide an explicit label for each form field. For those that want to know how it is done, use the label tag like this:

<label for=”search”>Search this site: </label><input type=”text” id=”search” name=”search_field” />

Note that the ‘for’ attribute in the label tag must match the id attribute of the input field.

Download this article in PDF format

Background

It has been argued (e.g. Penzo, 2005) that eye-tracking can augment standard usability testing methodologies by providing quantitative as well as qualitative data, and by providing insight into micro-behaviours on a site. Standard think aloud usability testing provides qualitative information about what testers are looking at and how they feel about a web page, but eye tracking can provide a wealth of other information such as:

• How testers’ eyes move around a page - the ‘saccades’
• How long testers look at elements of a page – the ‘fixations’
• How often testers look at page elements – the ‘number of fixations’
• How long they look at the page elements – the ‘mean fixation duration’
• The dwell time within each screen

The main visual outputs from eye tracking that most people are familiar with, is the heat map – basically a summary of fixations, and the gaze plot – a summary of where testers have looked on a page.

The Practicalities

Historically eye-tracking equipment was cumbersome, difficult to calibrate, fairly unreliable and it would not work for people wearing glasses or thick mascara. It has, however, improved significantly in recent years. It is now quick to calibrate, reliable (people can look away from the screen without disturbing the results), unobtrusive (you do not need to wear any special headgear) and works with most people (including those who like to wear lashings of mascara or thick specs). But it still requires specialist equipment which is expensive and it is confined mainly to lab based environments.

To produce meaningful heat maps or gaze plots requires large sample sizes – Nielsen Norman Group (2006) argue at least 30 testers are needed to produce meaningful heat maps; these large sample sizes then generate loads of data and a lot of post session analysis. Larger sample sizes and extensive data analysis both have implications in the commercial environment on speed of turnaround of a project and the project cost.

Interpreting Eye Tracking Data

But the real issue with eye tracking is the interpretation of data. A number of commentators (e.g. Spool, 2006, Graphpaper.com 2006) have questioned the value of eye tracking, partly because of the practicalities but also because the results can be misinterpreted:

• Eye tracking can show us where someone was looking on a screen, but it can’t tell us what they were seeing or thinking whilst looking at it – did they register what they were looking at, did they understand it? And if the tester did not look at something directly, did they see it with their peripheral vision?
• Does a long fixation mean that someone is looking at something because it is interesting to them, or because it’s very confusing and they’re having to spend time making sense of it?
• Does a busy gaze plot tell us that there are lots of interesting things to look at on a page, or that a user is confused about where to go to achieve their goals?

These commentators have also questioned whether eye tracking delivers additional diagnostic value obtained from the additional cost and time: “the good conclusions seem to be the same conclusions that a good UI designer (one who understands the desired effect of the design) would come up with without the aid of any tools, just going by their design instincts “ (Graphpaper.com 2006).

We have observed that the interpretation of heat maps has to be very task specific. If a tester is asked to look for a specific item, this will influence strongly they way they look at a web page, and can have major implications on the conclusions that can be drawn. For example, in a Nielsen workshop on eye tracking an example was given of a JCPenny.com home page, where much of the page was taken up with an image of teenage bedroom furniture and linen. The task that had been set was ‘look for a gift for a baby girl’. The heat maps showed that testers were looking at the main navigation – not the image. It was presented at the workshop that the image was a waste of real estate. Is that really the case or was it just that it wasn’t relevant to this task? It may well have contributed to an impression of brand values and showcasing product. It would be very easy to manipulate conclusions from heat maps if used without other qualitative feedback on a site. As Spool (2006) says

The colorful heatmaps are cool (or warm?) to look at, but what are they actually telling you? When someone is gazing at something, is it because they want to look there? Or because the page made them look there? Or because they are resting their eyes there?

Our experience

We have found that eye tracking is a useful additional tool in assessing usability – but that actually it is most useful when used in real time to enable our clients to get greater insight into how users look at page. It’s this real time use, rather than the production of heat maps, that our clients find to be most valuable.

It’s partly about engagement with the process. Spool (2006) relates the experience of usability testing at Google where the lab with the eye tracker gets most use because

the developers pay more attention to the test when the little dot is bouncing around the screen

It’s true – we notice how our clients focus on the screen when the eye tracker is on. And it enables people who have had less exposure to users on sites to understand how users look at screens – the way they look at navigation bars, the way they read text, the way they ignore advertisements!

Time and time again, whilst observing usability testing sessions using the think aloud protocol, we notice that what a person does is quite different from what they might say i.e. their behaviours are not consistent with their attitudes. Arguably eye-tracking enhances the outputs from usability testing by providing a different dimension to understanding their behaviours.

Analysing these two streams of data in parallel provides an even richer picture of issues which may get in the way of users achieving their goals, and which might affect the user experience on a site.

How and when to use it

The first and most obvious use of eye tracking is alongside conventional ‘think aloud’ usability testing. As an overlay, on the screen being viewed by a tester, the eye tracking shows precisely what the user is looking at. There is far less doubt about which screen elements are being looked at or have been noticed. This can add significantly to the understanding of the usability issues on a website. This is particularly valuable when a testing session, or a video of it, is being viewed by a client as it makes explicit, without explanation, a lot of issues that may be obvious to an experienced usability professional. We find it can be used concurrently with think aloud, rather than as retrospective think aloud (i.e. reviewing the tape subsequently with the tester) provided the tester is allowed to explore the site within a loosely structured scenario – focused around achieving one specific goal - rather then being in a tightly scripted format (which we don’t think is a very effective way to usability test a site anyway).

One area where we have found eye tracking to be of particular value is when trying to assess the effectiveness of specific elements when comparing two versions of a page design or wire frames, or the effectiveness of specific screen elements e.g. promotions, navigation bars etc. This type of use was investigated and validated by Bojko (2006) but again she comments that “eye movement measures on their own have a limited applicability in user experience research…and should almost always be used in combination with other measures, including behaviour and user attitudes/preferences“

We rarely produce heat maps for our clients. This is often because the budgets or project timescales do not allow for the numbers of testers or the scale of post session analysis.

But real time eye tracking, interpreted in the light of accompanying usability ‘think aloud’ evidence, provides greater insight than ‘think aloud’ alone, and enables our clients to get a much better understanding of what works and what doesn’t work on their web pages.

© Web Usability Partnership Ltd
Unit 15, Lansdowne Court, Bumpers Farm,
Chippenham, Wilts SN14 6RZ
Tel: 01249 444 757
Email: info@wupltd.co.uk
Web: www.wupltd.co.uk

“It’s my fault I can’t find anything”

On the face of it this would seem to be good news for websites with poor usability. If users don’t blame the site why should the site be improved? The reason is because the site does affect user behaviour.

Some recent, interesting research conducted by Harris Interactive for Tealeaf (a US analytics company) suggests that 41% of online adults who experience problems transacting would switch to a competitor or abandon a transaction entirely if they experienced an online transaction problem. So users’ behaviour is clearly affected by poor usability.

So why is this if they don’t blame the website? One reason could be the artificiality of testing in the lab. We know users often try and ‘please’ the moderator. But another important reason is that websites with poor usability make users feel ‘stupid’. Because users blame themselves, failures make them feel incompetent. How sensible can it be that users associate a website with being made to feel foolish? Will users stick with it or recommend it? Probably not.

The Harris survey also showed that the number of users switching to a competitor or abandoning a transaction, because of problems transacting on sites, is up 12% on 2007. This suggests that users’ tolerance with bad websites is on the wane.

So ensuring users have a good experience is even more important than ever – making them feel stupid is a good way to send them off to your competitors.

Now, while capturing user’s attitudes is useful feedback for some issues, attitudes are notoriously bad at providing reliable feedback on how easy it is to find information on a site. At the conclusion of a usability testing session we routinely ask users to score the website on a number of dimensions, one of which is “How easy was it to find information?” Often users will give quite high scores even when they have failed to complete any of the tasks that have been set.

Also different users will often give the same site very different scores. We offer them a 10 point scale and scores can very by 5 points or more. So clearly, users cannot provide an objective measure of the effectiveness of a site’s navigation, only their perceptions of it, otherwise they would all give a site the same score.

However, users’ perceptions of a site are affected by a whole range of factors including their familiarity with the web and sites similar to the one under test, their sex (women tend to score sites higher than men), their personalities (intolerant men, like me, score sites lower than tolerant men), the number of search strategies available to them, etc., etc. Now, when you are observing users in the lab a lot of these factors are obvious and you take them into account during your review of the user’ experiences on the site. Often the observers will conclude that a site is very poor at getting users to their goals whilst the user is happily saying it is fine.

So what happens when you are remote user testing? How do you know that the user’s assessment of how easy it was to find things is reliable? Our experience suggests that users’ attitudes are of very little value in determining the effectiveness of a site’s navigation: other methods are required that look at user behaviours, such as lab based testing, AB testing etc. Users’ attitudes can provide very valuable feedback to guide other aspects of a site’s development (e.g. what are their goals, does the site content meet these goals, what other information do they want) but can be an unreliable indicator of how easy a site is to use.

Clearly it makes sense for lobby groups to target high profile organisations for infringement of the US’s Americans with Disabilities Act (ADA) this way they get lots of publicity for their cause.

To date, there have been no convictions under the UK’s Disability Discrimination Act for failure to make a web site accessible – all cases have been settled out of court. However, it can only be a matter of time before such a case is brought. It is still sadly the case that very few sites conform to the W3C standards even at the most basic level. We often audit sites that claim A or AA compliance only to find accessibility issues in these categories.

Over reliance on automated auditing, rather than manual checking can be one reason for this, inappropriate interpretation of the W3C guidelines is another.

To my mind having an inaccessible web site is, in most cases, pretty inexcusable. If accessibility is thought about during the development process it is no harder to make site accessible than not – it just needs to be taken into account, like you would in a building when you build it. It needn’t cost any more, indeed it might even save money.

Because so many major UK sites still have poor accessibility it can only be a matter of time before there is a similar case in the UK and the site owners will only have themselves to blame – it will almost certainly mean that they simply have not taken the issue seriously.

Technium is a Welsh Assembly funded organisation providing accommodation and support for early-stage science and technology businesses. WUP undertook usability testing of the site in late 2007. This identified a number of problems including an ineffective information architecture and problems with the layout and design of the site. Subsequently WUP was commissioned to develop a new information architecture and wireframes for key page types. One of the key challenges was that Technium consists of  9 separate centres all of whom effectively wanted their own web sites. We had to come up with an information architecture that met the needs of users to have a coherent user experience as well as the Centres desire to tell their own stories. WUP undertook this development through a number of iterations each informed with user evidence. The development process involved representatives of the Technium centres to ensure that they bought into the new developments. The designs to the wireframes were subsequently developed by S8080, a specialist website design agency. The site has just been launched and has received considerable praise from users for being much easier to use.