Orange – Advanced prototype user testing with eye tracking

• The brief
• How we responded to the brief
• What we did
• The outcomes
• Timing and costs

Download this case study in PDF format

The Brief

Orange is one of the main mobile phone networks and broadband providers in the UK. Part of its website, Orange shop, sells mobile phone packages (to both business and personal users) and Orange broadband. As with other e-commerce sites, conversion rates on the site are vital.

Orange was embarking on a redevelopment of its online shop and wished to undertake user testing on an advanced prototype to set the direction for future functional, content and onsite retailing developments in order to improve conversion rates. The initial focus was on personal users of both ‘pay as you go’ and ‘pay monthly’ services.

How we responded to the brief

We proposed usability testing of the prototype, using eye tracking technology, on the Shop site with 6 testers, combining a mixture of Orange and non-Orange customers on ‘pay as you go’ and ‘pay monthly’ services. This testing would be undertaken as 3 unobserved tests in advance, followed by an observed session with 3 testers, involving key Orange stakeholders and designers. This observed session would be followed by a facilitated discussion, allowing the observers to identify the key issues with the site and agree actions in a single day.

What we did

We discussed the objectives of the Shop redevelopment and the proposed user journeys with the project manager and prepared a facilitation guide to test these.

We then recruited the 6 testers, and ran the testing sessions. All the sessions were conducted in our state of the art studios in Chippenham, 75 minutes by train from London Paddington.

The research session was recorded (audio recording and screen capture with tester picture-in-picture, as well as eye tracking, showing where the testers looked).

The observers watched the testing through a one-way mirror, capturing issues on post-its as they occurred. Following the testing, a discussion was facilitated by WUP to agree the issues and appropriate actions.

Following the testing, we detailed the outcomes of the research and the conclusions of the facilitated discussion in a report.

The outcomes

The research identified a number of usability issues with the Orange shop prototype, and provoked a discussion among the Orange stakeholders about how best to facilitate successful user journeys through the site and, therefore, how to take the development of the site forward. The stakeholders agreed which changes should be made to improve the prototype on the same day as the testing, speeding the development process.

“Many thanks for [the report]. We are working through this with [our design agency] this week- the results look extremely useful” Jeff Collins – Information Architect, Online sales

Timing and costs

The project was conducted in May 2008 and took just 2 weeks to complete from recruitment to delivering the final report at a cost of approximately £7,000.

Return to top