Usability testing

Usability testing reveals the issues that interfere with a good user experience – how users achieve their goals and the usefulness and relevance of content.

• Why and when
• Our approach
• How we usability test
• Outputs
• Fees

Download this product sheet in PDF format

Why and when

Usability testing reveals the issues with a web site that interfere with a good user experience – how users achieve their goals and the usefulness and relevance of the content – surfacing both behaviours and attitudes. Improving the user experience increases user satisfaction and:

• Increases conversion rates on e-commerce sites and completion rates for transactions on public sector sites
• Increases the successful take up of information on information sites
• Increases return visits and recommendations

Usability testing, applied well, can also help organisations become more user focused by communicating the insights gained from the research.

Usability testing can be used at all stages in a web site’s development from paper prototypes to final designs.

Our approach

The approach we adopt brings together rigorous user-focused usability research combined with a process to produce agreed and actionable results.

User Focused research

We adopt a user-led goal-oriented approach to usability testing that surfaces testers’ behaviours and actions to help inform thinking about both strategic issues (how users want to interact with the site) and tactical issues (the usability issues that enable or inhibit user goals being achieved). We use a facilitation guide that encourages a user led journey, with a combination of free exploration, broad scenarios, and loosely defined tasks based on the tester’s preferences.

We do not use a highly scripted approach employing a set of tightly pre-defined tasks as this does not necessarily address the issues that users are most interested in. We believe that testers will engage more realistically in the testing process if they are asked to undertake activities in which they are interested and that are relevant to them in their ‘real’ lives. The research data that results from this process will provide more insightful and relevant results.

Action Oriented Process

A key element of our approach is to encourage all those in an organisation who can influence the implementation of the session’s outcomes to attend at least some of the usability testing session and a discussion facilitated by WUP. During the testing sessions, observers are asked to record issues using cognitive mapping techniques for subsequent discussion, in order to capture their immediate reactions to the testers’ experiences. Subsequently, a WUP consultant facilitates a discussion of the issues to identify the priority areas to be fixed. This discussion means that the development team takes ownership of the research results, and develops a collective view of the priority issues to be rectified that enables the identification of actionable results.

WUP consultants bring to these discussions their experience of watching a large number of testers on a range of different web sites, as well as a detailed understanding of usability issues. As the situation demands, we draw on this experience to ensure the discussions are guided by this knowledge.

How we usability test

Depending on the number of users who will test a site, our research typically has four elements:

• Project preparation
• Recruitment
• Unobserved user testing
• Observed user testing

N.B. the number of testers required in any usability testing programme depends on a range of factors (e.g. what it is wished to achieve, the budget and time available and the site’s target audiences). However, our experience of undertaking testing on hundreds of websites shows that typically a relatively small proportion of usability problems account for the bulk of the bad user experience on a site. These problems can be identified with a small number of testers (e.g. 3-5). It is seldom particularly valuable or cost effective to test more than 12 users, even on sites with several different audiences.

Project preparation

We plan the research in consultation with the client so the client gets the ‘right’ solution – not an ‘off the shelf’ one. Prior to the research, we agree with the client’s project manager the key aims of the testing, the tester profiles, testing locations, timescales, etc. We prepare a facilitation guide for the testing sessions for approval by the client’s project manager in advance of the testing sessions.

Recruitment

We undertake our own recruitment – we do not use third parties. We believe this allows us to recruit more accurately to our client’s specification, provide greater flexibility, and treat testers in a decent and ethical way. We are experienced at recruiting testers from a wide range of backgrounds and occupations, and with a range of experience, including testers with visual, motor or cognitive impairment.

All potential testers undergo a detailed screening to ensure they match the tester profile – a profile of the testers is sent to the client project manager prior to the testing session for approval. Testers are paid a fee for their participation in the usability testing: this ensures their participation and demonstrates appreciation of the value of their involvement.

Unobserved User Testing

The unobserved user testing is conducted as follows:

• The unobserved testing is conducted either at our facilities or using a mobile usability laboratory in suitable testing locations (Mobile usability laboratory consists of: Laptop running Techsmith Morae, with webcam and microphone, to record screen, tester’s face, audio)
• The testing sessions are facilitated by a WUP consultant using a user led approach (see ‘User focused research‘ above)
• Following the unobserved testing, an interim report is produced that summarises the research outputs to inform the people attending the observed user testing

Observed User Testing

The observed user testing is conducted as follows:

• The observed testing day is split into two halves, with user testing (3 testers) in the morning and a facilitated discussion session in the afternoon
• The observed testing session is conducted either in our observation studios or other suitable facilities and observed by up to 8 client stakeholders (ideally all those involved in implementing the research outputs should attend the observed testing including third party site developers if used)
• The testing sessions are facilitated by a WUP consultant using a user led approach (see ‘User focused research‘ above)
• The observers are active participants, capturing issues as they arise. A second WUP consultant facilitates this process. (See ‘Action Orientated Process‘ above)
• A discussion session is held after the research to discuss the outcomes and identify the priority issues to be fixed. This session enables the development team to take ownership of the research results, and develop a collective view of the priority issues to be addressed. This approach means that the team will, within a single day, develop a shared view and have a detailed knowledge, and ownership, of the issues raised, and will agree the implications of these issues, the required actions and the priorities. (See ‘Action Orientated Process‘ above)
• The sessions are recorded onto a DVD capturing the test screen outputs, a video of the tester, and audio outputs for analysis purposes and subsequent viewing by the client

Outputs

WUP undertakes a qualitative analysis of the data. The data sources comprise the ‘real time’ issues captured by observers during the testing sessions and the DVD recordings, which are analyzed following the user research. Usability issues are sorted according to our Usability Framework, and used to support and augment the conclusions and actions agreed during the facilitated discussion session.

The outputs of the research are:

• Interim report of the outcomes of the unobserved testing
• A final report covering:
  • Background and objectives for the specific usability testing session
  • Key findings from the usability testing session covering the key usability issues
  • Verbatim tester comments to support the issues identified
  • Summary of facilitated discussion outcomes
  • A list of recommendations and a prioritized action plan as agreed at the facilitated discussion that would detail timescales and responsibilities (if agreed)
• DVDs of the testing sessions showing the screen being viewed, tester head shot and the audio of the session

Fees

Most of our work is tailored to individual client’s requirements and we quote an inclusive price for each piece of work. However, the following gives an illustration of typical costs:

• Unobserved user testing with 6 users, report videos etc. c. £4,000
• Observed user testing with 3 testers plus facilitated debrief, report, videos etc c.£5,000
• Combined observed and unobserved user testing with 9 testers, facilitated debrief, reports, videos, etc. c. £8,000

NB prices include WUP’s fees, tester recruitment and incentives, use of our observation studios (if appropriate), equipment hire, reports, DVDs, etc.

Return to top