WUP offers usability testing.
Download this product sheet in PDF format
Why and when
Usability testing reveals the issues with a web site that interfere with a good user experience – how users achieve their goals and the usefulness and relevance of the content - surfacing both behaviours and attitudes. Improving the user experience increases user satisfaction and:
- Increases conversion rates on e-commerce sites and completion rates for transactions on public sector sites
- Increases the successful take up of information on information sites
- Increases return visits and recommendations
Usability testing, applied well, can also help organisations become more user focused by communicating the insights gained from the research.
Usability testing can be used at all stages in a web site’s development from paper prototypes to final designs.
Our approach
The approach we adopt brings together rigorous user-focused usability research combined with a process to produce agreed and actionable results.
User Focused research
We adopt a user-led goal-oriented approach to usability testing that surfaces testers’ behaviours and actions to help inform thinking about both strategic issues (how users want to interact with the site) and tactical issues (the usability issues that enable or inhibit user goals being achieved). We use a facilitation guide that encourages a user led journey, with a combination of free exploration, broad scenarios, and loosely defined tasks based on the tester’s preferences.
We do not use a highly scripted approach employing a set of tightly pre-defined tasks as this does not necessarily address the issues that users are most interested in. We believe that testers will engage more realistically in the testing process if they are asked to undertake activities in which they are interested and that are relevant to them in their ‘real’ lives. The research data that results from this process will provide more insightful and relevant results.
Action Oriented Process
A key element of our approach is to encourage all those in an organisation who can influence the implementation of the session’s outcomes to attend at least some of the usability testing session and a discussion facilitated by WUP. During the testing sessions, observers are asked to record issues using cognitive mapping techniques for subsequent discussion, in order to capture their immediate reactions to the testers’ experiences. Subsequently, a WUP consultant facilitates a discussion of the issues to identify the priority areas to be fixed. This discussion means that the development team takes ownership of the research results, and develops a collective view of the priority issues to be rectified that enables the identification of actionable results.
WUP consultants bring to these discussions their experience of watching a large number of testers on a range of different web sites, as well as a detailed understanding of usability issues. As the situation demands, we draw on this experience to ensure the discussions are guided by this knowledge.
How we usability test
Depending on the number of users to be tested on a site, our research typically has four elements:
- Project preparation
- Recruitment
- Unobserved user testing
- Observed user testing
N.B. the number of testers required in any usability testing programme depends on a range of factors (e.g. what it is wished to achieve, the budget and time available and the site’s target audiences). However, our experience of undertaking testing on hundreds of websites shows that typically a relatively small proportion of usability problems account for the bulk of the bad user experience on a site. These problems can be identified with a small number of testers (e.g. 3). It is seldom particularly valuable or cost effective to test more than 12 users, even on sites with several different audiences.
Project preparation
We plan the research in consultation with the client so the client gets the ‘right’ solution -not an ‘off the shelf’ one. Prior to the research, we agree with the client’s project manager the key aims of the testing, the tester profiles, testing locations, timescales, etc. We prepare a facilitation guide for the testing sessions for approval by the client’s project manager in advance of the testing sessions.
Recruitment
We undertake our own recruitment - we do not use third parties. We believe this allows us to recruit more accurately to our client’s specification, provide greater flexibility, and treat testers in a decent and ethical way. We are experienced at recruiting testers from a wide range of backgrounds and occupations, and with a range of experience, including testers with visual, motor or cognitive impairment.
All potential testers undergo a detailed screening to ensure they match the tester profile - a profile of the testers is sent to the client project manager prior to the testing session for approval. Testers are paid a fee for their participation in the usability testing: this ensures their participation and demonstrates appreciation of the value of their involvement.
Unobserved User Testing
The unobserved user testing is conducted as follows:
- The unobserved testing is conducted either at our facilities or using a mobile usability laboratory in suitable testing locations (Mobile usability laboratory consists of: Laptop running Techsmith Morae, with webcam and microphone, to record screen, tester’s face, audio)
- The testing sessions are facilitated by a WUP consultant using a user led approach (see ‘User focused research‘ above)
- Following the unobserved testing, an interim report is produced that summarises the research outputs to inform the people attending the observed user testing
Observed User Testing
The observed user testing is conducted as follows:
- The observed testing day is split into two halves, with user testing (3 testers) in the morning and a facilitated discussion session in the afternoon
- The observed testing session is conducted either in our observation studios or other suitable facilities and observed by up to 8 client stakeholders (ideally all those involved in implementing the research outputs should attend the observed testing including third party site developers if used)
- The testing sessions are facilitated by a WUP consultant using a user led approach (see ‘User focused research‘ above)
- The observers are active participants, capturing issues as they arise. A second WUP consultant facilitates this process. (See ‘Action Orientated Process‘ above)
- A discussion session is held after the research to discuss the outcomes and identify the priority issues to be fixed. This session enables the development team to take ownership of the research results, and develop a collective view of the priority issues to be addressed. This approach means that the team will, within a single day, develop a shared view and have a detailed knowledge, and ownership, of the issues raised, and will agree the implications of these issues, the required actions and the priorities. (See ‘Action Orientated Process‘ above)
- The sessions are recorded onto a DVD capturing the test screen outputs, a video of the tester, and audio outputs for analysis purposes and subsequent viewing by the client
Outputs
WUP undertakes a qualitative analysis of the data in line with the WUP Usability Framework. The data sources comprise the ‘real time’ issues captured by observers during the testing sessions and the DVD recordings, which are analyzed following the user research. Usability issues are sorted according to our Usability Framework, and used to support and augment the conclusions and actions agreed during the facilitated discussion session.
The outputs of the research are:
- Interim report of the outcomes of the unobserved testing
- A final report covering:
- Background and objectives for the specific usability testing session
- Key findings from the usability testing session covering the key usability issues
- Verbatim tester comments to support the issues identified
- Summary of facilitated discussion outcomes
- A list of recommendations and a prioritized action plan as agreed at the facilitated discussion that would detail timescales and responsibilities (if agreed)
- DVDs of the testing sessions showing the screen being viewed, tester head shot and the audio of the session
Client examples
WUP have undertaken more than 100 usability testing projects for over 50 clients in both the public and private sectors. Recent work includes:
| Client | Project | Date |
|---|---|---|
| Legal Services Comission | Usability testing of Community Legal Advice site (6 testers) | March 2008 |
| QCA | Usability testing of prototype (12 testers) | Feb 2008 |
| British Antarctic Survey | Usability testing with eye tracking software (12 testers) | Jan 2008 |
| F & C Investments | Usability testing (3 testers) | Dec 2007 |
| easyCruise | Usability testing (6 testers) | Dec 2007 |
| Crerar Hotels | Usability testing (6 testers) | Dec 2007 |
| Early Learning Centre | Usability testing (6 testers) | Oct 2007 |
| The British Library | Usability testing of digitized newspaper archives (3 testers) | Oct 2007 |
| The University of Brighton | Usability testing (8 testers) | August 2007 |
Fees
Most of our work is tailored to individual client’s requirements and we quote an inclusive price for each piece of work. However, the following gives an illustration of typical costs:
- Unobserved user testing £450 per tester
- Observed user testing and facilitated debrief £5,450 (NB this fully inclusive price covers all WUP’s fees, tester recruitment and payment, observation studios, equipment hire, reports, DVDs and travel expenses etc.)
Terms & Conditions
- VAT will be added to all charges at the currently prevailing rate. Payment will be within 30 days on the presentation of our invoice.
- Additional work not identified above would be charged at WUP’s standard rates (£950/day for directors, £750/day for consultants, £600/day for facilitators, £250/day for support staff). Additional expenses not shown above would be charged at cost.
- Cancellation of booked sessions will incur cancellation fees as follows: cancellation within 4 weeks of the session, 50% of the fee is payable; cancellation within 2 weeks of the session, 100% of fee is payable.
- The rates quoted assume tester remuneration of £40 plus travel expenses up to a maximum of £10. If tester remuneration needs to be greater than this for any reason (e.g. securing individuals with high time value or on short lead times) then any costs above the basic charge will be invoiced at cost.
- It is assumed that the client will invite the observers to research sessions.