Our UX Research team at On-Off Group wanted to create a guide that could encourage and help people to start Usability Testing their products— wherever they might be in the development journey. Depending on the breadth of your research, Usability Testing can be a daunting task. But truth be told, you don’t need a whole team to run tests and your product doesn’t have to be 100% complete before testing either.
By following this guide, you’ll be able to start testing right away and gain the ability to incorporate user feedback into your design as early as possible. In Part 1 of the series we will discuss the basics of Usability Tests, how to prepare a Research Plan, and digital tools to use for Remote Usability Testing.
Usability Testing is a method of measuring the usability of a website, app, or other digital products. ‘Usability’ is defined as the extent to which a product can be used to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use (ISO 9241-11).
The methodology revolves around observing real users as they attempt to complete tasks of the product. The goal is to reveal areas of confusion and uncover opportunities to improve the overall User Experience (UX).
Users can feel frustrated when they find a product difficult to use. Poor usability can eventually lead to the loss of a product’s competitive advantage or worse, low customer retention. Think about it: there are countless other websites and apps available out there that might provide a better and more hassle-free user experience. This is where UX Researchers & Designers find common ground with the C-Suite and Business Managers.
Usability goals are business goals. Websites that are hard to use frustrate customers, forfeit revenue, and erode brands. — Forrester Research
This is why Usability Testing is crucial to product development. When you deeply understand the needs and pain points of your user as they use your product, you can deliver a better experience. This research method raises the questions: What kind of User Experience do you want to give your customer? How are you willing to improve your UX to keep their business?
In an ideal world, the correct answer to "When should you conduct a usability test?" would simply be "early and often.” But that's not always possible so testing during these two key moments below will do. Every project's development is unique but if you conduct tests during the moments we’ve described, you can be sure that you're creating designs that are validated by your user.
At a minimum, the baseline for testing is 5 users. This is because testing 5 individuals can help reveal at least 85% of the problems in an application. Once you cross the threshold at 5 and begin to add more test participants, the number issues that you’ll uncover reduces. This is because you’ll start to hear the same feedback from the different users.
With the magic number of 5, you’ll still need to consider your different personas or user segments. This is dependent on how many personas you have for your target market — for example, you have a site that will be used by both children and parents. Then the two groups of users will have sufficiently different behavior so it becomes necessary to test with people from both groups.
In testing multiple groups of users, you don't need to include as many members of each group as you would in a single test of a single group of users. The overlap between observations will ensure a better outcome from testing a smaller number of people in each group.
To conduct a Usability Test, you will need:
The Test Facilitator administers the tasks to be done and observes the Participant as they interact with the product and accomplishes the tasks. The Test Facilitator does this based on a prepared research guide that gauges the Participant’s experience. The Test Participant is expected to “Think Aloud” while accomplishing the task, providing feedback on the product. This is done, as much as possible, without the Facilitator’s direct instruction so that the Participant’s experience can be observed without interference.
Running a Usability Test might seem daunting at first but with adequate preparation, you and your team can accomplish a successful test. Breaking it down in stages will help you prepare what’s needed before the test so you can implement the test, analyze your data, and synthesize actionable steps with ease. It begins with preparing a Research Plan. This research plan will establish the scope of work to be done. It will help to keep the team aligned when implementing the research.
Step 1— Identify Business + Research Questions
For example, “What new features can we add to our app to entice our younger market to get insurance?”
This business question will prompt you to explore more questions such as:
These inquiries are now your Research Questions. Research Questions should provide you enough information to answer your big Business Question. By having well-defined questions, the User Research is given direction and a goal.
Step 2 — Set Success Metrics
This can be directly related to your Research Questions or geared towards your overall Business Question. For Usability Tests, we evaluate the effectiveness, efficiency, and ease of use of products. This is based on the ISO 9241-11 definition of usability: ‘extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’.
Each of these factors is gauged to determine if the product can be considered usable.
80% of the participants are able to finish applying for life insurance online.
Step 3— Determine Tasks to Test
Red Routes can help your team:
Here is a ready-to-use template of the Red Route Matrix that you can build on for your product. The top right is your most frequently and widely used feature, while the bottom left is your least used and more narrowly tailored feature.
In developing your product, you can use this Matrix to prioritize which features to build and test. You don’t need a fully working application to test your product. You should, however, make sure that what you are testing has been created according to the red route priority.
Step 4— Create Task Scenarios
Once you’ve decided on your tasks, you can write your task scenarios. These scenarios are meant to paint a picture, not guide the user on what to do. Here’s an example of a task scenario for getting an insurance product:
Task Scenario:
As a first-jobber, you were very excited when you received your 13th month pay. You figured that the adult thing to do would be to invest on something long term, like life insurance. You heard from your friend that you can apply for life insurance online through Insurance Company X.
Task: Get a basic package for life insurance online
Step 5— Write a Discussion Guide
Once you’ve finished identifying the scope of the research, you can now write your discussion guide. Your discussion guide is a script you’d follow when you conduct your tests. This helps ensure that the tests are standardized— you need to be able get all your participants to answer the same questions and accomplish the same tasks. This guide will:
Tip: Follow the sequence of questions and tasks but remember that a research guide is linear. Participants may take things in a different direction so adopting flexibility is key.
We recommend using Steve Krug’s Research Test Script as a guide for building your discussion guide. This example follows this structure:
Once you have the bulk of your Research Guide done, you have to determine what tools to use for your Usability Test. Although In-person Usability Testing is the preferred method, we have since adjusted our tools since the start of quarantine. Here are two tools you can use for Remote Usability Testing.
Basic Level: Get into Hangouts with your participant.
Google Hangouts is a platform that allows users to converse, present documents, and share device screens to users in the meeting. You may schedule meetings and send customized invitations to participants. These are accessible through email.
How it works: Have the participant open the prototype on their desktop and ask them to share their screen while they interact with the prototype
Advanced Level: Try your hand at Lookback
Lookback is a usability testing platform that assists in carrying out remote moderated and unmoderated tests through different devices. It allows test moderators to invite participants to a session, where they can share their device screens. Doing this allows both parties to see the participant’s journey and interactions with the product.
How it works: With microphones and cameras enabled, Lookback automatically records each session and uploads these to their cloud. This allows test moderators the opportunity to review the usability test after the session.
Important Note: Your participants may not be familiar with the aforementioned platforms, so it is best to brief them in an email about the tools they’d need to have before you start your scheduled session. In addition to this, let them know beforehand that they would need to have their microphones and cameras turned on for the session.
For optimal preparation, you may opt to allocate time for you and the participant to test your equipment prior to the test. This is so both parties are able to gauge whether internet connection and equipment are functional and ready to run a usability test.
