All you need to know about usability testing

What is Usability Testing?

Before we dive deeper into the discussion, let’s first define what usability testing is. It’s an examination of how real users interact with a product in order to identify its strengths, weaknesses, and issues that need fixing urgently. Usability testing revolves around quality improvement, which is based on five components:

• Learnability – how easy or difficult it is for users to complete a task the first time they use your solution
• Efficiency – how efficient is it to perform a task
• Memorability – how simple is it to return to using your product after not using it for weeks or months
• Errors or bugs – how quick and easy error recovery is
• Satisfaction – how enjoyable your design is to use

You might be wondering if it is worth conducting a usability test? We’ll get to that right now.

The Main Benefits of Usability Testing

Conducting usability testing regularly will bring you significant benefits, such as:

• Enhanced user experience – when it comes to product selection, customers have plenty of options. A good CX is absolutely necessary to prevent app switching.
• Improved conversion – shortening the path to purchasing will have a positive impact on boosting your conversion rate and can help project managers reach the target audience.
• Revenue increase – usability issues can result in revenue losses.
• Showing commitment to a great UX – working towards providing a seamless user experience and design process will show your customers that you care about them.

Different Types of Usability Testing

There isn’t just one type of usability test, but various. Let’s take a look at the main ones to help you choose the most suitable option for your business.

1. In-person usability testing

   This occurs when you’re physically in the same room as your participants. There are different in-person usability testing methods which you can use, including:

   • Lab testing introduces a controlled environment where testers can be evaluated based on target market characteristics. You can conduct them in-house or in a 3rd party facility.
   • Guerilla testing is about volume and involves testing with as many random people as possible.
   • Observational usability testing - the participants test your product in a quiet location while you act as an observer but without being physically present. You may use screen recording or video recording to observe users’ behavior.
   • Eye-tracking - see what parts of the user interface get the most focus and which parts are overlooked.

2. Remote usability testing

   As the name indicates, this takes place with you and your participants are in two separate locations. Here are the most common forms of remote usability testing:

   • Phone interviews are one-on-one sessions where researchers interview participants over the phone. Both parties can ask questions.
   • Digital card sorting - users receive digital cards and are asked to sort them in the way that makes most sense to them. This helps with designing the right information architecture.
   • Session recordings - using screen recording tools like LiveSession enables you to remotely test ideas and observe how users interact with your interface, helping you identify any areas that require improvement and create a seamless user experience.
     
     
     

3. Exploratory testing

   Users are given the opportunity to freely explore the product so you can check how they interact with the interface. Testing happens naturally and the evaluator has no influence over the participants.

4. Comparative testing

   This is used to compare your product against two or more solutions. It can help you generate new ideas, features, and functionalities.

5. Assessment testing

   It lets you conduct a general usability test. Usually this comes after exploratory testing and helps you assess how your design impacts usability.

Website Usability Testing - How to Do it Right

Now that you know what usability tests are and what methods you can use to conduct them effectively, it’s time to talk about website and mobile usability testing.

The former is about checking how visitors interact with your website. It’s done to improve the retention rate, reduce development process steps and support costs, assess what information is missing from your website, and boost sales.

How to Conduct a Website Usability Test

There are a number of steps that you should take to ensure a high effectiveness of your website usability test.

Step 1: Plan for your test

Decide on the scope – figure out what parts or aspects of your website you would like to evaluate. Is it the copy, navigation, or maybe certain apps?

What’s the purpose? – before you begin your test, you must decide on its goal. Is it to verify whether or not any information is missing from your website or to check if it’s easy to move from one page to another?

Schedule and location – are you planning to conduct your test remotely or in-person?

Participants – who will you invite to the tests, and how many participants will you need? Try to match the participants with your target audience.

Scenarios – you must prepare scenarios and questions to hand out to participants.

Source: Unsplash

Step 2: Run the test

You can select from two methods:

Unmoderated remote testing – great if you have a limited budget because no physical equipment is required to conduct the test, you don’t have to limit yourself in terms of participants’ locations, and there are no timezone constraints.

Moderated or over-the-shoulder testing - you can closely observe the participants and notice their expressions, let them ask you questions if tasks are unclear to them, gain feedback instantly, and can even ask participants to perform extra tasks if necessary.

Step 3: Analyze the findings

In order to put the data into good use, you need to analyze your results. For this purpose, you should:

• gather all of your data and write down the main facts
• segment the problems that were identified into common categories like login issues
• assign similar issues to the appropriate categories
• list all issues and prioritize them in line with their business or UX importance

A Few Tips on Website Usability Testing

• Number of participants – according to Jakob Nielsen, the precursor of website usability, it’s best to enlist no more than 5 participants.
• Don’t judge too quickly – don’t implement any changes unless you notice a clear pattern of similar issues
• Record your testing session – recording your sessions will help you keep track of the whole process. For this purpose, it’s worth using a session recording tool like LiveSession.
• Test early and frequently – the earlier you test the smaller your development costs will be and easier errors will be to fix.
• Run a website usability survey – including a survey on your website can help you boost customer satisfaction. You can ask your visitors not only whether or not they found what they were looking for, but also what should be improved.

Now, let’s take a look at how usability testing is performed for mobile devices.

What Is a Mobile Usability Test and Why Do You Need One

Understanding how to properly conduct usability testing for mobiles is as important as it is for desktops. In fact, it’s predicted that by 2025 nearly 75% of the world will primarily use their smartphones to connect to the Internet.

On the other hand, the competition for mobile users is already fierce, since there are jointly over 5 million apps in the Google Play and iOS App Stores. Given these numbers, your mobile user experience should be one of your top priorities.

The Advantages of Mobile Usability Tests

Thanks to mobile usability tests you will be able to, among others:

• Find areas that require improvement (for example, bugs or errors). IIn order to do this you can, for instance, use a session recording tool like LiveSession to observe how your users interact with your website on their mobile devices.
• Optimize mobile development costs. If you understand which features are most popular among your users or which areas require critical improvements, you’ll be able to spend your development budget wisely.
• Improve your retention rate. By understanding your users’ behavior and goals, you’ll be able to prevent them from ceasing to use your app.

Source: Unsplash

Below are the steps you should take if you’d like to conduct usability tests for mobiles.

Mobile Usability Testing Process

Here are the five steps you’ll need to take in order to ace your mobile usability tests:

Step 1: Define your objectives

As in the case of the aforementioned website usability test, you’ll need to define your goals. We advise you to schedule a team meeting to discuss what you already know vs what you want to find out about your users and/or app. The result should be a list of areas for you to prioritize in the test.

Step 2: Make a list of tasks

Now that you know which topics you want to collect insights on, build your questions around them. Organize tasks around various scenarios to test the most important features or use cases, inside out.

Step 3: Create your documentation

Before you proceed, write down all of the goals and tasks established in Step 1 and 2.

Step 4: Select the right participants

Don’t rush through this step. Make sure to choose the right participants who are representative of your users. Only then will the mobile usability test provide relevant results.

Step 5: Choose the methodology and run the test

Decide if you’d rather conduct a remote or in-house test and whether it should be moderated or not (we mentioned these types earlier). As users complete the tasks, write down any observations. Also, collect insights from them after they’ve completed the test. This data will be the basis for your unmoderated usability testing report, which we will discuss later on.

You can read about the entire mobile usability testing process in a dedicated section.

Top Usability Testing Questions You Need To Ask Your Audience

Now that we’ve reviewed what usability is and what it looks like for both websites and mobiles, let’s proceed to one of the most important elements of these tests – asking the right questions. In fact, collecting qualitative and quantitative feedback from your users is as important for usability testing as observing how they interact with your app or website.

That being said, it’s time to have a look at some types and examples of pre-prepared usability testing questions.

4 types of usability testing questions

User testing questions can be divided into four groups – screening, pre-test, in-test, and post-test.

Screening usability questions

Screening questions will help you pre-qualify your users for your UX testing. When you work on your usability testing, you should write down all the criteria that an ideal participant should meet. Among others, you should ask about:

Among others, you should ask about:

• Demographic data such as age, gender, occupation, and level of education or firmographic data such as company size, industry, etc. This information will help you e.g. determine if you’re targeting the right occupation for a given company size.
• Tech-literacy, i.e. how easy it is for your users to operate certain devices. Asking a such question will help you e.g. choose only those people who have high tech skills.

Pre-test usability questions

Pre-test questions will help you further narrow down or group your users. These include:

• Behavioral questions - for example, “how often do you engage with this (or similar) product(s)?”
• What do you use the website or app for? – to understand separate use cases
• Which features do you use most? - to identify users who are already familiar with your product vs those who are new to it
• What problems does this product help you solve? – to understand and group user goals

In-test usability questions

These questions should be inquired as users engage with your product or website. Among others, we recommend that you ask:

• When you log on, what’s the first thing you do? – to understand user motivation
• Do you like the interface? Is it easy to use? – to collect general impressions and nail down any UX/UI flaws
• Why did you decide to leave this page? – to understand exit intent

Post-test questions

Last, but not least, you should also question your users after they’ve completed the usability test. Ask your testers about their impressions, focusing in particular on the following:

• Given your experience, how frequently would you use this product/feature? – to find out how valuable the product is for users
• What were your feelings about the instructions? – to optimize future user testing sessions
• How likely are you to recommend this website or app to others (on a scale from 0-10)? – this so-called Net Promoter Score question will help you evaluate customer loyalty levels and understand key issues
• Do you know anyone that would like this product? – to find out how you can better determine your ideal customer profile

You can find a full list of recommended usability testing questions in a dedicated chapter of this guide usability testing questions in a dedicated chapter of this guide.

How to Become Great at Usability Reporting

With all the work you’ve put into organizing your usability test, you’ll now also need to ensure that your findings are put to use by relevant product teams. There’s no better way of doing this than by creating a thorough usability testing report Below, we review what usability reporting is, what it should include, and why it’s indispensable to the overall testing process.

What a Usability Testing Report Is

In the most simple terms, a usability test report is a document that you prepare after evaluating all of the feedback and insights from sessions with users. These include, among others, your own notes and observations, any evidential data (for instance, session recordings), as well as quantitative and qualitative feedback from participants.

Don’t worry if this sounds like a complicated process, as we will provide tried-and-tested usability report templates. You can follow the structure below:

Executive summary

The goal of the executive summary is to quickly onboard anyone who picks up your report to understand what it covers and what the key findings were. A paragraph will suffice – you’ll go into detail in the next sections.

Background summary

This section should be written in a way that allows anyone who hasn’t been involved in the usability test to quickly understand why you decided to run one. Include:

• what the test objectives were
• what products/features were tested (for example, website or app)
• who the testers were

Methodology

Next, explain what methodology and tools you applied. Did you use user session recordings, or perhaps you performed lab tests? This is the section where you should also specify all of the questions that you asked your participants.

Audience

Describe your participants as thoroughly as possible. Make sure to include their demographic/firmographic data and why you decided to recruit them in the first place.

Test results

Create a detailed description of the results. To improve readability, we highly recommend that you include a table that clearly specifies all key data, including task completion rates, the overall number of participants, and types of assignments, etc.

Findings and recommendations

Make a bulleted list of findings and a separate list for recommendations. The results should be tangible, i.e. this is no place for assumptions, only verified theses. You can also create a list that segments your findings via user scenarios.

In this guide, we’ve dedicated an entire section to usability testing reports - make sure to take a look.

Usability Testing Tools to Help Optimize Your UX

While usability testing might seem daunting at first, there are a bunch of usability testing tools that will make the entire process a lot easier. Here are the ones that we recommend using:

1. Mixpanel

   It’s your go-to solution if you want to gain a deep understanding of how users interact with your product, how and when they convert, and what changes you must incorporate to retain your users.

   

Source: Angel.co

2. Google Analytics

   A tool, known by every marketer, that lets you check your conversion rates, find your best-performing landing pages and those which need improving, discover traffic sources, as well as gain insights about your visitors. You can create detailed reports that you can easily share with your team.

   

   Source: Neil Patel

3. LiveSession

   LiveSession is a session replay software solution that lets you check how visitors engage with your website and spot any pressing issues that prevent them from converting. As a result, you can eliminate any bugs and roadblocks to improve the overall UX. LiveSession offers a free trial, so be sure to check it out.

   Create a seamless user experience and improve your conversion rate. Try LiveSession today.

   Test all LiveSession features for 14 days, no credit card required.

   Sign up free
   

   Source: LiveSession

4. Unbounce

   It offers a simple to use, drag-and-drop landing page and pop-up builder that doesn’t require any coding skills to use. It’s great for A/B testing, but what makes this tool really unique is its AI-powered Smart Traffic and Unbounce Conversion Intelligence features. They give you access to insights that are necessary to understand how users engage with your product.

   

   Source: ThemeForest

5. Usability Hub

   Usability Hub is a user testing and research platform that lets you gather feedback from real people to ultimately improve the UX of your app or website. It includes features like first-click tests, design surveys, preference tests, and many others.

   

   Source: UXpro

If you’re hungry for more information about usability testing tools then check out this article, where we discuss some more solutions along with their benefits.

Summary

As you’ve just seen, usability testing is essential if you want to outpace your competitors and consistently provide your users with a great UX. By incorporating usability testing as a continuous process, you’ll be able to improve your customer satisfaction levels, adjust your product to an ever-changing market, and optimize product development costs, among other benefits.

Now, if you truly want to master usability testing, we highly recommend that you read the remaining chapters of this guide. This section was just introductory reading, so there’s still much more to uncover about effective website and mobile usability testing, creating useful, comprehensive usability reports, drafting the right questions for testers, and more.

Chapter 1

What is usability testing?