Everything You Need to Know About User Testing

Reading Time: 18 minutes

User testing is a technique to quickly identify issues with your product that are occurring with your visitors and customers daily, allowing you to make quick adjustments and improve the overall user experience. 

It’s easy to underestimate the importance of the user experience and how it can greatly impact your conversion rates.

This article will provide an overview of the user testing process, how to conduct it, and the tools you can use to get feedback from potential users.

What is User Testing?

User testing is the process of evaluating a product or website by having real users interact with it to identify usability and design issues. 

It involves observing people interacting with your product or prototype while performing specific tasks and asking them questions about their experience. This helps you gain insight into how they see, think and feel about their actions.

The primary goal of user testing is not to create the perfect product or service but to build products that people want to use.

User testing can be applied at various stages of design, including:

• Conceptualization: The initial stage of testing an idea or concept for usability
• Prototyping: Creating multiple iterations of an idea as it’s being developed.
• Testing: Testing prototypes with real users to find flaws and improve usability before going live with the final product.
• Re-evaluation: Continue observing how users interact with your product after launch and make changes accordingly.

User testing can be done with prototypes, wireframes, or paper sketches. The important thing is not the medium (it can be anything from paper prototypes to ready-made software) but the way you conduct it. 

In most cases, it involves having an interview with real people who have never seen or heard about your product before. Then you ask them about their experience using this product and record their answers on video (or audio).

Why is User Testing Important?

Ask many designers, developers, and those involved in product development; they’ll tell you the product they’re working on is amazing.

This view is biased because it doesn’t take into account user experience.

To design a product that will bring maximum value to the user, the product experience has to be tailored to users, and this kind of feedback can only be gotten from users.

Quick note, user testing is a type of user research.

Listed below are some reasons why user testing is essential;

1. By commencing early-stage testing to identify flaws, you can revise the designs before full implementation which will be costlier for the business.

2. User testing is the surest and fastest way to get unbiased feedback from actual users that can’t be swayed and denied by decision-makers.

3. You’re more likely to get a higher conversion rate when you launch a product which you’ve gotten user feedback for than a product you didn’t conduct any such test.

4. User testing a product before launching means a reduced load for customer support because users can complete tasks alone.

5. The user testing process helps you identify valuable opportunities. In the course of actual testers using your product, you could get feedback asking for specific feature updates and preferences; this could go a long way in giving you a competitive advantage.

User Testing Methods

There’s more than one way to conduct product testing, and we’ve compiled a list below for you.

1. Surveys

If obtaining accurate quantitative data and insights about your product is your goal, surveys are a great way to go about it. 

With surveys, you can tell testers to complete a survey about your new product or an updated feature.

A significant advantage of surveys is that you create the questions, so you can ask broad questions or be very specific.

With surveys, you can perform user testing on a large scale because it’s sent to a large pool of user testers.

Also, surveys are a very inclusive product testing method because they can be done from anywhere, via any device.

When To Use Surveys

Surveys are the way to go if you want a lot of feedback in a short time. Surveys also allow you to dive deep into asking product-specific questions, providing valuable insight and feedback.

2. Focus groups

In this context, you get a group of real live testers (10 – 12 people) into a room, and you discuss various attributes of your product. This group session is led by a designer, developer, or product manager who asks the questions to ensure you get the desired feedback.

This group discussion usually lasts an hour or more and it focuses on the user’s concerns about a product, its features, and attributes that they’ve been made to use for a while.

When to Use Focus Group Testing

Focus groups are great for understanding what your users hope to gain from your product. With focus groups, you don’t just get verbal feedback; you can also pick up body cues that let you know how satisfied or dissatisfied testers are with your product. 

3. A/B Testing

With A/B testing, you split your users into different groups, show them different variations of your product, and record the one with the most desired conversion. 

This is a crucial product testing method because it ensures you’re serving your users with the best version of the product.

When To Use A/B Testing.

A/B testing is ideal if you want to get feedback about which version of an aspect of a product is most popular among your target audience.

For instance, you can create a page variation where the call-to-action wording is different from the control and monitor which one has more clicks and interaction. 

4. Usability Testing

Usability testing is when you provide a tester with your product or a new feature, and you want them to test for its usability.

Usability deals with a product’s ease of operation, usefulness, and utility.

Oftentimes, usability testing is said to be user testing and vice versa. In many ways, they’re not wrong, but that’s not the whole truth.

Difference between Usability testing and User testing.

User testing covers the entire range of user experiences a customer has with your product, prototype, or feature. This includes all their perceptions, emotions, preferences, responses, and behaviors in response to that item from the moment they obtain it to the moment they stop using it.

Usability testing is a method of user testing that focuses on the way that — and the extent to which — a customer can use your product, prototype, or feature to accomplish a specific goal. 

This plays a part in the user experience but is not the entire experience.

Moderated Vs. Unmoderated Usability Testing

Moderated usability testing means you have someone on your team sit down with live testers, explain what they want them to do, and get a live response as the testers are using your product.

Unmoderated usability testing means the opposite. There’s no sitdown with the tester. Most likely, you’ll get two videos. One with the user working with your product while the other video comes with the feedback.

Moderated usability testing is good if you want to watch the tester use your product and identity issues in real time. You also get to pick up non-verbal cues.

Unmoderated usability testing is great if you want to test your product with many testers from different locations.

When To Use Usability Testing

Usability testing is great at the beginning of a product’s development and mid-design phase.

This way, you save a lot of time and resources by ensuring your product works as it should for your customers.

User Testing Tips and Checklist 

It might seem simple, but getting your participants to use your product is one of the most important parts of user testing. 

Here’s a checklist to consider before you get started:

1. Decide what you want to test

To make the most of user testing sessions, ensure you’re clear on what you’re trying to learn before you start.

This means deciding what you want to test before user testing. Don’t just bring participants into a room and ask them questions about whatever comes up. You want to put your users in situations that will give you the information you need, which usually means setting up scenarios where they perform particular tasks or interact with specific features or pages.

For example, if you want to know how easy it is for people to find the registration page on your website, try creating a fake account and see if they can find it without help from anyone else. Or, if there’s something specific about your design that could be confusing for users, test that particular page or feature by asking them questions about it directly.

If your product is new or in development, you should try some variations of the existing design. See if any of them work better than the current version. If so, continue testing until you’ve found the best solution.

If your product is already live and tested by users (or at least by a few), then start from there. Look at the results from previous user tests or from analytics data, and see if there are any patterns that can inform your next round of testing.

For instance, the leading online learning platform Udemy used a remote unmoderated user testing method to test its popular smartphone app. They wanted to find out when, where, and why Udemy users favored the mobile app over the web version.

The researchers had participants create short videos showing their locations and activities during the learning process. The researchers found that instead of using the app while traveling or commuting, users preferred to use it while at home or in a coffee shop.

They used the key findings for the following year’s product and feature development.

This is what Udemy’s mobile app looks like nowadays:

 

2. Choose your target audience and find participants 

If you’re going to test with users, you need to know who those users are. These days, it’s so easy to get feedback from a wide range of people and sources, but it’s still important to define a target audience that you want to focus on.

The first question is: Who do you want to test with? If you have a website or mobile app, this might be obvious. But it can be trickier if you’re working on something more abstract, like a service or product launch.

An excellent place to start is by looking at existing customers — what kinds of people buy from you? Try breaking down these groups into demographics like age, gender, and location (as well as anything else that’s relevant).

Then ask yourself: What type of person would benefit most from using this new product or feature? 

For example, if you’ve launched a new e-commerce site selling women’s clothing online, maybe it makes sense to focus on young women between the ages of 18 and 35 who live in major cities such as New York or London. On the other hand, if your service helps companies manage their finances better, then perhaps you need to seek answers from middle-aged men with families.

Once you’ve chosen your target audience, it’s time to recruit participants for user testing. You can do this in several ways:

• Use social media (Twitter and Facebook) — If your product has an online presence, ask people if they’d be interested in helping out with user testing by posting on your social media accounts. If not, create an email list using tools like MailChimp and send out an email asking people if they’d be willing to participate in future tests.

• Make flyers — Print up some flyers and post them around town in places where people congregate, such as cafes or grocery stores. Don’t forget to include contact information!

• Hire someone — If you have enough budget, hiring a recruiter will allow you to reach more people and get responses faster than doing it yourself.

3. Make a recruitment server

A recruitment screener is a questionnaire that helps you identify potential participants who match your research criteria. The purpose of a screener is to reduce the number of candidates screened out so that you can focus on those who meet your criteria.

Screens must be easily completed to avoid losing potential participants. If a screen takes too long, some people may be discouraged from completing it, reducing the pool of potential participants.

​​A recruitment screener should include questions about:

• Demographic information (e.g., age, gender, education level)
• Cognitive ability (e.g., reading comprehension, math skills)
• Computer skills (e.g., typing speed)
• Physical abilities (e.g., dexterity) 
• And knowledge of the topic area of interest (e.g., familiarity with new technology).

Here’s an example of employment questions you can ask if you wish to screen people with some level of familiarity with a particular industry or leave out those working for competitors.

                                (Source)

Here’s another example of questions you can ask if you want to test with beginners, experienced users, or some combination of each about how familiar they are with a given product.

            (Source)

4. Write your user testing plan 

As you begin testing, you will want to keep careful notes of what happens. This is where your test plan comes in handy. 

A test plan template for user testing is an integral part of any project. You can use it to record the results of each test session, including user comments and feedback.

In addition, the test plan helps you to:

• Identify the purpose of your test and its goals.
• Provide clear details about what you are going to test, how you are going to test, who will be responsible for the task, etc.
• Prepare yourself before conducting the tests.
• Keep track of all the tests performed during each session and provide a detailed report after concluding all sessions.

A good test plan should include:

• The goal of the test session (for example, finding out how easy it is for users to find information on your website).

• The target audience (for example, people who visit your site once a month or more).

• The tasks that each user will perform (these can be simple things like clicking links or filling out forms). Make sure these tasks are relevant to the goal above and are not too difficult for a beginner user to accomplish.

• How many users will participate in each session? The more people you have testing at once, the better chance you have of getting reliable results from real people who represent your target audience. On the other hand, if you have fewer testers than needed for an accurate sample size, then it will take longer than expected to complete all tests and get results back from them.

In addition to helping us understand what we need to do as testers, a good test plan can also be used as an internal tool for other departments in your company who may need more information about what tests are being run and why. 

It can also be used as an internal tool for measuring testing effectiveness after some time has passed since the tests were run so that you can see which areas of your product were most heavily tested by real users versus those that were not tested at all.

5. Create your prototype 

Now that you have an idea of what your product will do, it’s time to create a prototype. It’s easy to get carried away with the design and forget that it needs to be tested by real people.

Here are some tips to help you create a prototype: 

• Create a wireframe or mockup of your product. This should include the main screens and functions of your app or website. You don’t need anything fancy here, just enough to show how the product works and what it does.

• Add any additional details (e.g., images) that will help make your prototype more realistic during user testing sessions with stakeholders or potential customers.

• Create an environment where users can interact with the prototype in a way that resembles real-world use cases: If they need to log in to an account, create one and give them fake credentials; if they need to type text into a form field, give them a keyboard; if they need to scroll through a list of items, add some fake content; etc.

In addition, if you have access to real devices (instead of using paper prototypes), then you can test on actual hardware as well as software platforms such as mobile phones or tablets.

For instance, in the usability study of Halo: Combat Evolved, the first video game in the Halo franchise, the Halo team formed a focus group of console gamers to test their game’s prototype and determine whether they had fun while playing the game. 

In case they did not, the team wanted to discover what deterred them from having fun. 

When game designers placed players in a large outdoor environment filled with enemies, they thought they would dash toward the enemies, sparking a huge battle.

However, the players had a different strategy. They did not put themselves in danger by moving closer to the enemies. Instead, they kept a maximum distance from them and shot from afar. 

Though a safe and effective strategy, it was monotonous and boring for the players.

To entice test participants to enjoy battling up close, the user researchers made changes to the size and color of the aiming indicator in the middle of the screen. It notified the players if they were too far from their enemies. 

Have a look at the finalized aiming indicator in action:

The tests confirmed these changes to be effective, as most of the players now engaged in combat from up close.

6. Create your test plan document 

A test plan is a written document that describes how you will conduct user testing. It is also a way to organize and document all of your research activities. It should be clear, concise, and easy to read so that anyone can understand it.

Key elements of a test plan document include:

• Test scope – includes who you’re testing with, what you’re testing on, and when you’ll conduct the tests.

• Goals & objectives – Describe why you are conducting this research and what you hope to learn from it.

• Key metrics – outlines how you’ll measure success during and after each round of testing (e.g., number of participants recruited per session).

• User profile – describes each person who will participate in the study (e.g., age range, technical proficiency level).

• Session details – include information about where each session will take place (e.g., location), who will conduct each session (e.g., facilitator), how long each session will last (e.g., duration), and how many participants will attend each session. 

Your test plan will also include information such as:

• When will you run the tests (e.g., at what time of day)?
• Where will you run your tests (e.g., which venue or location)?
• What kind of users or people will you recruit for each iteration?
• How many people do you want to recruit for each iteration?

User Testing Questions. 

When you’re conducting user tests, it’s important to ask questions that help you get at the heart of why users are performing certain actions. The following list of questions can help you frame your tests and explore different aspects of your application.

Screening Phase: 

The goal of the screening phase is to make sure that the user is qualified to participate. This helps ensure that you are only inviting people into your study who will be happy with the experience and who will be able to successfully complete it.

Here are some screening questions you can ask: 

• What is your age? 
• How long have you been using this app? 
• What is your job title? 
• What is your company size? 
• How many employees do you have? 
• Do you feel like you can trust the information the app provides? Why or why not? 

Pre-test Phase: 

Before you start to test, it’s important to get a sense of what your users think about your product. This will help you improve their experience with your product and make sure they have the right expectations.

Here are some questions that can help:

• If we were to invite you back to test more features in the future, would you be interested? (Yes/No).
• What do you like about this product so far? What don’t you like?
• What devices do you use to shop? 
• Are you comfortable with online shopping? 
• How much do you know about our site? What was your first impression of the site?
• What is your current level of interest in this product? (Very interested, somewhat interested, not very interested).

Test Phase: 

The test phase of the product development process is designed to collect feedback from users about their experience with the product. Test results are used to improve or evolve the product and make it more attractive to potential customers. 

To help guide your approach during test phase user testing, here are some useful questions:

• How long have you been using [this product]?
• Have you used it in the past? If yes, when was the last time, and how often do you use it now?
• What do you like about [this product]? (If they say nothing) Why do you use [this product]?
• Were there any problems with our homepage that prevented you from finding what you were looking for?
• How easy was it for you to find what you were looking for on our homepage?
• Did anything surprise you when visiting our homepage?
• Have you ever tried another [product] before using this one[?] (If yes) How would you compare this to other products that offer similar functionality as theirs? (If no) Would you consider using a different [product] if it had more of the features that yours has?

Post-test Phase: 

After users have completed the test, you should consider asking open-ended questions about their experience. These can help you get better insights into what they thought of your product and what they did (or didn’t do) during the test.

Here are some examples of post-test phase user testing questions:

• What was your overall impression of this experience?
• What was the most important thing you learned from this experience?
• How did you feel about the product during each step of the task?
• How would you compare [your product] to [competitors]?

User Testing Tools

User testing can be done in many different ways, but there are some tools that can help you save time and get more reliable results. 

Here are some of our favorites:

1. UserTesting

UserTesting is a startup-friendly platform that lets you create tests with just a few clicks. You can create your own custom tests to see how the user interacts with your product and what their experience is. You can also share insights in emails or integrate with applications such as Jira, Trello, and Slack.

To design your tests, UserTesting gives access to tools such as Adobe XD, InVision, Sketch, Figma, and others. You just need to install their plugins – and you’re good to go! 

The platform also offers A/B testing capabilities and works with any website or mobile application — including WordPress sites!

Other key features include: 

• It offers live video testing through desktop, mobile, and tablets.
• You can tag clips, highlight reels, and take notes in the video recorder.
• It also integrates with Qualtrics, Adobe Xd, Outlook Calendar, and Google Calendar.
• It lets you create tests for mobile prototypes, unreleased and in-store apps, AR/VR experiences, and more.

Pros:

• Quick feedback from users
• Easy-to-use and easy to install
• Customer journey mapping is available
• Multiple templates are available to create tasks
• Provides quantitative and qualitative user feedback
• The video playback option is available
• Guided support
• Allows unmoderated usability testing

Cons:

• International recruitment or testing not available
• Limited features available with cheaper plans
• Folder organizing is not available for tests

Pricing: 

• Free trial available
• Three paid plans are available, including Essentials, Advanced, and Ultimate
• Pricing is not available on the website

2. Optimizely

Optimizely’s visual editor is easy to use and allows you to create A/B tests without writing code. It gives you insight into what features users find most valuable and how well they understand them. 

Optimizely also comes with heatmaps and user feedback tools that make it easier than ever before to see where users struggle with your site or app.

Other key features include: 

• Audience targeting and visitor tracking.
• Test scheduling is available.
• Integrations, such as Shopify Plus, Magento, Delighted, Coreware, Instagram, WordPress, Office 365 Calendar Connector, Cache Buster, OpenPay, and more.

Pros:

• It enables behavioral targeting 
• It’s an easy-to-use and intuitive platform

Cons:

• Preview mode has some errors
• App integration is difficult

Pricing: 

Optimizely offers five plans, including Welcome, Content Management, Web A/B Testing, Feature Management, and Commerce. It offers a free trial for its Welcome Plan. 

However, the pricing is not revealed on the official website. You’ll have to request pricing for each of its plans. It also offers add-ons for each plan for additional features and upgrades.

3. FigPii

FigPii user testing software is a simple, easy-to-use tool for running online user tests.

It’s useful for both UX designers and product managers who want to get feedback from users on their website or mobile app.

You can use it to get actionable insights about what people think about your design or web page, how they use it, and how you can improve it.

FigPii analyzes user behavior using tools like: 

• Session recordings 
• Heat maps 
• Purchase events 
• Interaction data 

FigPii will suggest changes to your site that can help you optimize for conversions. These suggestions are based on concrete data, and FigPii will continue to learn as it tracks user behavior on your site and makes further suggestions.

Other key features include: 

• You can either record and watch every visitor to your website or just those who land on a certain page using session recording.
• It provides 16 filters to let you focus on only the sessions you desire. Filtering options include session length, landing page, exit page, user type, tags, and others.
• You can conduct a poll on a single web page of your website using FigPii.
• You can design on-site surveys for people accessing your website through a certain device.

Pros:

• Easy-to-use features
• Competitive price
• Great AB testing functionality

Cons:

• You might experience bugs during mobile sessions

Pricing:

• Starter – $49.99 /month (10,000 Visitors Per Month)
• Small – $99.99 /month (30,000 Visitors Per Month)
• Medium – $199.99 /month (60,000 Visitors Per Month)
• Large – $339.99 /month (Custom plan)

4. Hotjar

Hotjar is a user testing software that allows you to collect feedback from your website visitors. It provides you with heatmaps, visitor recordings, form analysis, and surveys – all in one place.

With Hotjar’s User Experience Testing Software, you can:

• See how people are really using your site. Optimize the experience of your users by understanding what they do on your site.

• Get feedback from real users before launching. Test new features with real users before launching them on your live site.

• Get instant feedback after changes have been made to your site. See the impact of the changes in real-time with Heatmaps and Session Recordings.

Other key features include: 

• You can see user actions such as mouse clicks, movement, and scrolling.
• Seamless integrations with third-party apps like Slack, Segment, Hubspot, Google Optimize, Optimizely, Omniconvert, Zapier, and more.
• You get access to pre-built survey templates and unlimited feedback widgets.
• There’s no storage limit for feedback.

Pros: 

• Instant feedback is available.
• Free trial available.
• Competitive pricing.
• Easy installation and easy-to-use

Cons: 

• Session recordings have limited filters.
• Some online reviews suggest that setting up forms is a bit difficult.
• You cannot label the recordings.

Pricing: 

Hotjar offers four plans under the Observe section, which allows you to visualize user behavior with Heatmaps and see what your users see with Recordings.

• Basic – Free (35 daily sessions)
• Plus – $31/mo (100 daily sessions)
• Business – Starts at $79/mo (500 daily sessions)
• Scale – custom (Unlimited daily sessions)

Then you get four plans under the Ask section, which gives you access to the Feedback and Surveys feature.

• Basic – Free (20 monthly responses)
• Plus – $47/mo (250 monthly responses)
• Business – starting at $63/mo (500 monthly responses)
• Scale – custom (Unlimited monthly responses)

1. Lookback 

Lookback is a video-based user testing software that allows companies to get immediate feedback from their potential users. It features virtual observation rooms, team chat, timestamped notes, shared insights, and more. 

It’s a great tool for startups and SMEs to validate their ideas before developing a product or feature and for established companies to ensure their products meet users’ needs. 

Key features include: 

• You can talk to your test participant face-to-face.
• Sessions are recorded in the cloud.
• You get touch indicators on taps and gestures, audio, face cam, etc.
• You can set up a sequence of tasks.
• Desktop, iOS & Android platforms available for testing
• You can take notes in the player.

Pros:

• You can take timestamped notes.
• You can tag your teammates
• You can share your live sessions or recordings via public links
• You can control access by making projects private

Cons:

• Limited browser compatibility
• Expensive for higher plans
• Bugs in editing comments

Pricing:

It offers four paid plans:

• Freelance – Starting $17 per month (10 sessions/year)
• Team – Starting at $99 per month (100 sessions/year)
• Insights Hub – Starting at $229 per month (300 sessions/year)
• Enterprise – Custom (Unlimited sessions)

Wrapping Up

User testing is a powerful tool for improving your website and identifying bugs and errors. If done correctly, it’s free, fast, simple to do, and can lead to considerable improvements in your site – or any smartphone application you create. 

This post contains a simple set of recommendations to help you get the most out of user testing and how to ensure your new site design is a success with your users. 

It’s a checklist for user testing that anyone can follow for the very first time and one that will improve on every subsequent test run.

Additional Resources

1. A/B Testing Guide: Everything You Need To Know About A/B Testing
2. How to Analyze A/B Test Results and Statistical Significance in A/B Testing