In this Usability Testing Guide, I covered the following topics -
What is Usability Testing?
Top statistics of Usability Testing
Top 8 benefits of website usability testing
What usability testing is not
Criteria for the usability of your website
Usability Testing methods at a glance
Top 6 Usability Testing Tools
Usability testing pro tips
Hire Usability Tester
10 min read
How user-friendly is your website? The best way to find out is with a usability test. This is not required for external help, a large budget, or extensive analysis. Instead, you can check the usability of your website yourself. We reveal what is essential.
What is Usability Testing?
Usability testing is about getting real people to interact with a website, app, or other product you've built and observing their behavior and reactions to it. Whether you start small by watching session recordings or rent a lab with eye-tracking equipment, usability testing is a necessary step to ensure an effective, efficient, and enjoyable experience for your users.
Usability is not only about ease of use but also about bringing something meaningful, having an objective in common with the user.
-- Marcus Österberg, Web Strategy for Everyone
How to conduct Usability Testing?
In a usability-testing session, a researcher (called a “facilitator” or a “moderator”) asks a participant to perform tasks, usually using one or more specific user interfaces. While the participant completes each task, the researcher observes the participant’s behavior and listens for feedback.
The phrase “usability testing” is often used interchangeably with “user testing.”
Usability Testing Tips and Examples | Google UX Design Certificate
Why Usability Test?
The goals of usability testing vary by study, but they usually include:
Identifying problems in the design of the product or service
Uncovering opportunities to improve
Learning about the target user’s behavior and preferences
Usability Testing Process
To make usability testing work best, you should:
1. Plan –
a. Define what you want to test. Ask yourself questions about your design/product. What aspect/s of it do you want to test? You can make a hypothesis from each answer. With a clear hypothesis, you’ll have the exact aspect you want to test.
b. Decide how to conduct your test – e.g., remotely. Define the scope of what to test (e.g., navigation) and stick to it throughout the test. When you test aspects individually, you’ll eventually build a broader view of how well your design works overall.
2. Set user tasks –
a. Prioritize the most critical tasks to meet objectives (e.g., complete checkout), no more than 5 per participant. Allow a 60-minute timeframe.
b. Clearly define tasks with realistic goals.
c. Create scenarios where users can try to use the design naturally. That means you let them learn it on their own rather than direct them with instructions.
3. Recruit testers – Know who your users are as a target group. Use screening questionnaires (e.g., Google Forms) to find suitable candidates. You can advertise and offer incentives. You can also find contacts through community groups, etc. If you test with only five users, you can still reveal 85% of core issues.
4. Facilitate/Moderate testing –Set up testing in a suitable environment. Observe and interview users. Notice issues. See if users fail to see things, go in the wrong direction, or misinterpret rules. When you record usability sessions, you can more easily count the number of times users become confused. Ask users to think aloud and tell you how they feel through the test. From this, you can check whether your designer’s mental model is accurate: Does what you think users can do with your design match what these test users show?
If you choose remote testing, you can moderate via Google Hangouts, etc., or use unmoderated testing. You can use this software to carry out remote moderated and unmoderated testing and have the benefit of tools such as heatmaps.
Top statistics of Usability testing
5 users will find 85% of usability problems when testing continuously. In a Poisson Distribution with a 31% binomial probability, five users in a test group can find 85% of the problems. However, as you add more than five users to a test group, you start exhibiting drastic diminishing returns, which means the more people you add, the less you learn.
50% of a developer’s time is spent trying to fix issues that could have been avoided. In a study from the University of Cambridge, researchers came to a similar conclusion: Those surveyed responded that 49.9% of their programming time was spent debugging. The problem is that this results in project delays and added costs. It is estimated that over $625 billion is spent on software production globally regarding wages and overhead, with $312 billion spent on debugging alone.
Problems in development are ten times more expensive to fix than during design. Any error found early in the process, such as during the design phase, is ten times less expensive to correct than fixing it during the design phase. Costs pile up even higher when trying to fix the problem after the product has been released.
Poor user experience with ads has contributed to the stigma, as 420 million people worldwide now use ad blockers. Does your product depend on advertisements for revenue? Then ad blockers might be your worst enemy. An estimated 420 million people use ad blockers to improve their experience while using an app or viewing a web page. This results in revenue losses of up to $35 billion by 2020 for publishers.
Even when they like the business, 50% of users will not use a website if it isn’t mobile-friendly. According to Statista, the number of mobile phone users worldwide is expected to reach 4.68 billion in 2019. This means over 60% of the world’s population will be dependent on their mobile phones. This also means that a great mobile experience is necessary when you want to do business online.
“Ease of use” is cited as the most essential quality for mobile apps by 97% of users. Undoubtedly, users place a high priority on how easy it is to use an app over other features. This is according to The State of Mobile Enterprise Collaboration published by Harmon. In the survey, 97% of business respondents place ease of use as the highest priority over mobile application security (89%), comprehensive features (72%), and training (42%). In a separate study by Accenture, users placed ease of use as the highest priority (33%) when buying a smartphone device.
79% of users who fail to find what they are looking for will look for another website. Speaking of ease of use, 79% of users admit that they leave and search for another website if they are unable to access or find what they are looking for.
86% of users want more information about a site’s products and services after landing on a homepage. Consumers want to learn everything about a brand and its products before purchasing. That is why 86% of users look for more information when browsing a website, 62% look for contact information, and 52% are more interested in the “About us” section.
The lack of a message can cause 46% of site visitors to leave. 2015 research uncovered that 46% of users refuse to stay on a website that lacks an effective message, i.e., they can’t tell what the company is or what it does. In the same study, 44% of users reported they would leave when there was a severe lack of information, and 37% left due to poor design or navigation.
Content sliders, or image carousels, don’t work—only 1% of users click on them. They are primarily used for content navigation and exploration.
Usability is about people and how they understand and use things, not about technology.
-- Steve Krug, Author
Usability testing can benefit your website at any stage of the development process, from prototyping to the finished product. You can also continue to test the user experience as you iterate and improve your product over time.
Top 8 Benefits of website usability testing
Usability testing can benefit your website at any stage of the development process, from prototyping to the finished product. You can also continue to test the user experience as you iterate and improve your product over time.
Employing tests with real users helps you:
Validate your prototype. Bring in users early on in the development process and test their reactions before locking down a final product. Do they understand what you're trying to do? Do they see the purpose of the product? Testing on a prototype first can validate your concept and help you make plans for future functionality before you spend a lot of money building out a complete website.
Confirm your product meets expectations. Once your product is completed, test usability again to ensure everything works as intended. How's the ease of use? Is something still missing in the interface?
Identify issues with complex flows. If your site has functions that require users to follow multiple steps (for example, an e-commerce checkout process), run usability testing to ensure that these processes are as straightforward and intuitive as possible.
Complement and illuminate other data points. Usability testing can often provide the why behind data points accumulated from other methods: a heat map might show that people aren't paying attention to an essential part of the landing page, but observing users in action can reveal why they're ignoring it.
Catch minor errors. In addition to large-scale usability issues, usability testing can help identify more minor errors. A new set of eyes is more likely to pick up on broken links, site errors, and grammatical issues that have been inadvertently glossed over. Individually, these incorrect details may not matter a lot, but collectively, they contribute to the perception of a site's professionalism and trustworthiness.
Develop empathy. It's not unusual for the people working on a project to develop tunnel vision around their product and forget they have access to knowledge that their typical website visitor may not have. Usability testing is an excellent way to develop empathy for the real people who are using and will be using your site and look at things from their perspective.
Get buy-in for change. It's one thing to know about a website issue; it's another to see users struggle with it. When it's evident that users are misunderstanding something, it's natural to want to make it right. Watching short clips of key usability testing findings can be a very persuasive way to lobby for change within your organization.
Ultimately provide a better user experience. Great customer experience is essential for a successful product. Usability testing can help you identify issues that wouldn't be uncovered otherwise and create the most user-friendly product possible.
Pay attention to what users do, not what they say.
-- Jacob Nelson, Co-founder of NNg
What usability testing is not
Several UX tools help improve the customer experience but don't qualify as 'user testing tools because they don't explicitly replicate the experience of real users testing a website for functionality:
A/B testing: A/B testing is an experiment with multiple webpage versions to determine the most effective. Unlike usability testing, which observes and investigates user behavior, A/B testing can help validate whether a particular approach works but can't tell you why.
Focus groups: When conducting a focus group, researchers gather a group of people to discuss a specific topic. The goal is usually to learn people's opinions about a product or service, not to test how they use it.
Surveys: Use surveys to gauge user experience. Because they do not allow you to observe visitors on the site in action, surveys are not considered usability testing, though they may be used in conjunction with it.
Heat maps: Heat mapping software visually represents how users move around the page by showing its hottest (most popular) and most excellent (least popular) parts. The maps allow developers to see how people, in aggregate, move around a website, but they are still technically not usability testing.
User acceptance testing is often the last phase of the software-testing process, where users go through a calibrated set of steps to ensure the software works correctly. This is a technical test of QA (quality assurance), not a way to evaluate if the product is user-friendly and efficient.
In-house proper use testing: people in your company probably always test software, but this is not usability testing. Employees are inherently biased, making them unable to give honest results that genuine users can.
Criteria for the usability of your website
Translated, usability means suitability for use. It describes how users can use a product or service in a specific application context to satisfactorily achieve their goals. A usability test can help you determine how well visitors get along with your website.
The term usability has only been used since the 1980s. Of course, how people get along with technology was already a topic beforehand. At least since the invention of giant machines like the steam locomotive and the number of terrible accidents caused by operating errors, it was clear that usability is an essential criterion for making technology accessible to people.
In contrast to the steam engine, there is, of course, no danger from your website. If the user-friendliness is correct, you can increase the length of time your visitors stay, reduce the bounce rate, or even generate leads.
The following components of your website play a role in this:
Content
The Internet has many advantages—and one problem: too much input. Decades ago, Germany was calculated to have an information overload of 98 percent. That means we don't even notice 98 percent of the information we encounter every day.
Of course, this also influences your visitors' reading behavior. So you have to give them a reason to take a closer look at your content.
It starts with your website's address. If it is unattractive or overwhelming, it will hardly be clicked. While the latter probably overwhelms well-meaning users, the former presents all the essentials at a glance.
Overall, the following questions are important about the usability of your content:
Is your content relevant and not unnecessarily long?
Are they well-structured and understandable?
Do they offer your visitors added value?
Do they cover the search intent of your visitors?
Usability is like love. You have to care, you have to listen, and you have to be willing to change. You’ll make mistakes along the way, but that’s where growth and forgiveness come in.
--Jefferey Zeldman, Principal Designer at Automattic
Design
Is anything that pleases allowed? It's not that easy. A clear/simple website design helps your visitors understand content immediately and without any problems. You should avoid constantly popping up windows, flashing buttons, or umpteen different fonts and colors.
Instead, the following questions are essential:
Does the overall design of your website provide a pleasant user experience?
Does the design help your visitors find their way around the site?
Are there visual hierarchies that help your visitors better understand the content?
Do you use images, videos, or similar things that offer added value to your visitors?
Navigation
One of the most critical usability factors is navigation. Your visitors can't find their way around if it's complicated or overloaded - and they leave your website again very quickly. As with design, the goal of your navigation should be to help visitors find information.
The following questions are important here:
Is your navigation generally comprehensible?
Is it logically structured and divided into hierarchical levels?
Is the number of menu items manageable? Between five and seven are easily perceptible.
Web conventions
You most likely use web conventions anyway. These are standards in website design that every user knows and somehow expects. You should not break with these conventions to avoid confusing your visitors.
Standard web conventions include these:
Your logo is at the top of the website. Users can click it to go back to the home page.
The navigation is either at the top of the page or on the left side.
Standard links to the data protection declaration, terms, and conditions or imprint are placed in the website's footer.
Hyperlinks are color-coded.
Don’t find customers for your products, find products for your customers.
-- Seth Godin, Blogger/Entrepreneur & Author
Creativity
Another quality feature is your website's creativity. It should offer visitors something they won't get anywhere else—something that will intrigue them and make them stay on your site longer.
As part of usability, you should ask yourself the following questions:
Is your website remembered by your visitors?
Does it offer your visitors something new that they haven't seen before?
Are there applications or similar that help your visitors - and differentiate you from the competition?
Usability Testing Methods at a Glance
You should not judge for yourself whether—and how many—of the questions presented can be answered positively. A usability test provides much more accurate results. There are very different approaches that can differ significantly in terms of effort and process.
In general, there are these variants:
Expert tests: If you don't want to help yourself, turn to experts. They have different ways of examining a website. One of them is to assess its quality using heuristic methods: The experts use existing guidelines, check the website against these specifications, and note any anomalies.
Qualitative test methods: With qualitative test methods, the focus is on the individual test results. What did this user do to get the information they wanted? What difficulties did this user encounter, and how did they detract from her user experience? Small test quantities are enough to find out a website's central problems.
Quantitative test methods: Quantitative test methods require more participants because they are intended to determine measurable numerical values. They are well suited to revealing general difficulties. For example, if none of the test subjects could solve a task, this is a clear sign that improvements should be made in this area of the website.
Online tools: Finally, online tools can be used to identify a website's weak points. Most are quick to set up, easy to use, and often even free in their basic version. Numerous online tools specialize in a wide variety of key figures. More on that is below.
Remote Test: Remote testing is any testing that happens when the participant and the researcher are in separate locations. In a remote test, the participants complete the tasks in their natural environment using their devices. The sessions are facilitated by online tools and can be moderated or unmoderated.
Usability testing most common method.Top 6 Usability Testing Tools
If you want to find out quickly and without much effort what the usability of your website is like, the following approaches are recommended:
UserTesting is one of the most popular platforms among usability testing tools.
Using it, you can quickly get feedback from real users in video or note format. I prefer to work with videos or participate in live conversations, as it allows me to observe users’ facial expressions, analyze intonation, and see their live reactions.
One useful feature is the Highlight Reel. It allows you to highlight key points that you want to bring to the team’s or client’s attention.
By the way, if you use Justinmind for prototyping, you’ll be interested to know that UserTesting is integrated with it.
HotJar is a key usability testing tool. Over 900,000 organizations currently use Hotjar for usability analysis, and in 2021, the tool was recognized as one of the leading services in EMEA.
I would highlight the key features you get when using the tool:
Heatmaps: maps clicks, scrolling, mouse movement
Feedback collection
View records during A/B testing
Tracking and recording of user behavior
Analysis of conversion funnel
Form Analytics
Create various questionnaires and surveys
Click comparison
The pluses for me are the ease of setup, which takes only a few minutes, and the visualization of the data, making it easier to analyze user behavior.
UserZoom perhaps requires no introduction, but I thought I’d add it to complete the picture. It’s a huge platform suitable for large companies, with around 120 million potential test participants.
Once you sign up and pay for the service, you get access to the complete package for online user research — from recording sessions on multiple devices to saving your driving route through the site/app, not to mention heatmaps. Plus, in UserZoom, you can run multiple studies at the same time.
I usually set up the study package so that only the add-ons I need for the project are there. This allows me to be more productive and not be distracted by redundant data.
People should never feel like a failure when using technology. Like the customer, the user is always right. If software crashes, it is the software designer's fault. if someone can't find something on a website, it is the web designer's fault_ The big difference between good and bad designers is how they handle people struggling with their design. Technology serves humans. Humans do not serve technology.
-- Joshua Porter, co-founder of Rocket Insights
Automated Examination: Online Tools
Online tools provide reasonable indications of where the problem is on your website. They cannot replace the concrete experiences of real users; they provide good starting points where action is needed. A big plus point: You can start testing immediately and without preparation.
One tool you should examine closely is Google's Lighthouse. Originally, the service was developed for testing progressive web apps; however, it can also test conventional websites.
You can measure numerous different factors and also receive specific recommendations for action to optimize your website. To receive evaluations that are as realistic as possible, you also have the option of filtering and adjusting the test conditions.
Quantitative method: Onsite survey
An on-site survey is a quantitative method with test persons. It allows you to generate feedback from many visitors to your website within a short time.
For this purpose, a questionnaire is integrated directly into your website, and visitors are invited to participate in the survey via popups or layers. You can decide whether you want to participate in the survey.
With an onsite survey, you can find out, for example,
how well visitors generally get along on your website,
if they have any suggestions for improving your website,
which functions they miss on your website.
You can use the onsite survey to gain insights into your website and get to know your target group better. Ultimately, your questions end up with those who visit your website anyway. In addition, the on-site survey contributes to regular quality assurance because you can use your questionnaire repeatedly without any effort.
Qualitative method: observation and questioning
Personal observation and questioning are easy-to-implement qualitative methods. Test subjects are assigned certain tasks, which they are supposed to solve under observation.
The good thing is that large test quantities are not necessary! According to a study by usability expert Jakob Nielsen, just five test subjects can uncover 80 percent of all usability problems. So, you only need five test results to increase the user-friendliness of your website enormously.
For a meaningful result, you should proceed as follows:
Find people who would like to help you. But don't make your life unnecessarily tricky by endlessly searching for the perfect subjects. It's about testing a regular website, so almost anyone can do it. Friends or relatives are good candidates.
Set your test subjects' tasks and observe how they solve them. To help them better adapt to the situation, invent a small scenario. This can look like this: “For the sake of the environment, you would like to switch from the car to the bicycle. Find a bike for your commute that suits you and is in the mid-range price segment.”
Important: Your subjects should be able to navigate independently. Under no circumstances should you give them specific instructions, as this will falsify the results.
The subsequent survey will provide you with even more details: How did your candidates find the level of difficulty of the tasks set? How did you come up with the respective solutions? If there were difficulties, why was that? The knowledge gained will help you improve your web presence.
Usability testing pro tips
Recruit realistic participants. People who would perform these tasks in real life
Use 5-8 participants for qualitative usability testing
Avoid influencing participants during testing. The easiest way to do this is to stay quiet during the study
Ask open-ended, neutral questions when you speak to the participant. For example, say, “What do you think about this?” instead of “Do you like this?”
Testing with more than five users results in diminishing returns
Sure, you can test with more people, and there is a chance that you won’t uncover any issues until you have your sixth or seventh user. But it is very unlikely if you’ve designed your tests to cover the core areas of the service. What is more likely is that by your sixth user, you’ll start to see the same patterns of behavior repeated over and over, and your value from the tests will start to diminish.
You may have seen Jakob Nielsen's great diagram showing the diminishing returns of testing with more than five users.
We agree with this entirely, and what many people forget about the accompanying article (Why You Only Need to Test with 5 Users) is that he advocates testing with five people, fixing the issues, and then testing again. This is very much the way we like to work. We encourage many clients who come to us with plans to recruit 15/20 people to reduce their scope and break the project into chunks that allow for design changes to be made before new users are brought in.
You don’t need a large number of users to identify usability problems. Don’t confuse usability research with other types of market research. Five people aren't near enough if you want to gauge opinions about competing brands. However, if you want to identify barriers to purchasing on your website, five users are usually enough.
Conclusion
If you want to know how user-friendly your website is, you can't avoid a usability test. You can hardly test your website yourself—after all, you already know how it works. Instead, online tools or test subjects who have a fresh look at the site will help you uncover problems you wouldn't spot on your own.
Of course, you cannot eliminate all difficulties that arise overnight. Improving user-friendliness is a constant development process that can significantly increase your website's success.
Credits
Hire A Usability Testing Expert Consultant!
Successfully completed 60+ Usability testing projects on conversion rate optimization (CRO), user experience (UX), usability testing and evaluation, and UX wireframing. 12+ years of experience in the field. Clients include Stanford-HCI, Accenture, Juniper Networks, Gemalto, Packt, Virtual Dental Care, Credello & much more.
Usability / UX reviews/audits
✔️ Conversion rate optimization (CRO)
✔️ CRO for Shopify, Magento, WooCommerce, BigCommerce, Bitrix, and other platforms
✔️ Usability testing / UX testing
✔️ Heuristic evaluation
✔️ UX wireframes
✔️ Supervision and corrected the work of graphic designers
✔️ design of complex online forms and questionnaires
✔️ Interaction design (IxD)
✔️ Human-computer interaction (HCI)
✔️ Information Visualization (InfoVis)
✔️ User behavior analytics: analysis of heatmaps, scroll maps, and user session recordings
✔️ Online travel (flight booking, hotel booking, vacation rentals, tours, and activities)
Tools: Axure, Balsamiq, Google Analytics, Hotjar, Crazy Egg, Lucky Orange, VWO, Inspectlet, FullStory, WebVisor, Clicktale, Mouseflow, SessionCam, Clicky, Smartlook, CUX, etc.
An expert member of Technical Committee 13 (Human-Computer Interaction) of the International Federation for Information Processing (IFIP).
Published 50+ academic articles on human-computer interaction, usability, user experience, typography, e-commerce, process control, accessibility, art exhibitions, and digital technologies in museums.
Hire A Usability Tester Now!