It took Unilever 45 generations and 449 designs to create a perfect nozzle which would dispense washing powder.
5,127 prototypes of Dyson’s cyclone technology were created before James Dyson believed the technology was fit to go into a vacuum cleaner.
Over 3 million years of trial and error have occurred for you and I to evolve into what we are now.
Testing, iteration and trial and error.
It’s what differentiate’s the “good” from the “bad” and the “great” from the “good”. Yet, we don’t seem to take this approach when it comes to UX.
This is what this article is about: user testing and measuring.
What is User Testing?
User Testing definition:
User testing is the process of collecting qualitative and quantitative data from the test user, whilst the user is subject to all aspects of a service or product. Qualitative or quantitative data is collected from the users test sessions to improve ux and is done using various methods and usually with the aid of a tool or usability testing service.
What is UX?
A quick summary, provided by Don Norman Jakob Nielsen, defines UX as: “All aspects of the end-user’s interaction with the company, its services, and its products”.
Likewise, Peter Morville and friends from Semantic Studios developed the UX honeycomb to encompass 7 facets of the UX.
In more detail:
- Useful: Does your website, product, service have a practical purpose? The greater the use, the greater the experience.
- Usable: Is your website, product or service able to be used for it’s purpose? Is it complicated and confusing or easy to understand?
- Desirable: Adding emotion within your design, through the use of imagery, identity, branding and other elements of emotional design.
- Findable: Can user’s locate what they’are after? Whether that be a piece of information or a product. Information and objects need to be locatable.
- Accessible: Proper use of alt text for images, organised headings for structured content, correctly labeled forms etc. (Top tips for making sites accessible)
- Credible: Will the user trust you? Design and design elements influence user’s trust and belief in you.
- Valuable: Can the user derive value from your website, product or service? Do you improve a user’s satisfaction?
How to test UX?
UX testing can be broken down into several methods. Each method is used to uncover insights from one or more of the facets of UX.
Even though the methods are being categorised under “user testing”, there are other phrases used to describe these methods, including: UX testing, usability testing (which actually is only one part of UX testing) and user experience testing.
User Testing Methods
User testing is the process of studying the usage of a product (e.g. a prototype of a mobile website) with real users testing that product. People who complete the “user test” are presented with a combination of scenarios, tasks and questions with the screen of the user’s device recorded for later analysis.
“If you want a great website, you’ve got to test. After you’ve worked on a site for even a few weeks, you can’t see it freshly anymore. You know too much. The only way to find out if it really works is to test it.” – Steve Krug – Don’t Make Me Think
There are two main types of user testing:
- Moderated test
- Unmoderated test
Moderated User Test
Moderated user testing is when the user completes set tasks on a product or service, whilst being observed by a researcher (moderator) in real time.
During a moderated user test, you have the ability to guide test users through tasks, probe and dive deeper into problems and receive real time feedback. Moderated user testing has the advantage of being able to explore a user problem in real time, but, has the disadvantage of biases and facilitation.
The most influential bias during moderated user testing is the Hawthorne Effect.
- Ability to probe users during a running test for more information
- Elicit real-time feedback and emotional responses
- Good sample quality with tight control over user activity
- Can be expensive trying to hire a lab
- You may not have the equipment to conduct an in-house moderated test
- Time to recruit, pre-screen, run test and analyse
- Limited to few locations (unless conducting remote moderated)
Unmoderated User Test
Similar to a moderated, an unmoderated user test is the process of assigning participants a series of scenarios, tasks and questions, usually being completed in a remote setting. E.g. the participants home.
Unmoderated user testing has some great benefits over moderated studies, for example:
- Large geographic diversity, with the ability to recruit a panel from around the world
- Easy to run with large sample sizes
- Low operating costs due to low compensation and no requirement for faciliation
- Timing is quick!
- Known upfront costs and good value for money
However, there are a few disadvantages and issues:
- Sample quality can be poor when recruitment is done incorrectly
- Can’t probe test participants during the test session
- If a user gets stuck in the test, you can’t direct in the right direction
Five Second Tests
Five second usability tests are best used to answer the following questions:
- What does/is this site used for?
- Who is the company?
- What benefit does this service or product provide to me?
How to create a five-second test ?
- Create a design of what you want to use in your usability test. In most cases, a homepage, landing page or product page. You can use a Google chrome extension to take a screen shot for you.
- Upload the design to a usability testing service, such as UsabilityHub.
- Set your usability test questions. What do the users recall?
Five second usability tests are a great way to deliver qualitative insights as well as quantitative data to produce results.
If there is an imbalance between the intent and the results, you most likely have a clarity (copy-writing) problem. Back to the drawing board…
The goal of card sorting is to understand how users or customers find information, navigate a system (e.g. using a menu), label information and organise the information provided.
To conduct a card sorting usability test for your website or app, you can use:
- Real Cards
- Pieces of Paper
- Sticky Notes
- Or Online Card Sorting Software
Open and Closed Card Sorting
See how members of a target audience group and label content of a website or app or learn how test participants sort pre-defined content items? Card sorting can be split into two types:
- Open Card Sorting: Participants are free to organise topics of content into groups that make sense to them. These groups of topics are then labelled by the participant, without pre-determined fields.
- Closed Card Sorting: Like open card sorting, closed card sorting asks participants to sort topics of content into pre-defined categories.
Used in the same way as card sorting, a tree test (aka reverse card sorting) is a UX research (usability testing) method for evaluating the “findabililty” of topics in a website or app.
Using a simplified text version of a site’s structure, user’s navigate a series of top-level topics, choosing a heading and then a list of subtopics. User’s complete the task when they are satisfied they have chosen the correct topic.
How to use tree tests?
- Create your “tree”. This is the structure of your website.
- Write a series of “find it” tasks. For example: You bought a new watch and need a protective case. Where would you look to a buy a protect case?
- Recruit participants for your test. More times than not, you want to recruit user’s of your website or product.
- Have participants complete your test tasks.
- Analyse the final results of your usability test.
Also to refine the structure of the site, a tree test can also be used to quantify the improvements. Studies can be done on the existing site as a baseline which can be compared against the outcomes of each stage of usability testing. – Stephen Byrne
A session replay is a non-audio video recording of a user’s journey and interactions on your site. They include user interface actions like scrolling, clicks and form interactions and are a usually anonymous way of usability testing your website.
Great for identifying user behaviours and trends and has an advantage over user testing, as test participants do not know they’re being observed. Therefore, users display their real on-site behaviour.
Surveys and Polls
Working in tandem, web surveys and polls are great for collating qualitative and quantitative data and feedback from your users.
With survey and poll tools allowing behaviour triggering, you can gather data from your users at the point they displayed a certain on-page behaviour
- A user just purchased an item: send them an incentivised survey to learn how hard or easy the buying process was.
- A user spends 5 minutes on the one product page: fire a poll to ask if there’s missing information.
The beauty of polls and surveys is they allow you structure a variety of questions.
Surveys can be a really useful UX tool to provide input for the design process. The key to a successful survey is establishing the objectives and information required from the study up front, then making sure the feedback questions asked cover them. – Chris Gray, Director, Nomat
Great for revealing user behaviour on pages, heat-maps are a graphical representation of where a user or users have clicked, scrolled to, mouse hovered or looked at, on a page.
Identify where a series of users have clicked on a specific page. The heat-map usually identifies high-level click areas as red, with lower levels of interaction ranging from orange, yellow, green and blue.
Based on a percentage of the page’s traffic, a scroll heat-map shows the percentage of users who scroll to a level/area on a specific page.
Reveal where your users are hovering their mouse/moving their mouse.
Eye Tracking Heat-Maps
Similar to how click heat-maps work, eye tracking heat-maps show where users look most on a webpage. Likewise, the heat-map reveals the path the user’s eye took when browsing on the page.
Mainly known as “voice of the customer“, intercept question is the ability for users to provide feedback on the UX or usability as and when they feel the need to.
Popular tools such as Hotjar and UserSnap, offer the ability to add a widget to your site, which allows user’s to select an area of the site they wish to provide feedback on.
This can allow the user to be open and honest about certain elements of your site, that impact the UX. However, the feedback received isn’t in-depth as user tests or session replays and can vary from page to page, limiting the sample size.
Even though sample size may be small, you can gather insights to tone and use of language used by the target audience and in turn, influence your own copy-writing to relate to the user.
A/B Testing – the best way to accurately measure and see if your changes are positively affecting the metric that matters for your UX.
If you’re new to A/B experiments, ConversionXL have this comprehensive guide for you to follow.
Being able to truly understand how a person feels during the UX, is UX researcher’s dream.
Are they frustrated? Do they enjoy this? Are they engaged of uninterested?
Being able to accurately understand a user’s feelings during an experience, helps us optimise and improve our UX.
Biometric studies encompass numerous research methods, including:
- Eye Tracking: The ability to track a participants eye movement – what they’re looking at, the eye path, how long they’re looking etc. Kind of like click-heat map but for the eye.
- EEG (electroencephalography): Measuring electrical activity in the brain via electrodes. Great for on-the-fly representation of emotions and tracking of cognitive activity.
- GSR (Galvanic Skin Response): Electrical conductivity is measured in to the small amounts of sweat generated by the pores on your skin. Provides insights into your emotional states.
For a complete list, with information on each method, check out this blog post.
User Testing Tools [List]
User tests: moderated and unmoderated
Five Second Tests
Surveys and Polls
- More of a service rather than a tool
- More of a service rather than a tool
- CoolTool sell eye tracking equipment
Conclusion: A User Testing Checklist
- Start by identifying the outcomes/goals you want to achieve by testing user experience.
- Ask what questions are important to be answered in the user test in order to achieve your outcomes/goals.
- Choose a ux research method that will reveal the answers to your questions. Likewise, be sure to pick a method that aligns with your time, cost and other resources.
- With the aid of a usability tool or service, conduct the ux test with real human testers and gather qualitative feedback and quantitative data.
- Analyse the test data, website design, iterate and repeat.