User testing is the process of collecting qualitative and quantitative data from the test user, whilst the user is subject to all aspects of a service or product. Qualitative or quantitative data is collected from the users test sessions to improve ux and is done using various methods and usually with the aid of a tool or usability testing service.
What is UX?
A quick summary, provided by Don Norman Jakob Nielsen, defines UX as: “All aspects of the end-user’s interaction with the company, its services, and its products”.
Likewise, Peter Morville and friends from Semantic Studios developed the UX honeycomb to encompass 7 facets of the UX.
In more detail:
Useful: Does your website, product, service have a practical purpose? The greater the use, the greater the experience.
Usable: Is your website, product or service able to be used for it’s purpose? Is it complicated and confusing or easy to understand?
Desirable: Adding emotion within your design, through the use of imagery, identity, branding and other elements of emotional design.
Findable: Can user’s locate what they’are after? Whether that be a piece of information or a product. Information and objects need to be locatable.
Accessible: Proper use of alt text for images, organised headings for structured content, correctly labeled forms etc. (Top tips for making sites accessible)
Credible: Will the user trust you? Design and design elements influence user’s trust and belief in you.
Valuable: Can the user derive value from your website, product or service? Do you improve a user’s satisfaction?
How to test UX?
UX testing can be broken down into several methods. Each method is used to uncover insights from one or more of the facets of UX.
Even though the methods are being categorised under “user testing”, there are other phrases used to describe these methods, including: UX testing, usability testing (which actually is only one part of UX testing) and user experience testing.
User Testing Methods
User Testing
User testing is the process of studying the usage of a product (e.g. a prototype of a mobile website) with real users testing that product. People who complete the “user test” are presented with a combination of scenarios, tasks and questions with the screen of the user’s device recorded for later analysis.
“If you want a great website, you’ve got to test. After you’ve worked on a site for even a few weeks, you can’t see it freshly anymore. You know too much. The only way to find out if it really works is to test it.” – Steve Krug – Don’t Make Me Think
There are two main types of user testing:
Moderated test
Unmoderated test
Moderated User Test
Moderated user testing is when the user completes set tasks on a product or service, whilst being observed by a researcher (moderator) in real time.
During a moderated user test, you have the ability to guide test users through tasks, probe and dive deeper into problems and receive real time feedback. Moderated user testing has the advantage of being able to explore a user problem in real time, but, has the disadvantage of biases and facilitation.
The most influential bias during moderated user testing is the Hawthorne Effect.
Pros:
Ability to probe users during a running test for more information
Elicit real-time feedback and emotional responses
Good sample quality with tight control over user activity
Cons:
Can be expensive trying to hire a lab
You may not have the equipment to conduct an in-house moderated test
Time to recruit, pre-screen, run test and analyse
Limited to few locations (unless conducting remote moderated)
Unmoderated User Test
Similar to a moderated, an unmoderated user test is the process of assigning participants a series of scenarios, tasks and questions, usually being completed in a remote setting. E.g. the participants home.
Large geographic diversity, with the ability to recruit a panel from around the world
Easy to run with large sample sizes
Low operating costs due to low compensation and no requirement for faciliation
Timing is quick!
Known upfront costs and good value for money
However, there are a few disadvantages and issues:
Cons:
Sample quality can be poor when recruitment is done incorrectly
Can’t probe test participants during the test session
If a user gets stuck in the test, you can’t direct in the right direction
Five Second Tests
Widely popularised by UsabilityHub, five second tests are the process of measuring user’s first impressions against the clarity of your site . Clarity is conveyed through the use of copy-writing.
Five second usability tests are best used to answer the following questions:
What does/is this site used for?
Who is the company?
What benefit does this service or product provide to me?
How to create a five-second test ?
Create a design of what you want to use in your usability test. In most cases, a homepage, landing page or product page. You can use a Google chrome extension to take a screen shot for you.
Upload the design to a usability testing service, such as UsabilityHub.
Set your usability test questions. What do the users recall?
Five second usability tests are a great way to deliver qualitative insights as well as quantitative data to produce results.
If there is an imbalance between the intent and the results, you most likely have a clarity (copy-writing) problem. Back to the drawing board…
Card Sorting
A popular UX research or usability testing method, card sorting is used to help design or evaluate the information architecture of a site and improve navigation.
The goal of card sorting is to understand how users or customers find information, navigate a system (e.g. using a menu), label information and organise the information provided.
To conduct a card sorting usability test for your website or app, you can use:
Real Cards
Pieces of Paper
Sticky Notes
Or Online Card Sorting Software
Open and Closed Card Sorting
See how members of a target audience group and label content of a website or app or learn how test participants sort pre-defined content items? Card sorting can be split into two types:
Open Card Sorting: Participants are free to organise topics of content into groups that make sense to them. These groups of topics are then labelled by the participant, without pre-determined fields.
Closed Card Sorting: Like open card sorting, closed card sorting asks participants to sort topics of content into pre-defined categories.
Tree Testing
Used in the same way as card sorting, a tree test (aka reverse card sorting) is a UX research (usability testing) method for evaluating the “findabililty” of topics in a website or app.
Using a simplified text version of a site’s structure, user’s navigate a series of top-level topics, choosing a heading and then a list of subtopics. User’s complete the task when they are satisfied they have chosen the correct topic.
How to use tree tests?
Create your “tree”. This is the structure of your website.
Write a series of “find it” tasks. For example: You bought a new watch and need a protective case. Where would you look to a buy a protect case?
Recruit participants for your test. More times than not, you want to recruit user’s of your website or product.
Also to refine the structure of the site, a tree test can also be used to quantify the improvements. Studies can be done on the existing site as a baseline which can be compared against the outcomes of each stage of usability testing. – Stephen Byrne
Session Replays
A session replay is a non-audio video recording of a user’s journey and interactions on your site. They include user interface actions like scrolling, clicks and form interactions and are a usually anonymous way of usability testing your website.
Great for identifying user behaviours and trends and has an advantage over user testing, as test participants do not know they’re being observed. Therefore, users display their real on-site behaviour.
Like Google Analytics, to get started with session replays, all you have to do is install a piece of javascript on you site to begin gathering data. The next steps will be analysing the data.
Surveys and Polls
Working in tandem, web surveys and polls are great for collating qualitative and quantitative data and feedback from your users.
With survey and poll tools allowing behaviour triggering, you can gather data from your users at the point they displayed a certain on-page behaviour
For example:
A user just purchased an item: send them an incentivised survey to learn how hard or easy the buying process was.
A user spends 5 minutes on the one product page: fire a poll to ask if there’s missing information.
Surveys can be a really useful UX tool to provide input for the design process. The key to a successful survey is establishing the objectives and information required from the study up front, then making sure the feedback questions asked cover them. – Chris Gray, Director, Nomat
Heat-Maps
Great for revealing user behaviour on pages, heat-maps are a graphical representation of where a user or users have clicked, scrolled to, mouse hovered or looked at, on a page.
Click Heat-Maps
Identify where a series of users have clicked on a specific page. The heat-map usually identifies high-level click areas as red, with lower levels of interaction ranging from orange, yellow, green and blue.
Scroll Heat-Maps
Based on a percentage of the page’s traffic, a scroll heat-map shows the percentage of users who scroll to a level/area on a specific page.
Move Heat-Maps
Reveal where your users are hovering their mouse/moving their mouse.
Eye Tracking Heat-Maps
Similar to how click heat-maps work, eye tracking heat-maps show where users look most on a webpage. Likewise, the heat-map reveals the path the user’s eye took when browsing on the page.
Intercept Questioning
Mainly known as “voice of the customer“, intercept question is the ability for users to provide feedback on the UX or usability as and when they feel the need to.
Popular tools such as Hotjar and UserSnap, offer the ability to add a widget to your site, which allows user’s to select an area of the site they wish to provide feedback on.
This can allow the user to be open and honest about certain elements of your site, that impact the UX. However, the feedback received isn’t in-depth as user tests or session replays and can vary from page to page, limiting the sample size.
Even though sample size may be small, you can gather insights to tone and use of language used by the target audience and in turn, influence your own copy-writing to relate to the user.
A/B Testing
A/B Testing – the best way to accurately measure and see if your changes are positively affecting the metric that matters for your UX.
If you’re new to A/B experiments, ConversionXL have this comprehensive guide for you to follow.
Biometric Studies
Being able to truly understand how a person feels during the UX, is UX researcher’s dream.
Are they frustrated? Do they enjoy this? Are they engaged of uninterested?
Being able to accurately understand a user’s feelings during an experience, helps us optimise and improve our UX.
Eye Tracking: The ability to track a participants eye movement – what they’re looking at, the eye path, how long they’re looking etc. Kind of like click-heat map but for the eye.
EEG (electroencephalography): Measuring electrical activity in the brain via electrodes. Great for on-the-fly representation of emotions and tracking of cognitive activity.
GSR (Galvanic Skin Response): Electrical conductivity is measured in to the small amounts of sweat generated by the pores on your skin. Provides insights into your emotional states.
For a complete list, with information on each method, check out this blog post.
Start by identifying the outcomes/goals you want to achieve by testing user experience.
Ask what questions are important to be answered in the user test in order to achieve your outcomes/goals.
Choose a ux research method that will reveal the answers to your questions. Likewise, be sure to pick a method that aligns with your time, cost and other resources.
With the aid of a usability tool or service, conduct the ux test with real human testers and gather qualitative feedback and quantitative data.
Analyse the test data, website design, iterate and repeat.
Torsten is CEO and founder of Userpeek. He is an old stager in the online business with 20 years of experience as an online marketer, conversion rate optimizer and UX strategist.
The best data collection method a researcher can employ in gathering quantitative data which takes into consideration data that can be represented in numbers and figures that can be deduced mathematically is the Questionnaire.
It is an important concept about Meta analysis and also the way to express it is easily understandable, so thanks for all. Studies that indicate a substantial impact of therapy are more likely than other studies to be published, published in English, referenced by other authors, and published numerous times.
The best data collection method a researcher can employ in gathering quantitative data which takes into consideration data that can be represented in numbers and figures that can be deduced mathematically is the Questionnaire.
It is an important concept about Meta analysis and also the way to express it is easily understandable, so thanks for all. Studies that indicate a substantial impact of therapy are more likely than other studies to be published, published in English, referenced by other authors, and published numerous times.