Usability testing comes in so many forms from heatmaps to actual face-to-face interviews. But these types of tests fall only in two categories: moderated and unmoderated usability testing.
What is the difference between these two? In this article, Moderated vs Unmoderated User Testing, we will identify the difference between these two kinds of user testing and also, we will help break things down for you in terms of what you need to know such as the best time to use these strategies and the pros and cons of the two when it comes to the design process.
Moderated user testing requires a person to be present and help facilitate or moderate the test. This person is required to work directly with the test participant. The role of the moderator is to guide and to assist the participant in completing the task. You can conduct a moderated test remotely or in-person.
However, this type of user testing requires thorough planning and it requires coordinating schedules with the participants. This kind of user testing is best conducted in a designated, quiet location, where the moderator and participant can interact without interruptions.
The process of conducting a moderated user test
As mentioned earlier, the moderated user testing is a long planning process, with many different stages, all towards understanding user behavior. The below steps are not standardized but these are enough basis for you to understand what the entire process entails:
1. The planning stage
Before the actual user testing stage, you need to plan the steps on how you can properly ask people to take the test.
Planning ahead will help you define your goals and the key elements needed for user testing. By this, it means that you need to go beyond product knowledge.
It is not enough to just know what your product is about. You need to know where you are heading with each of the elements needed to run a successful user testing.
Specify each element properly in order to properly execute them. This way, you and the team will better understand if the design lives up to your standards or if it needs re-designing.
This stage also includes narrowing down the important factors like the number of people needed to participate and where your participants will be conducting the user test.
In the planning stage, it is important to take note that the thorough you are with planning, the fewer the problems you may likely encounter in your moderated user testing.
2. Recruitment
You may choose to hire a 3rd party recruiter or conduct your own recruitment. This decision usually depends on one’s budget.
Recruiting is not an easy task. You may find it easy to hire 5 people but finding the right 5 people for your test makes a huge difference. For this reason, many user research teams prefer to spend on hiring a third-party recruitment service for help.
As an alternative, user testing tools that also provide recruitment can solve this problem easily. You may find some usability testing tools with advanced features and services that help you easily conduct moderated usability testing.
No matter which way you go with recruiting participants, make sure that you screen your participants to get the right people needed for your test.
3. Setting up the tools
There are a lot of things to define at this stage. Depending on your product, you need to decide which device you want to carry out the user testing.
Like for example, you have a website. You need to decide which browser you will be testing with, which operating system to use, and etc. All these matters and generally, you also have to consider what the majority of your target users prefer to use.
In addition, you also need to decide how you will be conducting your moderated usability test, which refers to the actual questions and tasks you will present to your users.
4. Conducting the test
Conducting moderated user testing should be done in a consistent manner, with the main goals in mind. Remember that you are conducting the test not to influence the participants but rather, take note of everything that is happening.
This can be easier said than done since it can be tricky not to influence the participants. A single action or word can still influence. Remember not to reveal or share too much information, especially information that is not required for the users to complete the tasks.
Also, you may want to record the test as it is easier to go back to the recordings and draw conclusions.
Refrain from drawing conclusions while conducting moderated user testing. This will lead to missing out details and may lead to biased observations.
5. Analysis and results
Getting data is just part of the whole user testing process, moderated, or unmoderated.
To complete the entire usability testing, you need to organize your thoughts. This process can be made easier depending on the user testing tool you use. Some of these tools offer you the ability to take notes on the video recording or some tools give you information arranged in graphs.
Remember that your report will be read by stakeholders as well so better to keep the report straightforward.
Use easy-to-understand words to better explain your findings. The use of visual aids such as graphs and charts help show larger data easily.
The pros and cons of moderated user testing
The pros:
You can ask questions during the test to better understand the user’s behavior
You can improvise during the test. For example, you can give an additional task to the user, which has only arisen from the test
The cons:
The process involved in planning a moderated user testing is time-consuming and not flexible as you need to be present to moderate the test
Documenting results requires preparation and technical tools
Moderation can alter or influence test results
Best practices in moderated user testing
Moderators should build trust with the participants in order to achieve the best user test results. The more the participants trust the moderator, the better and easier they can open up their thoughts and feelings.
Run a pilot test so you know the things that may arise to better improve the actual test.
Use prototypes in testing your features to keep things visual and user flow natural.
What is unmoderated user testing?
Unmoderated user testing is not monitored or guided. There is no need for the moderator to be present in conducting the test. The participants complete the tasks and answer the questions at their own pace, on their own time, and anywhere they choose.
Since there is no moderator, this strategy is faster than the moderated user testing strategy.
How do you conduct unmoderated user testing?
Just like the moderated usability testing, the unmoderated usability test also follows the same steps when it comes to planning the test.
Note that when it comes to unmoderated user test, you need to be very clear with the tasks as you won’t be able to facilitate the participants in the right direction if anything goes wrong. Thus, with this strategy, obtaining useful data depends on how clear your written instructions are.
To help you conduct the unmoderated usability test, you need to utilize a usability tool for testing. These tools will help record user sessions and also show the time taken on each touchpoint. Aside from this, participants can also record their feedback once the test is complete.
The pros and cons of unmoderated user testing
The pros:
The participants do not need a moderator to take the test
The participants can take the test anytime, anywhere, and on their own preferred place, and their own devices
Saves time because you do not have to plan and moderate the individual test
Very flexible since you do not need to be present to moderate the tests
The cons:
No way for the moderator to intervene when something goes wrong
The moderator cannot ask questions when it comes to better understanding the participant’s behavior
Best practices in unmoderated user testing
Test your usability tool first through a mockup test situation or run a pilot test to make sure that everything goes smooth
Include more test participants. You have this advantage since most unmoderated user tests are cheap and faster. Having more participants will serve as your safety net if some participants do not show up on the actual test
Follow the test participants’ post-study. You want to get a lot of feedback from them as possible in case you need to reach to them after.
When to prefer moderated or unmoderated user testing?
User testing is about “doing it”, preferably on a regular basis. It does not really matter whether you conduct a moderated or unmoderated user test, as long as you run the test and you are doing everything right.
However, if you have no or less experience in moderated user testing, we highly advise that you choose unmoderated user test instead since this strategy means less preparation and effort in terms of implementation. And on the practical side, you got the added benefit of not telling anything to the participants about your product and how to use it, and thus, in return, you avoid false-positive results.
Final thoughts
Being user-centric is not just a competitive advantage these days but it is a basic need. Having a human-centered mindset when it comes to the design process is important. In UX, we validate our design efforts through qualitative studies like the moderated and unmoderated user testing.
So which of the two is better? The answer is: it depends. Like any other research strategy, and as discussed, each one has its advantages and disadvantages. The choice depends largely on your preferences and how you want to reach your research goals.
