Until the mid-90s usability testing was conducted mostly in advanced laboratories, which meant that only large companies could afford them. Along with the development of the Internet a necessity to simplify these studies arose. It was done in order to make them accessible to everyone. This was the reason for the development of remote usability testing methods.
Remote usability testing can detect areas that hinder the execution of the user’s needs and have a negative impact on their satisfaction when using the site. The principle is simple: the current or potential users are trying to accomplish all sorts of tasks according to a previously developed scenario. The results from the observation of the behavior are the basis for the future changes. And what is their result? Increased conversions and user satisfaction.
Remote usability testing is also a great source of inspiration for A/B testing, because they increase their effectiveness (“A/B Testing is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.”)
There are many ways of testing usability. Some of them require direct contact with the user. The tests are usually carried out in a specially adapted research laboratories.
Sometimes, because of financial constraints and short deadlines for implementing the projects, we do not have the ability of conducting comprehensive studies involving users. In such situations you can help yourself with tools for usability testing through a browser – in other words use remote testing.
Remote usability testing is a method of research which assumes that the moderator and the person being tested are separated from each other spatially and/or temporally. Thanks to this you can get very detailed data about the behavior of people in a natural environment for them.
In the following sections of the article I will describe how to prepare and carry out remote usability testing scenarios.
The advantages of remote usability testing
Remote usability testing allows users to use their own computers, so they can work in their natural environment, which increases the quality of the data obtained. Other additional advantages are:
Small cost – a great solution for projects with a small budget. A series of studies through a browser can be less expensive than employing people in different cities being a place of residence of the focus groups.
Simplicity – most of the tools allows to automatically prepare, collect, analyze, and present data in a useful visual form with charts and images.
Speed – the preparation process usually takes only a few hours, therefore the time spent on preparing the report falls to minimum.
Flexibility – the tests can be turned on and off at any time or few smaller tests can be ran at the same time.
Efficiency – it is possible to reach out to people who might otherwise be overlooked. The lack of geographical restrictions makes it possible to obtain a representative users’ sample.
Remote usability testing can be carried “non-evasively”, which means that you don’t have to add any scripts to the site, and that works great with benchmark tests.
How does remote usability testing look like?
The users are sequentially given instructions through an interactive interface presented in the form of a horizontal frame.
The remote scenario tests are getting more and more popular in Poland.
Upon completing the task the information such as:
The percentage of people who completed the task,
The pages from which users “dropped out”,
The time required for the execution of tasks
Are visualized in the form of interactive statistics. After each tasks there is a possibility of inserting survey questions – it is an excellent way of gaining additional information and suggestions on how you can improve the site.
Remote usability testing is an excellent way to test hypotheses based on previous research with users, and on experts’ audits.
How to prepare the remote usability test
It is possible to distinguish the following 4 steps in performing remote usability testing:
Determining the purpose – why do we conduct a study?
Recruiting the participants – whom and how do we recruit?
Designing the test – what kind of questions are we going to ask?
Analyzing the results – what data are we going to present?
Why do we conduct a remote usability study?
The first step is defining a “research problem”, which means identifying the goal of performing a test. It is necessary to clearly define what we want to know, and what our assumptions are. The main objective should be divided into several specific objectives (reflected in the test questions), which should be expressed in a clear and measurable way.
In the course of setting goals it is good to mind the main aspects of usability:
Easy and smooth navigation,
A clear layout,
Information architecture,
Clear content,
Presentation of information,
Ease and efficiency of the search,
Aesthetics.
Remote Usability Testing Example
After performing classic usability tests for one of our clients we have found out that some people do not realize that furniture offered by the store can be bought with a $30 discount thanks to the store promotion “100 zł taniej!” (100PLN off!)
Sample product card – a folding table priced $380.
How many site visitors were not aware of the promotion? We were not sure, so we decided to check how big the problem is. We invited 1,000 people for testing, recruited through a pop-up placed on the store’s site. In the instructions we asked the clients to find a given piece of furniture, and then provide us the information about the price. After 24 hours we managed to gather the results (1,600 people took part in the test, and 1,000 actually finished all the tasks – only these results were taken into account). It turned out that only 20% of people were able to provide the correct price. It means that 800 potential customers were not aware of the promotion.
Most users were not able to provide the correct price of the product.
Based on the results of the tests, all the product pages were changed in order to visualize the promotion better.
The new way of informing people about the discount.
How and whom are we going to ask?
There are many ways of attracting people to tests. The quality of the results depends primarily on the selection of a representative group, one that will reflect the target audience of the site or service. The people required for remote tests can be recruited by:
A pop-up placed on the page,
A direct link to the survey (through email, social media, etc.),
An online panel of respondents,
A recruitment agency.
Example
A Polish marketing agency from Poland called Interactive Solutions decided to involve their users in the process of redesigning the page of the Aussie brand (hair care products). The agency decided to integrate remote tools with an international research panel and chose a group of respondents according to key demographic criteria: country, sex, age, and employment.
Respondents from the Great Britain were chosen for the tests.
The tests included general questions related to the buying habits of the consumers (a survey). Then, the time required to get to know the contents of different versions of the page was measured (Remote test scenario). Finally, the people tested were asked to rate the attractiveness of the layout (click testing – not to be confused with click tracking). The release of the study and analysis of the results along with sending to the client a recommendation how to proceed took 24 hours.
The results of the tests proved that the layouts were designed to conform to the target group.
What are we going to ask?
In my opinion, the hardest part of the preparation for the test is the correct formulation of the tasks and survey questions. The tasks should refer to the most common tasks that users perform on the page. What is more, they must be understood by people taking part in the tests. Poorly worded questions will negatively influence the quality and relevance of the obtained data and of the drawn conclusions.
Example
Usually the template of a survey question for scenario tests looks as follows:
Imagine that [introduction] and you would like to achieve [the need/the goal]. Check if the site contains the [place/product], which would allow you to realize [the need/the goal].
The scenario testing can be expanded with survey questions. The received responses will help to understand the users’ behavior, their goals, and the sources of their needs.
Questions examples:
What in your opinion would make placing an order at the store easier?
Do you think that the information provided on this website is understandable?
What do you like the most on the site? Can you tell us why?
What do you not like on the website? Why?
Do you have any other comments, questions or concerns?
Important – the pilot run
Always make sure that the scenario is valid before starting testing. You can do that by giving it to a dozen people. Thanks to this you will learn whether the tasks are understood, whether there are any technical difficulties (e.g. a change in the page layout) or whether you haven’t forgotten about anything. This allows you to identify potential problems and potentially save several hours.
What data are we going to present?
Well-planned and carried out tests will be in vain if their results will not be presented properly. A good report should:
Use simple language,
Relate to the collected results,
Include the conclusions of the study.
Example
It is good to keep in mind that the recipients of our report can be different (developers, management, graphic designers). Not for everyone jargon and abbreviations will be understandable. Several years of practice show that the report which is shorter is better than the longer one (and it is also more difficult to write). Fortunately, in the case of remote testing most charts and graphics can be simply pasted into the presentation.
At the end one small anecdote – Blaise Pascal, a famous French mathematician and philosopher, once wrote in his collection of letters:
“I have made this longer than usual because I have not had time to make it shorter. “
To sum things up
Remote usability testing helps you to look at your website with your user’s perspective and draw conclusions based on observations of their behavior. Raw quantitative data tells us how hard something is failing. Expanding tests with qualitative data (surveys: opinions, comments, preferences) allow us to understand why this happens.
There are many tools which allow you to test remotely. In the case of remote usability scenario testing it is worth mentioning:
Each of them has its own strengths and weaknesses. Personally, I like to combine remote usability testing with moderated usability testing. My answer for the frequently asked question “which one is better” is: it depends. Remote tests allow you to explore a narrow fragment of the discussed issues in detail. Although they aren’t able to replace the classic usability testing, they can complement them. They have been gaining in popularity for several years because of their very low cost and fast time-to-results.
I hope that the above examples will be useful in your work and will help you in preparing your first test.
Torsten is CEO and founder of Userpeek. He is an old stager in the online business with 20 years of experience as an online marketer, conversion rate optimizer and UX strategist.
