When it comes to product development, ideas can be unpredictable. Everyone on the team has ideas to improve a new or existing product.
While this is a good initiative, the UX team can have an idea overload. What is the best way to resolve this incident?
The UX team should test ideas with the target audience in order to know which idea is worth pursuing and which ones should be left for later.
This is what successful concept testing is about. And in this article, Quantitative Concept Testing, we will discuss more about successful concept test.
We will also look into a particular method called quantitative research concept testing, touching the following subtopics:
Concept testing lies in the process evaluating, predicting, and understanding consumer responses and reactions towards a new or existing product or service.
The goal of doing the product concept test is to minimize any risks, which helps the product's potential to raise profits. This strategy allows companies to guarantee that the product concept will be appealing enough and be valuable enough to get more customer purchases.
In simple words, product concept testing is part of a marketing research strategy that many brands utilize to measure the engagement, readiness, and willingness of their target customers towards their product or service. This applies to both existing or new products or service concepts.
Product concept testing involves both quantitative and qualitative methods to fully understand how the target market will reach a particular idea when presented in the form of a product.
The benefits of concept testing
Provides brands unbiased perspectives for the product idea, which helps save time and money
Any member in the UX team with an idea or multiple concepts can have bias, which can wrongfully assume the idea will be a successful product, feature, or service.
The UX team should organize a concept test market to produce actionable insights to avoid this from happening.
Allows marketers to dig deeper into the concept's micro aspects
When it comes to concept testing, the UX team and the marketing team can tweak, amplify, and eliminate several product elements regarding its aesthetics or features.
Concept testing probes into the target market's minds via online concept test such as surveys and focused groups adapted for the purpose of what they are made for.
The UX team can take several methods to test their products or services before even forwarding them from product development process into a product launch.
Here are the types of product concept testing:
Single concept evaluation
This method of concept testing involves test participants, who are first presented with a single concept evaluation. The test participants are given an in-depth review of the product concept, including its features and functions.
After the test, the participants are asked to answer a survey wherein they rate and evaluate the product concept by answering a series of questions. The questions are about whether the product concept is appealing to them or not. This strategy is fast, unbiased, and provides a high response rate.
Comparison testing
Compared to the single product concept evaluations, comparison testing involves testing two or more product concepts.
The participants are provided with a breakdown of multiple concept ideas to choose from. The participants are only required to select the best concept that managed to sway them.
The UX researchers can form interferences based on the participants' answers to determine which product concept showed to be more widely accepted in the concept test.
However, there is one slight lack of context for comparison testing. There is no explanation from the respondents as to why they chose that particular product concept over the other choices. There is also no constructive criticism on the test multiple concepts that did not interest them.
Monadic testing
Monadic testing is another multiple ideas concept evaluation strategy that differ from the comparison testing method regarding how participants are approached.
In the Monadic testing method, the participants are divided into several groups, depending on the number of product concepts that are evaluated.
Each of these groups is provided with a different product concept in isolation, allowing the participants to analyze further the idea given.
UX researchers can ask several follow-up questions to the participants about the product ideas to know what they liked about them and why they liked them.
In contrast to comparison testing, the monadic testing method adds context to the participants' choices. Thus, the results are more thorough, and this helps UX researchers better understand the insights based on each group's likes and dislikes.
Since this method provides context, the tests are relatively more costly. The tests should also be conducted on a large scale. The more product concepts that UX researchers need to test and evaluate, the more participants they need to form the focus groups.
Sequential monadic concept testing
The sequential monadic concept testing is similar to multiple concept evaluations. This type of testing involves presenting all kinds of concepts to the test respondents in random order. The target audience, however, is split into multiple groups.
Sequential monadic testing does not require a larger group of participants, making it easier and less expensive to do. But there are also several limitations.
For example, each group is provided with all multiple concepts, and the questions are very lengthy, which can lead to a low completion rate and can create a non-response bias that can alter and affect the results.
Proto monadic concept testing
Whenever a sequential monadic test follows a comparison test, it is called proto monadic testing. The grouped participants will first be given an in-depth reveal of each concept to evaluate each idea in its entirety.
The succeeding comparison test will enable the participants to reveal the concept they prefer over the other ones.
The latter results have a significant role in verifying the results of the sequential monadic test, and whichever the participants chose will have more positive feedback in the sequential nomadic test.
How to plan your quantitative concept test?
Identifying segments and markets
Several essential questions include: which locations are most important to your team? Which customer segments are most important to compare to each other?
Note that the general population will always be the least expensive to reach. The costs can increase as the segments become very targeted or specific.
We highly suggest that your UX team provide each participant with no more than seven concepts to evaluate, just to limit the survey length.
Setting up the MaxDiff
The MaxDiff or also known as the best-worst, involves participants in a survey, indicating the best and the word options out of the test set given to them.
This is often implemented within an experimental design to obtain a relative ranking for each option provided.
An example of MaxDiff, when a participant is asked to pick the most and the least important factor when it comes to choosing a selection of food in a restaurant, the survey looks something like this:
And as the survey participant progresses through the sets in the design, we get a fuller picture of what is essential and the least important.
When creating your MaxDiff, write a one-sentence text description for every concept. Do not write more than 20 concepts. Add at least one benchmark concept, totaling 21 concepts as the limit.
Another factor to consider is it is also essential to include your key stakeholders such as the content strategist, design, or product manager to align on the concepts and descriptions to achieve an impact.
Setting up the sequential monadic survey
Make sure to choose your metrics. You may find that your needs may be different. However, you may follow through with the given metrics when it comes to surveys:
· Overall opinion
· Likelihood to use
· Uniqueness
· Comprehension
· Unmet needs
· Effect/experience
Guidelines for concept descriptions
These are the five key elements when it comes to product development process concept descriptions:
1. Consistent length and detail
All test concepts should be similar in length and level of detail.
2. Clear language
Always be clear, simple, and specific. Write your voice that is appropriate to your target market research. Use neutral, consumer-friendly words and phrases, not the company's language.
3. Uniform in phrasing and tone
Always be consistent when using verb tense, punctuation, tone, and text formatting like bold, italics, etc.
4. Singular ideas
Focus on the man concept idea. Testing double-barreled concept tests will only obscure understanding and create chaotic reactions from your test participants.
5. Address the needs
Ensure that the new concept speaks to the consumer's needs and address relevant, underlying pain points.
Survey design overview
At the start and end of surveys
Better start the survey with a screener that defines your target market research, including the basic demographics, to guarantee a good test sample.
Then end the survey with additional demographics like income, language, etc., critical for your audience profiling and segmentation. Do not include demographics that are intended for quotas or defining your audience.
Design and programming
The survey design and programming should continually be optimized for smaller screens (tablets and mobile phones) using accordion or banked grids, shortened texts, and limiting open ends.
Timelines
The timelines can vary a lot as it depends on the project's specific context. Most UX teams typically spend 5-7 weeks from start to finish.
However, the length depends on the number of markets, audience segments, metrics, and other tested concepts.
The crucial part of this process is generating your recommendations based on how you interpret the data.
The data from MaxDiff and metrics you've gathered will help provide precise results from the ranked order lists and attribute ratings, respectively.
Here are the tips on combining the data sets:
The metric ratings help in explaining the MaxDiff ranking reasoning
The concepts that score high in the metrics and the MaxDiff should be your top recommended concepts.
The concepts with low MaxDiff rankings and high metrics ratings can indicate untapped value.
The concepts that rank low in comprehension but rank higher in other metrics may better be utilized in qualitative concept testing with prototypes.
It is not just about MaxDiff rankings
The lower-ranked concepts with a high score in other metrics might be worth pursuing.
Check and clarify the value proposition and execution first. Look for emerging patterns such as commonalities across the high-ranking concepts.
Some metrics you have may be better off in terms of importance for your UX team
For example, if the concept is high on likelihood to use but low on effect or experience, then the idea may be too risky to pursue.
Expect that scores may differ across all audience segments
Meet with your UX team and discuss audience segmentation to determine which audience segments are most valuable. You may also include a discussion on the potential tradeoffs, depending on your concepts.
Take the opportunity to also check the differences between segments. These can help your team parse ranking scores with slight variation between them.
For example, the concept may perform well in younger age groups, while the other concept may perform well in older age groups. You may then recommend pursuing both concepts, so you address a more comprehensive range of people.
The user input should only be the one consideration when making the decision.
You can balance the research with other elements like materials cost, team goal metrics, product team interest in a concept, etc.
All these should be discussed with stakeholders when it comes to prioritizing things, mapping concepts, etc.
Conclusion
This article is a good head start to help you put together a list of recommendations for your product team.
We recommend that you organize everything into three sections:
· Ideas worth pursuing
· Ideas that need more discussion
· Ideas to dismiss and why
When done right, quantitative concept testing is an approach that getting insights from your target market helps your team prioritize a product concept efficiently.
This testing method reduces any wasted efforts and increases the probability of launching a new or existing feature in a product or service.
