Have you experienced meddling in the middle ground between discovery research and usability testing? In this case, you and your team members have several concepts in mind; however, they aren't fully formed ideas. You are torn about usability testing to see if you are going in the right direction, but the concepts are not just usable.
In this article, Concepting Testing UX, we will discuss that instead of usability testing, in the case where the UX team had an idea of how their users might or not be able to use a particular flow but still had little knowledge of the concepts' validity.
This article will also tackle the following sub-topics:
Concept testing UX is a research method that involves getting user feedback during the upfront part of the design process to give feedback on potential solutions. Concept testing UX at the beginning of product development allows users to share an idea's initial shaping to solve a problem.
For example, if a large bank wants to make it easier for customers to enroll in direct deposit, you would be fairly safe to assume that allowing customers to complete the enrollment process online from start to finish would be an easy win to solve this problem. However, with limited resources, including time, money, and available labor, decisions need to be made:
These are only a few questions of many that you can immediately come up with based on the concept of enrolling in direct deposit online, a relatively simple idea.
Concept testing allows you to reduce risk while increasing focus on user input and building a product to meet user needs. Concept testing does not take the place of being visionary; however, being visionary without insight from others might be analogous to throwing spaghetti at a moving target and getting lucky now and then concept testing will help slow down that target or increase your spaghetti-throwing accuracy.
When to use concept testing
Concept testing is the final stage of product discovery. It is a process whereby you introduce a product concept to your target audience to determine if it’s worth following through with the idea. What you’re testing could be as innovative as a new product, an entire ad campaign, a revamped web design, or new branding.
Concept testing can be applied at multiple stages of the product development process, such as during ideation, prototyping, defining marketing messaging, or just before the product launch. We’ll look into when to run a concept test in more detail below.
There’s no one way to conduct a concept test, either. Some teams opt for concept testing survey, while interviews and focus groups are useful for gathering qualitative information. The method you go for will depend on your goals and resources.
Concept testing involves evaluating, predicting, and understanding consumer responses and reactions toward a new or existing product or service.
The product concept test's goal is to minimize any risks, which helps the product's potential to raise profits. This strategy allows companies to guarantee that the product concept will be appealing enough and valuable enough to get more customer purchases.
Simply put, product concept testing is part of a marketing user research strategy many brands utilize to measure their target customers' engagement, readiness, and willingness toward their product or service. This applies to both existing and new products or service concepts.
The benefits of concept testing
Concept testing proves valuable because it validates that your customers want the product. Idea validation is essential to UX user research and a necessary step in creating successful products.
Customer response to your product determines its success or failure. If customers don’t like the concept during usability testing, they aren’t likely to buy it once it’s on the market. Getting their feedback during development is crucial to an effective launch. You can use customer feedback to make necessary adjustments to help your product succeed.
Concept testing is also highly flexible, proving useful to businesses of virtually any type or size. You can ask customers about any aspect of your idea to gain valuable insights into every product detail.
We could go on for days about the advantages of concept testing, but here are just a few of the reasons why this technique is an essential part of product success:
Save time and money
According to research from Harvard Business School, approximately 95% of new products fail. A whopping amount of resources go down the drain when a product flops, from the money spent to create it to the effort exerted by the product development team.
While concept testing requires a slice of your budget, that investment is far less costly in the long term than launching an idea that fails. For example, once a software product is launched, correcting an error can cost 100 times as much as it would have if it were fixed in the development phase. Studies also reveal that development teams waste 40 to 50% of their time on avoidable reworks, a reality that can be avoided with concept testing.
It provides actual value for early validation, contributes to prioritization, and eventually increases the overall quality of our product at a low cost.
Validate and launch with confidence
Concept testing also provides data to support your idea and get buy-in from stakeholders or other team members. Bringing a new concept to life can be daunting, and management may understandably be cautious about taking that risk. However, once you have the user research from concept testing on your side, even your most cautious team members will see the potential in your product concepts.
Discover new business opportunities
Concept testing can expose a new path for your company, whether optimizing your product's existing features or pivoting entirely.
Your assumptions about what makes your concept great are just that—assumptions. So usability testing shows what aspects of the idea appeal to your customers, which you can then capitalize on when developing or highlighting the concept's best attributes in your marketing campaigns.
Even a concept test that appears to fail still has value. Once you evaluate what matters to your target audience through concept testing, you can use those learning opportunities to tweak or completely change your product to maximize its success.
Concept testing is an iterative part of the product development process and can be done at several stages to test and validate key decisions. You can and should use concept testing multiple times in the process, from ideation up until launch.
Here are some specific points in the product development process when concept tests can be run:
At the discovery and ideation stage
Concept testing is handy during the product discovery stage because it narrows down the solution you should move forward with for a particular user problem.
Concept testing is an important part of the ideation phase when we’re testing alternative solutions to a problem, and it can provide incredible insights into what customers want.
Testing at this stage helps you check your team’s assumptions and validate product ideas with customers before moving into the development stage.
During the design stage
Once you’ve started designing your product, there are numerous opportunities for concept testing to be of use. You can test the particulars of design concepts, from low fidelity, such as the layout of a low fidelity web page, to high fidelity prototypes, like user flow, graphics, and branding.
Early [design] concept testing allows for multiple iterations quickly and easily, making the final developed concept more precise and better optimized.
Before the product launch
It’s never too late to conduct a concept test. Running a concept test before a product launch allows you to catch any last-minute changes that need to be made. A pre-launch concept test can also help you figure out which marketing ideas to move forward with your launch campaign, from messaging to graphics.
Once you have decided on concept testing, there are a few best practices to follow:
Step 1: Setting the objectives
Knowing what you want to learn ensures you choose the correct method and helps you outline the rest of your test. With clear goals, your tests will have focus, and you will get the information your team needs by the end of the concept test. Some sample objectives for the retirement plan example might be:
Compare the understandability of the three new retirement plans
Examine how participants currently feel about signing up for retirement and how they respond to the new concepts
Identify pain points and confusion with the three new retirement plans
Step 2: Recruiting participants
Once you set your objectives, you must figure out who you want to recruit. Ideally, you are targeting people who would be using the product/feature in the future. They can be users of your current product/service or a competitive product with similar concepts to what you are testing. Creating a screener survey is the best way to ensure you get the right people. Ideally, in the survey, you will speak to about ten people per segment for concept tests to get the most robust survey results.
Step 3: Reviewing the concepts
We recommend for the concepts be completed three to five days before the first test. By having everything ready ahead of time, we have time to review the ideas we will be testing. Examining the concept with the designer gets me familiar with the flow and the information they want from the test.
Step 4: Outlining the flow of the test
Once the concepts are reviewed with the team, the test flow should follow. If testing more than one concept, ensure you vary the order. Here is a breakdown of a 60-minute concept test:
Introduction: 3 minutes about the test, who I am, signing an NDA/consent form, instructions
Warm-up: 5 minutes for asking general questions to get the participant in the conversation mindset. Some warm-up questions are, "what hobbies do you love?" "what do you do in your free time?" or "what have you watched recently that you loved?"
General questions: 10 minutes to focus on the problem space of the concepts. For example, if we were testing different meal kit plans, we would use this section to ask about meal kits or cooking habits in general.
Concept A: 20 minutes focusing on concept A. Start with general impressions and have participants explain the concept to me as they would friends or family members. Then, dive deeper into the different details of the concept.
Concept B: 20 minutes focusing on concept B in the same way as above.
Follow-up (optional): A 5-minute buffer to follow up on the concepts and check if the participant has anything else to discuss or add.
Outro: 2 minutes of thanking the participant, answering their questions, and explaining any next steps, such as when they can expect the incentive.
Step 5: Create questions
Once there's an idea of the flow, time to create the questions for the general section and concepts A and B. Here are some common questions during concept tests:
Walk me through your overall impression.
How would you describe this to a friend or family member?
Have you seen something similar (ex: a competitor)? What do you think of the competitor? What do you think of this?
Have you used something similar to this in the past? Why or why not?
Walk me through how you have used something similar to this in the past? What was great? What was terrible?
What is confusing?
What is missing?
If you could change anything, what would you change?
Step 6: Think about quantitative data
Since we are talking about qualitative concept testing, it is essential to consider adding quantitative data into the mix. For example, you can ask participants for a satisfaction score at the end of each concept. Additionally, you can follow up moderated testing with unmoderated testing to bolster your numbers.
Step 7: Do a dry run
Once everything is together, conduct a dry run to ensure the flow makes sense for the team and participants. Sometimes dry runs can uncover gaps or questions we didn't think to ask. Always do a dry run before the first participant, no matter what test to conduct.
Step 8: Synthesize the data
Once the concept tests are over, it is suggested that you go back to each question asked and write each participant's responses. For example, "Have you used something similar in the past?" Tally up how many people did or did not. Then, for the "why or why not" follow-up question, write down the answers and cluster anything similar between participants. This method gives me an idea of the overall responses and feelings toward each concept and which one resonates better with participants.
Moderated versus unmoderated testing
As mentioned above, unmoderated testing can be a great way to get a larger sample size to answer your questions. We recommend starting with moderated testing, as it will give you deeper insights than you can glean from unmoderated testing. By proceeding in this order, you can also clean up the script to make it as efficient for unmoderated testing as possible.
Here are a few tips on testing and validating your idea with your target audience.
Set a goal
Just trying to gauge if customers “like” your product concept is a bit too vague. Come up with a precise aspect to measure like, Does my target market find my product useful enough to purchase it?
And if you want to take this a step further and be more specific, set up targets for your tests. For instance, if you’re concept testing a new feature idea, your goal could sound like this:
“We want to validate that most of our users (80% or more) find this feature useful.”
Being precise with test goals will allow you to quickly assess if a concept is worth moving forward with or if you need to tweak it.
Ask the audience the right questions
You must carefully consider the language used in the test and how questions are phrased so that customers can understand them and you'll receive rich feedback. The right user research questions will reveal the insights necessary to validate your concept or evaluate what it’s lacking.
Poorly conceived questions like, Did you prefer Product A over Product B? will leave you still asking, Where did we go wrong? While the previous question may seem benign, it introduces an inherent bias towards Product A. Asking simply, Which product did you prefer? will get you an unbiased answer.
Tap into your target audience
There’s no point in conducting idea validation with participants who don't fit the demographics of your customer base. That’s why recruiting the right participants is key. With their feedback as guidance, you can develop and launch a successful product that meets customers’ needs.
Since qualitative concept testing allows you to ask broad questions and review multiple concepts with users, you’ll have a lot of data to inform your project.
In addition to better understanding how your product or experience can address your customer's needs, you should have a better idea of what concept (or what parts of each concept) align with those needs.
This understanding will allow you to narrow in on a single concept or iterate to create a new concept that’s a better fit.
Conclusion
In a nutshell, the goal of concept testing is to develop, refine and nurture new product and service initiatives around solid consumer insights to reduce the probability of failure and increase the probability of a successful launch and in-market performance.
The ability to better understand consumer needs is more relevant than ever before. Successful concept testing improves new product development, optimizes marketing strategy, reduces time to market, and keeps customers coming back for more.
Remember that concept tests are not about preference testing or seeing what concept users prefer. When concept testing is done right, it saves time, money, and energy. Most importantly, this essential, money-saving method lets you know you’re on the right track and gives you the insights you need to move forward confidently.
