Generative and evaluative research is very different. The goal of generative research is to define the problem you like to create and design a solution for.
Evaluative research, on the other hand, helps evaluate and understand an existing design, whether it is in prototype, final, or some other form.
In this article, Generative vs Evaluative Research, we will discuss further what these two research design types are, how they differ from each other, their importance, as well as the similarities of approaches that are associated with them.
In the world of UX research design, research is an important part of solving problems as well as narrowing them down to the “right” problems of the users.
Thus, a UX designer’s job is to better understand the users, which entails going beyond the initial assumptions and putting themselves in the user’s shoes to better design products that are needed and to provide a better user experience.
In the span of creating these products, UX researchers do not strictly follow any scientific methods. It is more about creating a hypothesis and testing them, allowing room to change prototypes as needed, and based on the outcomes of the research.
Having said that, there are research methods that are used to gather these data. And two of these research methods are the generative research method and evaluative research method, which we will explain more of them one by one below.
Generative research method
What is generative research?
Generative research is a method of research that helps the researcher develop a deeper knowledge of users and to better understand them in order to achieve its goal of finding solutions, opportunities, and innovations.
This research method is also sometimes called discovery research sometimes also known as exploratory research.
As mentioned, the goal of generative research is to look for opportunities, solutions, and innovations. These can be new products or experiences, or they can be just an update or improvement of an existing one.
When it comes to identifying new innovations or solutions, you need to define the problem you are trying to solve. This will help you understand how people live and behave, which includes their environments, attitudes, opinions, and perceptions.
When it comes to generative user research, keep an open mind since you have no idea of the problem you are trying to find and solve yet.
Why is generative research important?
Direct interaction with users allows to understand better the discovery of things like the users’ mindset regarding a specific problem, which later helps in finding a feasible solution.
Users are perceived from passive consumers to active participants in the design research and process of a product.
People are creative in nature and considered to be experts at understanding their own ways in life.
The generative user research method is a great starting point to develop innovative ideas through collaborative efforts.
How do you conduct generative research?
1. Concept validation
This is a process of showing the design concepts to users. The goal of this process is to let the users see and feel whether the product is of use to them.
This stage also allows the UX researchers to get valuable feedback and ideas from the users.
Validation of users’ needs, feedback, and ideas, helps UX research design researchers narrow down the design concept.
Here are some user research methods used to gather data for concept validation:
Stakeholder interviews
Diary studies
Focus groups
Persona development
2. User journey mapping
This is a timeline of actions that involves the relationship of your brand to your customers.
This is a visualization of your users’ interactions with your product, from their own point-of-view.
The user journey mapping creates a timeline of the relationship points between your organization and your customer.
Some of the mapping techniques you can apply are:
User case and workflow
Storyboard
Process design testing
3. IA and taxonomy creation
Taxonomy is the system where you classify and organize things. As a general concept, this aims to understand a set of subject-specific concepts and creates a vocabulary of these concepts. This helps in organizing the physical things related to those concepts, which makes it easier to find and interact with.
You can use the following methods to help you with taxonomy:
Card sorting
Tree testing
Qualitative and quantitative card sorting
User experience evaluation
Examples of when you can use a generative research method
When tracking users to record their activities, behaviors, and thoughts on a daily basis
This type of user research is made possible when the researcher follows the participant every day for a given period of time.
An example of this is shown in the case study of online grocery shopping for Publix, where the UX research team visited participants to their home and accompany them while grocery shopping to get to know their shopping behaviors and apply these behaviors to the online shopping app:
When you want to know the users’ attitudes, preferences, and opinions.
This is where you better understand your market’s point-of-view and use these insights to create better solutions and experiences.
An example of this Uber user research case study, where the study aims to collect riders and drivers’ behaviors in terms of pick-ups. They used the insights to build a better Uber app and add features that are needed by its users:
When you want to understand people’s actions, thoughts, and feelings using focused groups.
Focused groups are where participants can interact and communicate with others on a topic discussion, while the researcher gathers in-depth details about the topic.
Focused groups are important in understanding people and their motivations. However, this should never be used as a replacement for behavioral user research like usability testing or ethnography.
Here is an example of a focused group case study for Kitchen Garden:
Evaluative research method
What is evaluative research?
Evaluative research or also known as evaluation research is identified as a type of research used to assess a specific problem to ensure usability based on the users’ wants, needs, and desires.
The goal of the evaluation research method is to test an existing solution of an existing problem and see if this meets the people’s needs, is easy to use and access, and is fun to use.
This type of research should be conducted during the entire development cycle, starting from the early concept of design research until the final stages of the product design.
Why is evaluative research important?
The insights and results that you get through this research demonstrate the impact on the product funders.
This kind of user research suggest improvements of an existing problem for a better product and for any continued efforts.
It can seek further support for any continued product or program.
The gathered insights in the user design research might be useful data to be shared with other teams or organizations.
It helps determine if the product is appropriate to replicate in other locations with similar needs.
The types of evaluative research
Formative evaluation
This is a type of early evaluation research, which occurs during and after the product design, but before it is widely implemented.
This method is used to understand the need for any intervention or tentative decisions on how to implement or improve a product.
During the formative evaluation, it collects information, which is then sent back to product designers and administrations to develop and enhance the product and maximize the success of the intervention.
Process evaluation
The process evaluation addresses these questions:
What was done?
To whom?
How?
This is carried out at some point in a project, where we can determine how and how well the delivery goals of the product are met.
The process evaluation research plays an important role in improving interventions by providing the necessary information to change the delivery strategies or product objectives in a changing scenario.
Outcome evaluation
This method is used to identify consequences and to establish those consequences that are attributable to a product. This kind of evaluation answers the following questions:
What outcomes were observed?
What do these outcomes mean?
Just like the process evaluation, outcome evaluation can be conducted at intervals during an ongoing program, and the researchers see that such periodic evaluation is needed to monitor its goals.
How do you conduct evaluative research?
1. User interviews
In this type of interview, UX researchers question the users about the product and then record their answers. The nature of the questions depends on the intent of the research. However, the main goal is to get insights that can be useful for product ideation and development.
2. Ethnography
This is where you watch and observe the users’ behavior in their natural environment.
Ethnography is a first-hand observation of the user’s interaction with things around them. This is considered as an effective way for the UX research team to learn if a product is useful or has flaws and needed further development.
3. User surveys
These are a set of questions that evaluate the users’ preferences, opinions, and attitudes towards a product.
Surveys are given to a sample audience that represents a larger population. You can ask closed and open types of questions in surveys.
4. Card sorting
This is a very simple technique that provides insight into how users structure data in their minds.
In an open card sort, users are given a deck of randomly shuffled cards and are asked to organize them in whatever way they prefer.
Once the cards are sorted, the users are asked to explain why they arranged the cards in a particular manner.
During the closed cards sort, users are asked to place the cards from the deck to pre-existing groups. This is also documented by the assigned researchers.
Examples of when you can use an evaluative research method
When you want to ensure the design meets your users’ expectations as early as possible.
This can be achieved by getting feedback on working designs and prototypes. You can see an example case study below where they validate the design app to the users:
When you want to optimize existing experiences
This is usually achieved by asking your participants to complete the key tasks on live designs.
For example, this user research on election experience, where the UX researchers studied how the whole US election process experience is for people and how they can improve the user’s experience during the election period based from the risen problem that users encounter during the research process.
When you want to test real-life experiences.
Like the video example below showcasing real lab experiences for biomechanical students. This provides insights on how students can see and feel the importance of their course in real life:
Which is better, generative vs evaluative research?
There is no best approach when it comes to these two types of research. For one, the generative and the evaluativonresearch methods have different goals. Secondly, these research types are conductive at different times in the development and design process.
However, both are considered very important in UX research. The bottom line is: if you want to create a great user experience, you can try both generative and evaluation research methods.
Choosing the right method
When it comes to knowing what type of research to conduct: generative vs evaluative research or a combination of both, it is important that you know your goals and objectives.
If you have a clear mind of what you are trying to achieve, based on where you are currently at, as well as in your research goals, objectives, synthesis, and analysis methods.
Final thoughts
In a broad sense, we can get to conclude that the generative research method helps define and create a product or service that is based on the needs of its market.
On the other hand, the evaluation research method helps refine and improve an existing product.
The best time to apply the generative method is in the early stages of the design process before everyone is committed to the product design. Whereas, evaluative research is best used when the design is taken form and can be used by its users to gain further insights.
Generative research can be a one time off and in-depth, while the evaluation research should be done regularly throughout the design process, development, and product delivery.
It is important to know exactly the difference between these two types of research methods to avoid difficulty in the research process. Not all research goals and objectives can be achieved with combined methods of generative vs evaluative.
Evaluation research should only be applied when you understand the problems you are trying to address, when you want to create a specific experience, or when you strive to get the best implementation.
Additionally, evaluation research will not help you build the right product or service. Evaluation research is where you ensure that what you have built is right or whether you are building the right thing for your users.
There is no better research method as both generative vs evaluative methods are equally important in delivering the best possible product or service.
