It is always such a relief when you’ve successfully completed your study. Whether you conducted surveys with existing users or had conversations with potential users, you’ve had a lot of information on hand.
The next logical step would be: what do you do with this information?
This article, Analysis of User Interviews, will tackle just that--- How to communicate user research findings so stakeholders can perfectly understand the data the way that UX researchers do. And more importantly, utilize the data to improve the user experience. In this article, we will also discuss the following points:
In this article, we’ll go over everything a UX researcher needs to know when it comes to data analysis. This will include using the data to tell an interesting and meaningful story and crafting deliverables that are appropriate for stakeholders. We will also discuss several best practices for creating the deliverables.
But before we delve any deeper into these topics, let us begin by defining what user interviews are.
What are user interviews?
User interviews are considered qualitative UX research methods conducted to better understand the users who interact with the company, and its products or services.
In UX user interviews, researchers will select and interview the target user to find out about their views on a topic by asking several questions. The interviews are usually recorded.
Unlike the focus groups, research interviews are conducted one-to-one and can take place in person, over a video call, or via a telephone call.
The purpose of user interviews is to find out the following data:
User experience
User behaviors and interactions
User beliefs
User likes and dislikes
Aside from the following data above, a user interview can also be used to explore a user’s immediate reaction to a stimulus. It provides insights on their reasoning as to why they would act in a certain way.
And by taking a semi-structured or structured approach to user interviews, researchers can quickly and easily obtain user experience insights to help improve products and services.
There are two types of user research, and these are:
Qualitative research
These are interviewers that collect non-numerical, qualitative data. It is often conducted with open questions and face-to-face interactions.
The qualitative type of research provides emotional or experience data that provides insights into a user’s emotional decisions and how they perceive the company, brand, product, or service.
Quantitative research
These are interviewers that use numerical data from sources that collect information over time and convert it into datasets or closed questions on a demographic survey.
This quantitative kind of research provides economical or operational data such as user demographic metrics and historical census reports.
Both the quantitative and qualitative kind of research are useful for supporting business UX research decisions.
The qualitative research provides answers to how user think and behave, which explains ‘why’ users behave in a certain way.
The quantitative research provides insights on the user’s past activities and answers to ‘what’ happened.
You also need to understand that it is common for one type of research to support the other since quantitative data provides context to qualitative experience, while qualitative data provides meaning to the fact-driven findings of quantitative research.
The importance of conducting user interviews
There is no doubt that user interviews are important to the improvement of the overall user experience (UX) because they provide critical insights into what your users think about your product, service, app, or website.
User interviews are used for several reasons:
Provide general insights about what a user thinks about your products or services. These insights give the company information as to why users connect or don’t connect with the company’s products or services.
The information gained on a user can also be utilized in your marketing personas or user journey mapping models for better user engagement planning.
Investigates how a user will react to a new product, an added feature, or a solution to a known problem. The insights from these data give the UX teams the confidence to move ahead with designs.
Through user feedback, you get the users’ perspective on your products or services. This will guarantee that you meet the user’s needs.
The insights from user interviews can help in business decisions, as users are better understood.
User interviews are quick and cheap to do. UX researchers can easily conduct a user interview anywhere and get results real-time.
Data are easily gathered and collected
UX researchers can collect data by using digital tools and end-to-end software that are readily available.
These tools help collect and analyze data using integrated dashboards, which help researchers keep organized and at the same time find results faster.
New data can also be uploaded in real-time to help guide new strategies and changes, resulting in a proactive user experience.
Simple to conduct
User interviews are very simple to do. No matter what kind of interview you are conducting-- whether it be structured interviews or semi-structured interviews, all you need to do is to form questions ahead of time, interview a user and record their answers. No special training is required for conducting a user interview.
The interviewer should accurately remember and recall the user’s answers to fully understand the information received. However, it is possible that the interviewer may recall the answers incorrectly. Likewise, the interviewer can also interpret answers and bias their own recall.
Room for distraction
If the interview is handled by taking notes of the insights, the interviewer’s attention may be diverted, missing out on some vital user insights.
Additionally, jotting down notes could also distract the user and make them feel uncomfortable speaking.
Some user questions can result in unreliable insights
One example is asking a user about his thoughts on how a product can be used in the future. The insights can be unreliable as it asks the user to speculate beyond their knowledge.
The same can be said for design questions. You must understand that most of these users are not familiar with design processes so the responses are not accurate reasons why a design should be changed from one version to another.
To avoid this scenario, it is important to conduct user observation by watching the user’s behavior with a designed solution. This will help UX researchers understand if the new design is working.
Sample can be limiting
The sample size is always dependent on the number of interviewing staff, therefore can be limiting. The sample size can also be affected by the area in which interviews are conducted and the number of qualified respondents in the area.
As a resolution, we suggest that you conduct your interview across multiple areas to get the quantity and quality of data you are looking for.
Indicators of a good insight
How do you know that your analysis was a success? While there is no hard rule of accurate measures to determine a good insight, you can take the following indicators that show the right direction:
Data should be trustworthy
For insights to be reliable, you need to prepare data that is based on evidence from your user interviews.
Be careful of the common problem such as cognitive bias. This is the tendency to confirm what researchers already believe. This can distort the data analysis and create misguided decisions.
Despite the importance of ground your research insights in evidence, you should always remember that there are limits to qualitative data.
For example, user interviews will not yield statistically significant results. It will rather focus on the strengths of qualitative data in revealing causal relationships, emotional states of users and thus far unnoticed perspectives.
Relevancy of data that should fit to the UX research goal
The span of finishing Interview analysis will likely take several hours to over the span of days.
Since this timeframe, it is likely easy to get distracted in detail, losing sight of the bigger picture. This may result in research findings that have nothing to do with the initial research questions.
To avoid unintended distractions, always be reminded of the main questions to be answered. It is useful that you make them visible during the user interview process.
Uncovering unexpected and hidden insights
In UX research, it is normal and at the same time okay if you’ve found your insights to confirm previous beliefs.
However, be careful of changing evidence so you could produce new insights. Similarly, it is always recommended that you take a deeper look into your data instead of just scratching the surface. It could reveal unexpected results or entirely new topics that can be useful for the team. These unexpected findings multiply the value of user interviews.
We have known that user interview analysis is an important step in any UX research project. However, this doesn’t mean analysis only happens during this designated time.
We know that the human brain can immediately pick up and process new data by making sense of things. Ideas can spark anytime while observing emerging patterns during the interviews process.
Additionally, a team recap after each interview is another opportunity to identify early ideas while memory is still fresh. We recommend that you capture all these ideas immediately as they arise.
There are two possible approaches when it comes to creating main analysis within the course of the UX research:
Analysis in one go
This approach is creating the analysis after all interviews are completed. The data needed is readily available from the start, which makes it easier to recognize patterns as more related evidence becomes available.
You can expect to get a longer block of analysis with less interruptions for this strategy. The only disadvantage of this approach is getting fatigue and eye strain across the teams.
Batch-wise analysis
This approach divides the interviews into batches. Thus, you can expect shorter analysis sessions after each batch.
One advantage in this approach is that you can always adjust the research questions of the upcoming interviews.
This approach also allows preliminary results to stakeholders or is often used when there is not enough time for analysis after the last interview.
From a practical standpoint, the advantage of using this approach is that it is easier to find several shorter blocks of time for busy stakeholders than one large block.
However, a disadvantage of this approach is the higher switching cost. You need time to mentally prepare the analysis. In particular, it is a disadvantage to get evidence into short-term memory, which happens multiple times in user interviews.
So how much time should be allocated for analysis? The answer will always vary. UX researchers should avoid underestimating the time and at the same time reserving too much time.
We recommend you decrease the scope and focus on the most important topics first. On the other hand, you could always do more analysis while timing the sessions.
User interview analysis: A step by step guide
Let us now start analyzing our research data.
First and foremost, you should always know the importance of good note-taking. Why? Because well documented notes and data is the basis of a good analysis.
Make it a habit to consistently document all interviews. When it comes to working collaboratively with stakeholders, make sure that you are able to block sufficient time in their calendars and inform them up front about what to expect.
The first step goal is to prepare you to connect and get the data into the short-term memory. This is like loading information into a computer and work with the loaded data.
In practical terms, this means carefully reading the interview notes. We recommend that you also get the other team members involved in the interview phase to take notes.
To make the familiarization into a team activity, assign each stakeholder to a participant, let them read through the respective notes, and present themselves from their assigned participant’s perspective to the team.
Make sure you take time to discuss each participant with the team. Expect there are usually more interviewees than team members, thus repeating this multiple times is recommended.
In this section, we show you four techniques that will serve as starting points for you. You can flexibly utilize and adapt these techniques based on your needs.
Techniques to analyze user interview data
Structure data into themes
To make the qualitative data comparable across user participants, you need to assign the user responses to more generic themes or thematic analysis.
In UX research, qualitative data is considered unstructured and very difficult to analyze, thus the need to assign these to generic themes (thematic analysis).
You can utilize the topics that you initially asked during interviews. These make good starting points for the thematic analysis.
When it comes to working with digital tools, you can use tags as a practical way to assign notes to thematic analysis.
A tag is a label that indicates which theme a note belongs to. With regular text editing software, tags are not usually allowed. In this case, you can use a spreadsheet or a dedicated user research tool instead.
Is it necessary to come up with the tag names before starting with the tagging? Or create the tags on the go?
The answer is both. Both are possible. UX researchers usually need to iterate over these tags when working through the data as new themes come up or two themes merge into one.
After tagging the first few interviews as a team and building a common understanding, you may want to split up and do the remaining tagging in smaller groups or by the individual in order to progress more quickly.
If you do not have the software to do the tags, you can manually do tagging data through the use of post-its. When using post-its, write one response per post-it and place similar responses together on the wall.
Make sure you take note of the participant’s name or a short sign to have a link to the raw data and be able to get the context again. Using colors are also great for segmentation.
Look for cross-participant connections and cluster related evidence
After you have organized the data into thematic analysis, you can start digging into each of these themes separately.
You can use filters in software or apps to focus on one tag in particular and then look for commonalities or contradictions among the responses.
Encourage the team members to share their thoughts. This can help form new concepts and understanding.
The next thing you do is to pull together related observations into clusters, or a method called affinity mapping or affinity diagramming.
This method enables you to connect pieces of evidence to build up a broader understanding.
When it comes to working with post it notes, just rearrange them to fit your observed clusters.
Affinity diagramming can be a challenge to do in a spreadsheet. A good workaround is to put the cluster a note that belongs to in an additional column next to the tags.
It usually takes one hour or more to work through each topic, depending on the amount of data at hand.
Make sure that you pay attention to newly emerging themes and be prepared to split up or unify existing ones.
Just remember there are no hard rules to doing this process. You can easily start over with a newly discovered theme. This is one way you iterate your way towards the insights.
Use segmentation to reveal underlying patterns
Getting different perspectives when it comes to looking at and analyzing data is very helpful, especially when you want to have a deeper understanding of a topic.
Do not forget to use the metadata about the participants. These data can be a key to discovering several hidden patterns.
Some metadata include the participants’ job title, the size of the company, demographic data, or level of experience with a certain product.
Analyze across themes
Besides changing the perspective, another method to use when it comes to getting a clearer understanding of your user data is analyzing the raw data across themes.
Once we have identified themes and had a deep dive into each in the above previous steps, we can now zoom out and look at the bigger picture.
We need to identify how the themes relate to each other. We also have to understand the theme's relative importance, chronological order, or causal relationships. You can use color-coded post it notes to indicate a theme and put the respective name on it.
After doing the extensive analysis phase, the next thing to do is to utilize these insights and the final step is to turn what you learned into a tangible output.
Here are the reasons why you need a tangible raw data output:
The data is easier to convey insights to stakeholders, especially to those who were not directly involved in the project. This also helps the stakeholders retain your findings.
This will initiate the transition towards putting the insights into actionable points and helps move things from learning mode to doing mode.
The best form of output depends on your initial research questions. Here are some commonly used outputs:
List of pain points and opportunity areas
User journey such as highlights and lowlights
Jobs to be done
User personas
Have a concrete plan of the next steps to convert your data findings into action. This could be a decision, prototyping session, or design talk.
Also, you may want to think about how to store your data and findings so they can easily be accessed anytime. This will also make it easier for stakeholders to go back and look up certain aspects of the research at any time.
Common mistakes to avoid
We all know that experience is the best teacher. Thus, we highly recommend that you always practice improving your user interview analysis skills.
But here are some common mistakes to avoid to help you improve quicker:
Be careful in terms of quantifying data
Be careful with formulating quantitative statements based on qualitative data. This is true in terms of percentage values. This easily leads to wrong generalizations about market sizes.
For instance, “35% of the participants mentioned data security as the problem.”
User interviews aren't designed to yield statistically relevant results. The best way you can do this is to create hypotheses about the general market that can be validated using quantitative data.
If you want to use figures, just stick to the “10 out of 100” format to remind your audience that the total numbers are low.
Generally, using numbers is not a bad idea. It helps spot outliers such as an opinion that only one participant had.
Be clear all the time
Qualitative data is complicated as it is. However, after analyzing the data, the goal is to make everything crystal clear in the end.
As UX researchers, we need to reduce uncertainty as much as possible. You need to have enough certainty to make a sounding decision about what to do next: And that is to turn what you learned into a tangible output.
Conclusion
As UX researchers, we all have our own understanding of user needs. We are all constrained by our biases and mental models.
The process of looking for patterns in what the users are saying through user interviews helps us in thinking why they might be saying those things.
We unlock the underlying opportunities, and pain points, and unlock opportunities that your users have by analyzing the data we gathered from user interviews.
Utilizing the available software or using old-fashioned sticky notes or excel sheets are important tools to make our data analysis grounded through evidence and facts.
We hope that this article made it easier for you to make sense of your user research data and identify underserved user needs faster.
