How to analyze free-text responses to open-ended survey questions
In this article we will cover:
- Why we include free-text responses in surveys
- Why we ask open-ended questions in surveys
- How we analyze free-text responses
- Reporting the results
- Visualizing qualitative survey data
Why have free-text responses in surveys?
Free-text responses in surveys contain a wealth of information. Analyzing these free texts can help us understand how people feel about a topic and why they feel that way. These responses can also be helpful for guiding the planning of future survey questions.
Respondents’ comments can reveal unanticipated topics and issues that would otherwise have been missed. In free text, people get to tell us what they really care about and why, rather than how they feel about the points we chose to ask about. So, we find out what’s really important to them.
Why ask open-ended questions in surveys?
- To understand what is really important to respondents, and explore their opinions in greater depth.
- To gather contextual and detailed information about successes, barriers, and challenges.
- To gain a deeper insight into the reasons respondents responded in the way they did – such as why they agree or disagree with a survey statement, or why they like or dislike something.
- To collect data that covers topics or issues we were unaware of, or did not cover in our tick box questions.
- When we don’t know what closed-question responses to use.
Analyzing free-text responses
Compared with tick box questions, free-text responses in surveys are difficult and time consuming to analyze. We have provided data analysis support to organizations for this aspect of their in-house surveys. We’re highly experienced in this kind of qualitative data analysis and can work though comments thoroughly yet quickly. This includes responses in multiple languages which we translate prior to coding.
How we analyze qualitative data from open-ended survey questions
We always manually read through and code the responses. We don’t use automated tools for analyzing text. In our experience, manual coding provides far more accurate results and a better representation of the information collected – paying attention to the context of comments, and not just the frequency words or phrases occur.
Sometimes we use predetermined categories, when there are specific questions to answer. For example, a client might want to know the reasons for liking or disliking a particular concept. When using this deductive approach, additional categories are added as they are identified in the coding process. Other times the coding is inductive – themes are identified through reading the tests, and developed into codes or categories. We code the data using either NVivo or Excel. From this process, we develop a code book (a list of codes and categories). It’s best to have two or more researchers involved in coding the data. We can then compare each other’s coding, discuss discrepancies, and agree a coding frame. This helps ensure validity.
Further analyses are carried out, as appropriate to an individual project. This can involve thematic analysis for a more detailed account of the data, or quantifying the qualitative data – to calculate frequencies and percentages for each category or topic.
Reporting the results
How we present the results varies according to our client’s requirements, but can include:
- An Excel sheet of categories and sub-categories, frequencies & percentages
- Summary of key topics and themes with example quotes.
- A summary report presenting the data visually.
- A full written report, presenting findings from a thematic analysis.
Visualizing qualitative survey data
When preparing a summary or full report, we often present key themes visually, using a range of charts, diagrams and figures to describe or explain concepts. Some examples are:
- Infographics
- Quotation Banks
- Project and Mind Maps
- Word Clouds or Packed Bubbles
- Word Trees
- Flow Diagrams & Hierarchical Charts
- Coding Stripes
Summary
This article considered why and how we analyze data collected from free-text responses to open ended survey questions.
What we covered
- Why have free-text responses in surveys?
- Why ask open-ended questions in surveys?
- Analyzing free-text responses
- Reporting the results
- Visualizing qualitative survey data