How to improve survey response rates

By QualitasResearch

Why do we care about survey response rates?

Survey researchers have always been concerned about response rate. It may not sound very exciting, but it’s important to us. Yes, really! It’s because a low response rate can (though not always) reduce the validity of survey results. In other words, the degree to which a survey actually measures what it claims to measure.

If a survey is lacking in validity, the results have little meaning. And this is a problem if you want to use your survey findings to learn something, or make generalizations from them, to make improvements to something, or inform decision-making. Response rate isn’t the only thing that affects validity – but we’ll discuss all of that in another article.

For this reason, researchers have tried to identify the factors that influence survey response rates. In this article we will summarize what research on this topic has shown. First of all though, what is a response rate?

What is a response rate?

A response rate is the number of people that complete a survey divided by the number the survey was sent to (the sample). It is normally expressed as a percentage. So, for example, if you send an email survey to a random sample of 2000 physical therapists in Canada and (after sending 2 or 3 reminders) you get 700 completed questionnaires returned, you have a response rate of 35%.

Let’s now look at the key factors that can affect survey response rates.

What influences survey response rates?

The research on this topic shows that important factors are:

Financial Incentives
Survey Length
Respondent Pre-notification
Follow-up Contacts
Topic relevance

Financial incentives

It is fairly common in survey research for survey respondents to be offered a financial incentive to encourage them to complete a survey. This can be cash, a voucher, or a lottery where respondents are entered into a prize draw for the chance of winning a prize. Another incentive that we’ve used is to make a donation to a charity for each completed survey. This might depend on who the respondents are, as well as the survey topic.

Does financial incentive improve survey response rate?

Yes! (Especially if it’s cash in advance.)

Which works best – cash, vouchers or lotteries?

A token cash incentive in advance has a considerable positive effect on response to mail surveys. Your response rate can double. Providing a cash incentive in advance (such as sending $2 with the paper survey) is more effective than the offer of a gift certificate or a lottery approach (often used in web surveys).

What about web surveys?

Financial incentives have a positive effect on response to web surveys, but the effect is small (less than 5%). Online surveys are not suitable for the most effective incentive (immediate cash reward). In web surveys, the incentive is typically a voucher or lottery on completion of a survey. If using a lottery approach rather than a voucher, lotteries with small prizes, but a high chance of winning, may work best.

Survey length

In mail surveys, questionnaire length has a big effect on response rate. Each additional page reduces responses by 5%. Response rate drops considerably after 4 pages. So, if you’re sending it out by mail, you might want to keep the questionnaire short! Questionnaire length may also affect data quality, if respondents become bored or tired.

For online surveys, short surveys have a higher response rate, but longer surveys (within reason, of course!) can still generate a reasonable number of responses. There is a small correlation between perceived length of an online survey and the number of questions answered. This is something to keep in mind when designing an online survey. You don’t want it to appear too long.

The type of question also matters.

Having to write in responses to open-ended questions requires more effort, which can prompt less motivated respondents to quit the survey. I have to admit that this has caused me to drop out of a few online surveys half way through. It’s a good idea to keep open questions to a minimum, unless you have highly motivated respondents. The topic must be of high importance to them to invest this much effort.

Respondent pre-notification

Contacting respondents (by letter or email) before sending a questionnaire can increase response. It also increases the speed of response, for email and mail surveys.

Making questionnaires, letters and emails more personal helps improve response to a survey. It’s a good idea to send personalized invitations to a survey.

Follow-up contacts

With mail surveys, follow up contact improves response significantly, and so does mailing a second copy of the questionnaire to non-respondents. Sending multiple reminders is effective for getting more responses – and better than sending only one.

If you’re mailing a survey, make sure you include a return envelope. If budget allows, sending the questionnaire by registered mail can have an effect on response rate too.

Topic relevance

Questionnaires are more likely to be returned when the topic is of interest or relevance to the respondent. The survey topic is strongly related to response rate, for mail, online and email surveys. It’s important to clearly indicate the relevance/importance of your survey in the information you provide to potential respondents.

Summary