Elevate Pro Tips for Building Useful Surveys
Surveys are one of the most common data collection tools we encounter at Elevate, and for good reason! Surveys can be a helpful and relatively inexpensive way to gather quantitative (and sometimes qualitative) data from respondents about their attributes, perceptions, knowledge, attitudes, and behaviors. Surveys, when poorly designed, can also generate data that is not useful or usable. Here are some of our recommendations for designing quality surveys.
Avoid these common survey question mistakes:
Double-Barreled questions: Double Barreled questions are questions that touch on two ideas but allow for only one answer. These questions can create inaccurate responses because the respondents could be responding to one part of the question and not the other. For example, the question “How satisfied are you with your pay and job conditions?” is double-barreled because it asks about BOTH pay and job conditions. The best way to fix this is by separating the double-barreled question into two distinct questions.
Leading or Biased questions: Leading or Biased questions limit or guide responses, typically by building in assumptions. While the intention of these questions may be to gain specific information, they can create a narrative that you are attempting to get a specific answer from the respondents. For example, the question “Would you vote for a woman for President if she were qualified in every other aspect?” implies that female presidential candidates are generally unqualified. Using neutral language to make assumptions explicit or offering assumptions as a response option are solutions for these types of questions.
Develop well-crafted response options:
Mutually Exclusive responses: Mutually exclusive response options ensure that respondents’ answers only fit into one category. This reduces confusion for your respondents and makes analysis much easier. For example, in the question below a respondent who eats out 1 time per month would have to decide whether to select the first option or the second option. Double-check your response options to make sure that none of your categories overlap.
Exhaustive responses: An exhaustive list of responses provides all possible options that could make up a response list. This ensures that respondents have an option that accurately captures their particular situation. In the question below, if a respondent had any other reason for why they play baseball outside of the ones listed, they would be forced to skip the question or choose a response that doesn’t reflect their experience. Simply adding an “other, please describe” option can be an easy way to ensure that all of your respondents could answer the question.
More Survey Pro Tips
Limit open-ended/qualitative questions. Qualitative questions both increase the burden on respondents and the time it takes to analyze survey responses. It’s important to be clear about what you hope to learn from qualitative questions and how you plan to use that information before including them in your survey.
Consider participant fatigue and its impact on data. The longer a survey is and the more difficult it is to complete may lead to respondents skipping questions or not paying close attention to question responses. This may show up as missing data or inconsistent responses that make it difficult to draw conclusions from the survey.
Pilot test for accessibility and cultural relevance. Have colleagues or a few members of the group you are trying to reach with the survey read through the questions and provide feedback before you deploy the survey. They can provide important insight on reading levels, fonts/font sizes, and word choices that may create challenges for respondents as they complete the survey.
Use incentives to increase the response rate. It may be obvious, but offering respondents something in exchange for their time completing your survey can really boost the responses you receive. One way to structure survey incentives is to enter respondents who complete the survey in a raffle for gift cards, experiences, or other small tokens of appreciation.