Simple. Specific. Short.
Every question you include in your community engagement surveys, whether open-ended or closed-ended, should meet these three basic criteria.
These characteristics are critical for three reasons:
They ensure that survey participants understand what you are asking,
They ensure that they can answer fully and confidently, and
They ensure that they can provide the information you’re seeking for decision-making.
Because each of these criteria has several considerations, we’re breaking them up into bite-sized bits. In this edition, we’ll focus on the first question requirement: How to keep it simple.
Let’s walk through a few best practices to simplify survey questions.
Lose the big words, acronyms, and jargony language.
Your survey shouldn’t mimic the SAT, so your questions shouldn’t include words that get you bonus points in Scrabble. Sometimes people wrongly assume that big words help establish authority and demonstrate that they are trusted experts. The result is often the opposite: The use of complicated words tends to alienate respondents and leads to lower response rates.
Acronyms and jargon are common in a lot of industries, including government. It’s easy to forget that our work language sounds foreign to others. You may know what TOD or BRT are, but your survey respondents probably don’t. Most respondents will skip the question or close the survey altogether if they don’t know what you’re talking about. More motivated respondents may turn to Google for answers, but there’s no guarantee they’ll find the right answer. For example, when you search “TOD,” the first page of results shows information about ‘transfer on death.’ This is of little help for transit-based surveys. Instead, explain the concept in plain language.
Plain language means more than simply spelling out the acronym. You may think that spelling out ‘Transit-Oriented Development’ is the answer. But that offers minimal clarity to the average person. “Building spaces that have housing, jobs, and services together near public transit” is a lot easier to understand. Don’t worry so much about being technically correct; focus more on creating a shared understanding with your survey participants.
Survey questions should require minimal, if any, background knowledge. Anyone–whether it’s your 16-year-old daughter or your 86-year-old grandfather–should be able to read, understand, and answer your survey questions. Respondents should not have to do extensive research or have a dictionary nearby to participate.
Avoid double negatives.
“Do you favor or oppose not raising taxes to support affordable housing?” Confused? So are survey respondents. Nothing trips people up more than double negatives, or the use of two negative elements in the same question.
If they favor raising taxes to support affordable housing, their response to this particular question would actually be “oppose” because, technically, they oppose not raising taxes. This is counterintuitive, leading to unnecessary confusion. The question becomes much more clear by simply removing ‘not’ and rewording the question to read, “Do you favor or oppose raising taxes to support affordable housing?” Like we always say, it only takes one word to bias a question. This is the perfect example of the weight of a single word.
Because there’s no way to know how many respondents misread or misunderstood the question, you can’t be confident in the results you got from double-negative questions.
Ask one question at a time.
Another common survey mistake that can confuse respondents is when multiple questions are squeezed into one; that’s called double-barreling. Questions like, “Were the meeting time and location convenient for you?” seem straightforward but they can be troubling for respondents who may have found the time convenient but not the location, or vice versa. How should they respond? Separating this into two questions (“Was the meeting location convenient for you? and “Was the meeting time convenient for you?”) would make it easier for them to answer fully and honestly.
Besides minimizing confusion, splitting up double-barreled questions provides you with better data. Let’s say that you ask your community: “On average, how often do you bike, walk, or rideshare to work?” If someone selects “twice per week” from the list of options, what have you learned exactly? You know that twice per week they bike, walk, or rideshare to work. You have gotten no insight as to which they use. Do they rely solely on a bike? Do they mix it up? Asking the question this way doesn’t provide you with these answers and limits your ability to draw strong conclusions about how survey participants get to work.
To make a question simple, ask only one question at a time. Resist the temptation to shorten your survey by merging multiple ideas into a single question. Not sure if you are asking a double-barreled question? Look for words like ‘and’ and ‘or’ to see if you’ve accidentally sneaked a second (or even third) question in there.
Conclusion
Keep it simple. Clear, easy-to-understand survey questions lead to higher participation rates and more specific data. If you want people to respond to your engagement survey, make sure your questions are clear and easy to understand. Confusing questions can discourage participation or prompt participants to fill out the survey based on best guesses. Either way, you lose a great opportunity to engage the community and get meaningful feedback.
Stay tuned for more on writing stronger survey questions.