Strong survey questions have three basic characteristics: They are simple, specific, and short. We’ve already covered how to write simple questions. We’re now pivoting to the second criterion of strong questions: Keeping it specific.
Like simplicity, specificity is critical for three reasons:
It ensures that survey participants understand what you are asking,
It ensures that they can answer fully and confidently, and
It ensures that they can provide the information you need for decision-making.
Here are some tips on how to write specific survey questions.
Keep It Specific
In order to accurately respond to your survey questions, respondents need to understand what you are asking. Part of this understanding comes from question simplicity - like not using technical jargon or double-barreling. But the ability of respondents to understand your question also depends on the degree of specificity you provide as a proper framework for their responses.
A good way to think about specificity is in terms of the parameters, or boundaries, that keep a participant’s response relevant to the question you are asking. It’s like when you’re trying to make plans with someone; it’s far easier for you to suggest a time when they’ve indicated what times and days they are free. These are the parameters that guide your suggestion of when to meet up.
Common parameters in community engagement surveys involve questions related to who, when, and what. If you want to get useful data from your survey, clarify what these mean by being specific about each, as you’ll see below.
Setting Parameters
As you write your survey questions, consider all the different ways that respondents could interpret the words you’re using. If there are certain interpretations that you want to encourage or discourage, you will want to provide the specific meanings of the words used.
Consider this common survey question:
How often do you use public transit?
Always
Often
Sometimes
Rarely
Never
While this question seems simple, it isn’t specific enough to tell respondents what they should consider when answering the question. What time frame are we talking about? Is this on average during a week, a work week, a month, a year? Is this pre-COVID? Now? What do you mean by public transit? Does that include scooters? What is considered rarely? Often?
Can you see how confusing it might be for respondents? When you clearly articulate what you’re looking for, it’s easier for respondents to understand how to answer and they’re more likely to do so. This also provides you more accurate data because they are actually answering the question you’re meaning to ask.
To set parameters, include every relevant clarification as part of the question. These parameters may include time ranges, units of measurement, definition of ‘you,’ and what’s being measured. A stronger question would be something like:
Specificity Strengthens Data
If survey respondents have to guess what your words mean (like what “often” means) then they’re not answering your question the way you intended. Questions shouldn’t force respondents to guess. You won’t get accurate data or be able to draw accurate comparisons if you can’t guarantee that respondents are interpreting the questions in the same way.
If you want to draw conclusions about apples, we need to make sure we are all talking about apples.
To illustrate this, let’s revisit the original question: How often do you use public transit?
If I were to answer that question today, I would say ‘sometimes’ because I hop on an ebike provided by a local bikeshare program once in awhile. I’m not necessarily clear on the line differentiating sometimes from rarely, so I define it myself.
Someone else takes the same survey and says that they ‘sometimes’ use public transit because they take the bus when they don’t feel like paying for parking. Like me, they don’t know the difference between sometimes and rarely, so they also use an arbitrary definition.
What you see in the data are two people who say that they ‘sometimes’ take public transit. But you don’t have the context that one is thinking about ebikes and one is thinking about the bus. If you conclude that these two respondents have similar habits, you would be wrong. If you concluded that the first respondent (me) sometimes takes the bus, you would be wrong again..
You would get much more accurate data by using the second, more specific question about how many days we take a trip on a public bus. While specificity doesn’t guarantee that all respondents will interpret the question as you intended, it makes it more likely that they will and will strengthen your data.
Conclusion
Keep it specific. Respondents should not have to guess what type of information you are asking for.
Writing specific questions helps you get better data and helps your respondents understand what you’re asking. If you’re not sure whether your questions are specific enough, ask a friend for feedback: “What does [insert question] mean to you?” If their answer is different than what you intended, you may need to add specificity.
One caution: Parameters are helpful but if they’re too strict, you risk biasing the question and unintentionally pushing respondents into answering in a way that may not reflect their perspective.
Stay tuned for more on writing stronger survey questions, subscribe to the NineteenEleven blog, or schedule a consultation to let us handle your survey writing for you.