In The Survey-Writing Moonwalk, we recommended working backward to create a survey: Creating a list of data goals that serve as a blueprint for the questions you’ll need to ask. Now that you have that list, you’re ready to start drafting survey questions.
But before you start putting pen to paper (or fingers to keyboard) consider what type of question is most appropriate to get you the data you need. There are two basic question types: closed-ended and open-ended. Each serves a purpose and, if used correctly, can get you the information you need.
Closed-Ended Questions
Closed-ended questions include a predefined list of potential responses from which participants can select at least one answer. In other words, you provide parameters for your respondents and limit what they can contribute. Common formats for closed-ended questions include select one, multiple-choice, likert scale, and ranking questions.
Closed-ended questions result in quantitative (numerical) data like counts and percentages that are easily converted to tables, charts, and graphs. They are also useful when there is a finite number of options available. For example, if you know that there are only four feasible locations for the new recycling center, you would want to use a closed-ended question to garner feedback about those specific locations.
Benefits:
Providing parameters results in more useful data because your respondents understand what information you are looking for and you have given them plausible options.
These questions are often faster and easier to answer.
They are also simpler to analyze and report because data easily converts to numbers.
Limitations:
Limiting response options is not always ideal or appropriate, especially if there are more than a handful of potential responses.
You miss an opportunity for “thinking outside the box” responses.
These questions can inadvertently bias responses by requiring participants to select from a list of options that may not align with their perspectives.
You get less detail, less explanation, and less insight into why people responded the way they did.
Open-Ended Questions
Open-ended questions are free-form, with no predefined answers. They prompt respondents to answer the question in their own words, resulting in qualitative data. Qualitative data are descriptive and cannot easily be understood in terms of counts and percentages. Because the analysis of qualitative data is subjective, it is best that this work is done by a trained analyst.
Open-ended questions are useful for getting information from respondents that can’t be easily captured in a list or when the list of options would be excessively long. The data you get is often richer and more complex than what you get through closed-ended questions. For example, if you wanted to find out where people hike most often in Raleigh, you could use an open-ended question to avoid listing out all 75 trails. You could also include an open-ended question to ask respondents why they prefer certain trails to others, in their own words.
Benefits:
Open-ended questions can feel more empowering to respondents because they can use their own words and they aren’t constrained to particular responses.
You can better explore respondents’ reasons, justifications, and thought processes.
Respondents may provide innovative solutions or insights that would be missed with predefined options.
Limitations:
You are more likely to get responses that are off-topic or irrelevant.
It takes more effort for respondents to answer open-ended questions, which can translate into lower response rates.
Qualitative data are more difficult and time-consuming to analyze than quantitative data.
Conclusion
There’s no magic template when it comes to writing the right survey question. Sometimes closed-ended questions are the most appropriate; other times, you can only get the information you need with an open-ended prompt. Often, you’ll find that a strategic blend of closed- and open-ended questions offer the best blend of quantitative and qualitative data. If you focus on the data you want to collect from respondents, you’ll find the right balance between open- and closed-ended questions.