A closed-ended question includes a predefined list of options from which participants can select, thus restricting potential responses. Closed-ended questions are useful when there is a finite number of possibilities or you want numerical data that is quick and easy to analyze, like counts and percentages.
Deciding to use a closed-ended question is only the first step. Different varieties of closed-ended questions ask for different responses and provide different data insights.
In this post, we break down the two closed-ended question types most commonly used in community engagement surveys. Understanding the distinction between the two is critical for ensuring you get the data you need.
When you ask respondents to select from a predefined list of options, you are asking a selection question. You can ask participants to select only one (single select or multiple choice) or as many options as they need to fully answer the question (multiple select). However you present the options - using checkboxes, fillable bubbles, or a drop-down menu - selection questions provide data that is easy to quantify and summarize, like the number or percentage of people who chose each option.
Let’s say that we want to find out what aspects of residential development our community is most concerned about. To get feedback, we publish a survey that includes a selection question that asks respondents to select all of the potential effects of development that concern them from a predefined list.
The hypothetical results look like this:
What do we learn from this question? As you can see in the graph above, more respondents were concerned about their neighborhoods becoming less affordable (70%) than they were about any other potential consequences. Fewer respondents (60%) expressed concerns about the demolition of existing viable homes as a result of development.
So this must mean that the primary concern among respondents is affordability, right? Not necessarily. Because the survey asked residents to select all options that concerned them, the data simply reflects that 70% of respondents selected neighborhood affordability as one of their concerns. If you want to draw conclusions about how these concerns relate to one another, you need to ask a ranking question.
Ranking questions ask respondents to express preferences or concerns in relation to one another by sorting the list into a particular order (e.g., highest to lowest). There are a couple of different ways to ask participants to rank options. The most straightforward tactic is to instruct respondents to order the items on a list.
Let’s rephrase the question from above so that we are asking respondents to rank their concerns from highest to lowest, excluding anything on the list that is not a concern at all. Now the hypothetical results look like this:
What do we learn from this question? On average, respondents ranked the demolition of existing viable homes as the most concerning consequence associated with development (the closer the rank score to 1.00, the higher the rank). The loss of neighborhood affordability was the second-highest concern, followed by the loss of green space and tree canopy.
So this means that more people are worried about existing viable homes being demolished, right? Again, not necessarily.
Looking at the results of both survey questions together can provide clarity:
Although a larger number of respondents said that neighborhood affordability was a concern (70%), they ranked it as less of a concern (3.17), on average, than the demolition of existing homes (2.63). Looking at results from both types of questions allows us to identify the concern that the largest number of respondents selected AND what respondents ranked as their highest concern.
The conclusions we can draw will depend on what question type we use to gather the data. The selection question allows us to determine which concerns are the most widespread, while the ranking question provides insight into what concerns are the most pressing for respondents.
Don’t fall into the trap of believing that closed-ended questions are all basically the same and differ only in how they are presented. Because the insights you can get from a survey depend on the type of closed-ended question you ask, you will want to be strategic when drafting your survey to ensure you get the information you need. You will also want to be careful when analyzing the data so that you draw the appropriate conclusions and accurately represent the findings.
Schedule a free 15-minute consultation call to learn more about how NineteenEleven can streamline your survey-writing process and help you get the data you need.