Survey Says...What Exactly?

"82% of survey respondents believe there are too few types of housing available in their community."

What does that finding actually tell us? What can we reasonably conclude from it? And what happens when we need information it doesn't provide?

Imagine a city updating its comprehensive plan. Housing availability has become a recurring concern, and city staff are considering whether to recommend changes that would allow additional housing types in residential neighborhoods. They survey residents and find:

Survey question:

Which of the following best describes your opinion about the variety of housing types available in your community?

Response	Respondents	Percentage
There are too few types of housing available.	1,230	82%
There is about the right variety of housing types available.	180	12%
There are too many types of housing available.	45	3%
Not sure.	45	3%
Total	1,500	100%

That's a pretty clear finding. But what can we conclude from it?

Does it mean the city should prioritize expanding housing options right now? That residents would support allowing more types of housing in their neighborhoods? Or that they would support any policy intended to increase housing options?

Maybe. But not necessarily.

What we know is that most respondents believe their community lacks sufficient housing variety.

That may be enough information for the city to begin exploring additional housing policies.

But suppose the survey asked a few more questions. What else might we learn?

1. Is housing a priority?

We know that 82% of respondents believe their community has too few housing types. But does that mean expanding housing options should be one of the city's highest priorities?

Let's look at what happens when respondents are asked to consider housing alongside other community issues.

Survey question:

Which of the following issues should be the highest priorities for your city government over the next two years? Select up to three.

Community issue	Selected as a top priority
Maintaining roads and other public infrastructure	62%
Improving public safety	55%
Addressing traffic and transportation	50%
Expanding housing options	34%
Supporting local businesses and economic development	32%
Improving parks and recreational facilities	26%
Protecting natural areas and the environment	22%
Expanding community programs and services	16%

*Note: Respondents could select multiple options, so percentages do not total 100%.

Now compare this with our original finding. While 82% of respondents believe their community has too few housing types, only 34% selected expanding housing options as one of their top three priorities.

Does that mean the other 66% don't think housing is important?

Not necessarily. They may recognize a need for more housing options while believing other issues deserve more immediate attention.

The city now has a more complete picture. Most respondents perceive a need for more housing variety, but when asked to identify their highest priorities, more respondents selected infrastructure, public safety, and transportation.

That's information the original 82% finding couldn't provide.

2. Which housing types?

Let's say the city decides to explore expanding housing options. Does the original 82% finding mean respondents consider all types of housing appropriate for their community?

Let's look at what happens when we ask about specific housing types.

Survey question:

Which of the following housing types do you consider appropriate for your city? Select all that apply.

Housing type	Considered appropriate
Townhomes	72%
Small apartment buildings	64%
Duplexes	51%
Accessory dwelling units (ADUs)	38%

*Note: Respondents could select multiple options, so percentages do not total 100%.

As the table shows, 72% of respondents consider townhomes appropriate for their city, compared with just 38% who selected ADUs.

Does that mean residents oppose ADUs?

Not necessarily. 

Respondents who didn't select ADUs haven't explicitly said they oppose them. We only know that 38% identified ADUs as appropriate for their city.

And does the original 82% finding mean that respondents support allowing all four housing types?

Clearly, we can't draw that conclusion either.

What we've learned is that respondents distinguish among housing types. That's useful information if the city is considering which types to allow.

But there's another question worth asking: Just because respondents consider a housing type appropriate for their city, does that mean they consider it appropriate in their own neighborhoods?

3. Where should additional housing be allowed?

We've learned that respondents consider some housing types more appropriate for their city than others. But does that mean they feel the same way about those housing types in their own neighborhoods?

Let's compare what happens when we ask respondents about their own neighborhoods.

Survey question:

Which of the following housing types do you consider appropriate in your own neighborhood? Select all that apply.

Housing type	Appropriate citywide	Appropriate in own neighborhood
Townhomes	72%	44%
Small apartment buildings	64%	26%
Duplexes	51%	40%
ADUs	38%	34%

*Note: Respondents could select multiple options, so percentages do not total 100%.

Look at small apartment buildings. While 64% of respondents consider them appropriate for their city, only 26% consider them appropriate in their own neighborhoods.

Does that mean respondents are opposed to small apartment buildings in their neighborhoods?

Not necessarily. 

We haven't asked whether they support or oppose allowing them. We only know that fewer respondents selected small apartment buildings as appropriate for their own neighborhoods than for the city generally.

And notice that the differences aren't the same for every housing type. There's a 38-percentage-point difference for small apartment buildings, compared with just four points for ADUs.

These findings give the city a better understanding of how respondents distinguish between housing they consider appropriate somewhere in the community and housing they consider appropriate where they live.

But there's still something we don't know: Would respondents support allowing these housing types in their neighborhoods under specific development conditions?

4. Under what conditions?

So far, we've learned which housing types respondents consider appropriate for their city and their own neighborhoods. But does considering a housing type appropriate mean they would support allowing it under any circumstances?

Let's consider parking. (And y'all know why...)

Survey question:

If the following types of housing were proposed in your neighborhood without off-street parking for their residents, would you support or oppose allowing them?

Housing type	Support	Neither support nor oppose	Oppose	Not sure
Townhomes	27%	14%	53%	6%
Small apartment buildings	15%	11%	68%	6%
Duplexes	24%	17%	52%	7%
ADUs	22%	19%	51%	8%

*Note: 'Strongly' and 'Somewhat' categories for support and oppose are combined in this table.

Look at duplexes. Earlier, 40% of respondents considered duplexes appropriate in their own neighborhoods. But when asked whether they would support allowing duplexes without off-street parking, only 24% expressed support.

Does that mean the lack of off-street parking explains the difference?

Not necessarily. 

We asked two different questions. To understand whether parking changes respondents' support, we'd also need to know how they would respond if off-street parking were provided.

What these findings do tell us is how much support and opposition respondents express for allowing each housing type under a specific condition.

And that's something neither the original 82% finding nor the earlier questions could tell us.

What should we take away from this?

We started with a finding that 82% of respondents believe their community has too few types of housing available.

That's useful information. But look at everything we might have assumed from that finding alone.

Does it mean residents want housing to be an immediate priority? That they consider every housing type appropriate? That they want those housing types in their own neighborhoods? Or that they would support allowing them regardless of parking arrangements?

Not necessarily.

Each additional question gave us a more nuanced understanding of respondents' views. We learned that recognizing a need doesn't necessarily mean prioritizing it, that people distinguish among housing types and locations, and that support for specific proposals needs to be understood in the context of the conditions presented.

So, should the city have asked all those additional questions?

That depends on what it plans to do with the answers.

If the city only needs to determine whether residents perceive a lack of housing variety, the original finding may be sufficient. But if it's developing specific housing policies, the additional information could help shape those decisions.

And here's the important part: Asking additional questions only helps if we're prepared to consider what the answers tell us. There's little value in collecting nuanced information if we're going to ignore those distinctions when making decisions.

Before interpreting survey findings or deciding what else to ask, consider:

1. What does this finding actually tell us?

2. What conclusions can we reasonably and responsibly draw from it?

3. What additional information would help us make the decision, and how would we use it?

   
   

A survey finding can be accurate and useful without supporting every conclusion someone might draw from it.

Additional questions can give us a more complete picture. The responsibility is to make sure that picture informs the decisions we make.

[Note: All survey findings in this case study are hypothetical. They are not findings from an actual community survey.]