The old objection to open questions: nobody can read five hundred answers
The reason surveys went closed-ended was never that tick-boxes were better. It was that somebody has to read the free text, and at scale nobody does. That is the objection worth meeting head on.
Why it is hard to measure
- A closed question counts itself. It is also easy to get wrong, because the categories were fixed before anyone answered.
- Hand-coding free text is the traditional answer. It is slow, expensive, and two coders reading the same answer often disagree.
- The categories matter more than the totals. If the coding frame is wrong, a tidy chart is confidently wrong.
- Any researcher worth trusting wants to see which answer landed in which category before believing the counts.
Questions only a conversation can ask
“In your own words, what is the single biggest thing standing in your way right now?”
“If you could change one thing about how this works, what would it be, and what would that fix for you?”
Where SamSurveyor fits
Free-text answers are classified into categories drawn from the answers themselves, not from a list fixed before anyone replied. Every category can be opened and read back against the responses inside it, so the categories are checkable rather than a black box. Ordering them by frequency shows which few issues account for most of what people said. The same categories can then be read across defined groups, so open answers feed a genuine A/B comparison between population segments.
Hear this discussed
America's Survey Radio runs live. Tune in, then ask a question and Alex and Sam will take it on air.
Listen to Survey Radio