RESEARCH & COMMUNITY VOICES
You asked your community what they thought. The responses arrived, the charts were prepared and a few clear themes emerged.
It is easy to feel that you now have the picture.
But who did you actually hear from?
That question matters because the people who answer a survey may not share the same experiences as the people who never see it, cannot complete it or choose to leave it unanswered.
An unanswered survey does not explain itself
Imagine someone opening a survey on their phone between work and collecting a child. They intend to return to it, but do not. Someone else reads the first question and is unsure what it means. Another person wonders whether an honest answer could affect the support they receive.
These are possible scenarios, not findings about a particular community.
There are many reasons someone might not participate. Silence does not tell us which one applies, and it certainly does not tell us what that person thinks.
A bigger response count is not the whole answer
A response rate alone cannot tell us whether survey findings are biased. What matters is whether the people heard from differ from those not heard from in ways relevant to the question.
Stats NZ’s Guide to good survey design explains that missing responses can bias results when the people who do not answer differ from those who do. It also recognises that the effort involved, difficult questions and concerns about how information will be used can affect participation. These are useful considerations for community organisations in Aotearoa. They do not tell us why any particular person left our survey unanswered.
Read Stats NZ’s Guide to good survey design, sixth edition
For a community organisation, the concern is practical. Could the voices missing from the survey change the decision you are about to make?
Be careful with the word ‘community’
Consider an illustrative survey circulated through an organisation’s email list. The responses may offer valuable insight into the experiences of those subscribers.
They may say much less about people who have never joined the list or have stopped using the service.
Writing ‘among those who responded’ can be more accurate than writing ‘the community told us’. That small distinction helps readers understand whose experience is being described.
It does not diminish the value of the responses. It gives them an honest context.
Listening includes recognising the gaps
We do not need to guess what absent people would have said. We need to recognise where our knowledge ends.
Research becomes more useful when an organisation can discuss both what it has learned and what remains uncertain.
Before the next decision, it is worth pausing over one question: whose experience might we be missing?
HumanScope Research supports community and stakeholder research, surveys and evaluation, helping organisations interpret what they hear with care.
Explore research and evaluation support
HumanScope Research. Understanding people. Strengthening impact.