Sometimes the evidence tells a tidy story. Participation rises, people describe a good experience and the outcomes move in the hoped for direction.
Sometimes it does not.
Attendance may look steady while staff are noticing more complex needs. Interviews may be warm and positive while a formal measure barely shifts. A few vivid stories may suggest transformation, while administrative records show that many people leave early.
This is not necessarily a problem to tidy away. It may be the point where the evaluation becomes most useful.
When evidence points in different directions, the task is not to choose the most flattering signal. It is to ask what each source can see, what it cannot see and why the difference matters.
Good evidence is wider than one measure
The Department of the Prime Minister and Cabinet says good evidence draws on current data, contextual and other knowledge, people’s experiences and research. That mix matters because no single source can carry the whole explanation.
Numbers can show pattern and scale. Conversations can reveal meaning. Staff observations can notice practical shifts that a survey never asks about. Community knowledge can explain why the same service lands differently in different places.
Each source brings a viewpoint. None arrives without limits.
An illustrative example
Imagine a youth programme that records strong attendance and receives enthusiastic comments at the end of each session. This example is hypothetical. A longer term follow up, however, shows little movement in the outcome the programme hoped to influence.
One response would be to decide that the positive comments are merely anecdotal. Another would be to dismiss the longer term measure because it feels too blunt.
A more curious response asks what each signal is actually describing.
Perhaps the sessions are welcoming and valued, but the intended outcome takes longer to appear. Perhaps the programme is reaching young people who face barriers that the measure does not capture. Perhaps attendance reflects the appeal of the space rather than the effect of the activity. Perhaps the outcome goal needs to be reconsidered.
The disagreement does not settle the answer. It helps locate the next question.
Do not average away the tension
It can be tempting to compress several sources into one overall judgement. That may be useful for a headline, but it can hide the most important learning.
If one group reports a strong experience and another does not, the average may look acceptable while access or relevance remains uneven. If attendance improves but outcomes do not, the programme may be good at engagement and still need to examine what happens after people arrive.
The gap between sources can point to timing, definitions, missing voices, different expectations or a change in context. It can also expose an assumption that has gone untested.
Staff insight is evidence, but not the final word
People delivering a service often hold detailed knowledge. They see small changes, repeated barriers and moments that never reach a form. Their insight deserves a place in the evidence story.
It also benefits from being set alongside other views. Staff may see progress where participants feel pressure. Participants may value something that was not part of the original programme logic. Records may show a pattern that no individual person can observe.
The aim is not to rank these perspectives automatically. It is to understand their relationship.
A responsible conclusion can remain nuanced
Evaluation does not always end with a clean verdict. A responsible conclusion might say that participants value the service, that access appears uneven, that early change is visible in some areas and that more time or a different measure is needed to understand longer term outcomes.
That kind of conclusion can feel less decisive than a single score. It is often more honest and more useful for the next decision.
Questions worth sitting with
Which source is closest to the experience you want to understand?
Whose perspective is missing from the apparent disagreement?
Are the sources looking at the same time period and the same definition of success?
What might be true at the same time, even when the findings appear to conflict?
What decision becomes possible when the tension is described clearly rather than smoothed over?
Evidence that disagrees may not be asking for a winner. It may be asking for a better explanation.
HumanScope Research supports community organisations to make sense of mixed evidence and communicate nuanced findings with care.
Register for research and outcomes support
Sources and further reading
Department of the Prime Minister and Cabinet: Evidence and evaluation
Department of the Prime Minister and Cabinet: Making sense of evaluation