Most programme teams are doing a great deal. You are delivering the work, responding to the needs in front of you, and then turning all of it into a report that funders, boards and partners can understand.
When reporting time comes around, the instinct is often to add more: more indicators, more data points, more pages. Almost every team we work with at Fundamental knows this pressure.
But an evidence-based report is not defined by how much data it contains. What matters is whether the evidence supports what you are saying has changed, and how confidently you can make that claim.
Start with the claim
Take a simple example. Twenty people attended a programme.
That tells you that 20 people took part. It does not tell you whether they learned something, used what they learned, changed their behaviour or saw an improvement in their circumstances. You need other evidence to make those claims.
This distinction can easily get lost in a report. Delivery went well, attendance was high and feedback was positive. It can then become easy to move from saying “we delivered the programme” to “the programme created change.”
There may well have been change. The question is whether the evidence allows you to say so.
The same applies when reporting findings such as:
80% of participants reported increased confidence.
That may be an important result, but the percentage alone is not enough to understand it. Eighty per cent of how many people? Was everyone asked, or only those who completed the programme? When was confidence measured? Do we know where participants started?
You also need to consider the quality of the evidence itself. Are there important gaps in the data, and does the information represent the people or experiences you are making a claim about?
The type of evidence matters too. Quantitative data can show scale and patterns. Qualitative data can help explain people’s experiences and why something may be happening. Programme records tell us what was delivered. A participant’s story may help explain a finding, but one person’s experience cannot tell us what happened for everyone.
Where possible, look across different sources of information. If programme records, participant feedback and interviews point in a similar direction, you have a stronger basis for interpreting the finding.
Being clear about the claim, the evidence that supports it and the context needed to interpret it helps prevent a report from saying more than the evidence allows.
Be clear about what you know and what you do not
Organisations are often under pressure to demonstrate progress, particularly when funding depends on showing that a programme is making a difference. Saying “we are not sure yet” can feel risky.
This can lead to findings being presented with more certainty than the evidence supports.
Social programmes also operate in complex environments. A youth employment programme, for example, may help participants build skills and look for work, but what happens afterwards can also be affected by transport costs, household income, local job opportunities, safety and other factors outside the programme’s control.
There will therefore be times when the evidence does not support a definite conclusion. In these cases, it may be more accurate to say:
– The evidence suggests…
– Participants reported…
– These results are consistent with the change we hoped to see, although we need more evidence to understand whether the change is sustained.
Funders also influence how comfortable organisations feel reporting in this way. If uncertainty is interpreted as weakness, grantees may feel pressure to present more definite conclusions.
Asking organisations what they still do not know can be as useful as asking what they have achieved. It allows reports to be clearer about what the evidence shows, where its limitations are and what still needs to be understood.
Before you finalise a claim, ask four questions
- What are we saying happened or changed?
- What evidence supports that statement?
- What does the reader need to know to interpret the evidence properly?
Consider factors such as sample size, response rate, timeframe and how the information was collected.
- Are we saying more than the evidence allows us to conclude?
If so, what can we say with confidence, and what remains uncertain?
These questions can help you check whether the language in your report reflects the evidence you actually have.
Reporting that supports learning
An evidence-based report does not need to show that everything worked. Its purpose is to give a fair account of what happened, what changed, what the evidence tells you and what you still need to understand.
Sometimes the findings will show progress. Sometimes they will show mixed results or raise new questions. Both are useful if they help you understand the programme better and make informed decisions about what happens next.
Continue building your evidence practice with Fundamental
If your organisation is working to improve how it collects, uses and reports evidence, this is the kind of practice we explore through Fundamental.
Seeds of Strength for Organisations supports teams to strengthen their MEL systems and reporting practices, while Seeds of Strength for Practitioners supports individuals to strengthen their applied MEL skills.
Explore Fundamental’s learning pathways at fundamentaltools.co.za/courses
Tools for Change.
If you would like to build these skills with your team, explore Fundamental’s practical learning opportunities for Monitoring, Evaluation and Learning: fundamentaltools.co.za/courses