Impact Assessment, Evaluation...
Impact Assessment, Evaluation and Impact Evaluation: Understanding the Differences
Illustrative Example: Project Unnati
1. One Programme, Different Questions
Consider Project Unnati, an illustrative financial inclusion programme for adolescent girls in rural communities through digital financial literacy training, which has ended.
The organisation wants to understand what the programme achieved.
At first, the question seems simple:
Did Project Unnati work?
But different people may mean different things when they ask that question.
The programme team may want to know:
Was the programme implemented as planned, and did it achieve its intended results?
The organisation’s leadership may want to know:
What changed for the girls, how significant were those changes, and are the benefits likely to continue?
A funder may want to know:
Did the programme cause the changes we observed?
All three questions are about the same programme. Yet they require different kinds of evidence and may call for different approaches.
The starting point should be the question the organisation needs to answer. The evidence and methodology should then be chosen to answer that question, and the conclusions should reflect what the evidence can reasonably establish.
2. The Question Determines the Approach
The same programme can be examined in different ways depending on what the organisation needs to understand.
For Project Unnati, the organisation may want to know whether the programme was implemented as planned, whether participants achieved the intended outcomes, what changed for them, or whether the programme caused those changes.
For example, knowing that 65% of participants now use digital payments independently tells us about their current behaviour. If we want to understand how much their behaviour changed, we need a basis for assessing change. If we want to understand what influenced that change, we need additional evidence about participants’ experiences and circumstances. If we want to establish whether the programme caused the change, we need evidence that can support a causal conclusion.
The starting point, therefore, should not be the label given to the study. It should be:
What does the organisation need to know, and what decision will the evidence support?
From there, the appropriate evidence and methodology can be determined.
Question → Evidence → Method → Conclusion
3. Where Do Monitoring and Evaluation Fit?
Before looking at impact assessment and impact evaluation, it helps to briefly clarify where monitoring and evaluation fit.
The two terms are often used together as M&E, but they serve different purposes.
Monitoring is the ongoing process of tracking programme implementation. It helps an organisation understand whether planned activities are taking place and whether the programme is progressing as expected.
For Project Unnati, monitoring might look at:
- How many girls enrolled?
- How many completed the digital financial literacy training?
- How many sessions were delivered?
- How regularly did participants attend?
These measures help answer:
Are we doing what we planned to do?
Programme evaluation goes further. It examines the programme more systematically to understand its design, implementation and results. It may consider whether the programme was relevant to participants’ needs, whether it was implemented effectively, whether it achieved its intended outcomes, and what factors influenced its results. The ILO describes evaluation in terms of assessing programme design, implementation and results.
For Project Unnati, an evaluation might ask:
- Was the programme relevant to the girls’ needs?
- Was it implemented as intended?
- Did participants achieve the intended outcomes?
- What worked well and what could be improved?
In simple terms:
Monitoring tells us what is happening. Evaluation helps us understand how well the programme is performing and what results it is achieving.
An evaluation may use different types of evidence, including baseline and endline data, programme records, surveys, interviews and other sources, depending on the evaluation questions and design.
When the assessment focuses on broader changes and effects associated with an intervention, including what changed, for whom, to what extent, and what may have contributed to those changes, impact assessment can provide a useful approach.
4. What Is Impact Assessment?
The term “impact assessment” is used in different ways across organisations and fields. In this article, we use it to refer to a structured assessment of the broader changes and effects associated with an intervention, including what changed, for whom, to what extent, what may have contributed to those changes, and what factors may influence them. This is consistent with the OECD-DAC understanding of impact as significant positive or negative, intended or unintended, higher-level effects.
The programme’s Theory of Change can help frame this assessment by showing how the intervention is expected to contribute to the intended outcomes and longer-term changes. There are different impact assessment frameworks that can be used to structure an assessment. One useful framework for structuring questions about impact is the Five Dimensions of Impact, now hosted by Impact Frontiers and developed through the Impact Management Project (IMP),which uses five dimensions to examine impact:
- What: What outcome or change occurred?
- Who: Who experienced the change?
- How much: What was the scale, depth and duration of the change?
- Contribution: What contribution may the programme have made to the outcome?
- Risk: What could affect the likelihood, sustainability or nature of the impact?
For Project Unnati, this could mean asking:
- What changed in the girls’ use of digital financial services?
- Who experienced the change?
- How much did their behaviour or financial capability change, and what was the duration of that change?
- What contribution may the programme have made?
- What risks could affect the outcome?
Importantly, impact assessment does not automatically mean proving that the programme caused the change. It can examine the programme’s contribution to observed outcomes using multiple sources of evidence. The strength of that conclusion depends on the assessment design and the evidence available.
This distinction becomes important when we move from asking:
What changed and what contributed to it?
to asking:
Did the programme cause that change?
That is where impact evaluation becomes relevant.
5. What Is Impact Evaluation?
Impact evaluation is a form of evaluation designed to examine the causal effects of an intervention. It seeks to determine whether, and to what extent, observed changes can be attributed to the intervention rather than to other factors.
This is different from simply showing that outcomes changed or examining what may have contributed to those changes. The focus is on causal attribution.
For example, suppose Project Unnati finds that independent use of digital payments increased from 35% at baseline to 65% at endline.
The baseline-endline comparison shows that an observed change occurred. An impact assessment can then examine the nature and extent of that change, who experienced it, what may have contributed to it, and what risks may affect it.
If the organisation’s question is whether Project Unnati caused the observed improvement, the evaluation needs to address a counterfactual:
What would have happened to these girls if they had not participated in Project Unnati?
Impact evaluation uses a design that can support causal inference. Depending on the evaluation question and programme context, this may involve an experimental or quasi-experimental design that provides a credible basis for assessing whether the programme caused the observed change. The ILO identifies determining whether observed changes can be attributed to an intervention as a defining feature of impact evaluation.
This does not mean that impact evaluation is automatically a better version of impact assessment. It is appropriate when causal attribution is central to the question the organisation needs to answer.
In short, impact assessment examines broader changes and effects associated with an intervention, while impact evaluation focuses more specifically on causal effects.
6. Is an Endline the Same as an Impact Assessment?
This is where the distinction becomes particularly important in practice.
Many programmes follow a simple sequence:
Baseline → Programme implementation → Endline
- What changed?
- Who experienced the change?
- How much did things change?
- What may have contributed to the change?
- What risks could affect the outcome?
A baseline and endline can therefore be important sources of evidence for an impact assessment, but they are only part of the overall assessment.
For example:
Baseline: 35% of girls independently use digital payments.
Endline: 65% independently use digital payments.
This tells us that there was an observed change of 30 percentage points.
It does not, by itself, tell us why the change occurred or how strongly it can be linked to Project Unnati. That depends on the questions being asked, the evidence available and the assessment design.
So:
An endline measures outcomes at a point in time. An impact assessment uses evidence to examine broader changes and effects associated with the intervention.
And when the question becomes whether the programme caused the observed change, we move into impact evaluation.
Project Unnati : An illustrative example
7. The Evidence Should Match the Question
Once the question is clear, the evidence should be designed to answer it.
Different questions may require different sources of evidence:
| What we want to understand | Evidence that may help |
| Was the programme implemented as planned? | Programme records, monitoring data |
| Did outcomes change? | Baseline and endline data, outcome measures |
| What changed and for whom? | Quantitative and qualitative evidence |
| What may have contributed to the change? | Participant perspectives, programme records, contextual evidence and comparison across multiple sources and methods |
| Did the programme cause the change? | A credible counterfactual or other causal evaluation design |
No single data source necessarily answers every question.
For example, quantitative data may show that digital financial capability increased, while interviews may help explain how participants experienced that change. Programme records can establish what participants received, while comparison evidence may be needed to examine causal attribution.
This is why triangulation matters. Bringing together evidence from different sources can provide a stronger basis for interpreting findings and identifying where evidence agrees, complements or challenges other evidence.
The principle is simple:
The question determines the evidence. The evidence determines what we can reasonably conclude.
8. Which Approach Should an Organisation Use?
| If you need to know… | Possible approach… |
| Was the programme implemented as planned? | Monitoring / Evaluation |
| Did the programme achieve its intended results? | Evaluation |
| What broader changes were associated with the intervention? | Impact Assessment |
| What may have contributed to the observed changes? | Impact Assessment |
| What risks may affect the outcomes or longer-term effects? | Impact Assessment |
| Did the programme cause the observed change? | Impact Evaluation |
An organisation does not automatically need the most complex approach. If the purpose is to understand broader changes, what may have contributed to them, and factors that may affect the results, an impact assessment may provide useful evidence. If the decision depends on establishing whether the programme caused the observed change, an impact evaluation may be appropriate.
The key question is therefore not:
Which approach is more rigorous?
It is:
What evidence does this decision require?
9. Conclusion
The appropriate approach depends on the question the organisation needs to answer. Evaluation can examine programme performance and impact, while an impact assessment can help examine broader changes and effects associated with an intervention. When the central question is whether the intervention caused the observed change, an impact evaluation may be appropriate.
A baseline and endline can provide important evidence for an evaluation or impact assessment, but they do not automatically establish causal impact.
The starting point should therefore always be the question:
What do we need to know?
From there:
Question → Evidence → Method → Conclusion
The goal is not to use the most complex methodology. It is to use an approach that provides credible evidence for the decision the organisation needs to make and to be clear about what that evidence can and cannot establish.
Need Help Choosing the Right Approach?
Choosing the right approach starts with understanding what you need to know and what decision the evidence needs to support.
MIRA Impact Consulting works with NGOs, CSR foundations and social impact organisations on impact assessment, evaluation, MEL and programme data. If you are planning an assessment or evaluation and would like to discuss the appropriate approach, get in touch with us.