Integrity awarded excellent scores for quality evaluations

Integrity is thrilled to share the excellent scores received for recent high quality evaluations. Throughout 2022/23, Integrity has worked with the UK Government’s Foreign, Commonwealth, and Development Office (FCDO) to design and deliver evaluations of a number of the Humanitarian and Development programmes. The evaluations provide insights and new evidence on three of FCDO’s large delivery programmes, covering £130m in UK aid spend in the areas of Humanitarian Assistance, Social Protection and Climate Change. All three evaluations were awarded excellent scores by the FCDO contracted independent peer review process, Evaluation Quality Assurance and Learning Service (EQuALS). Here we share key lessons from our work.

Why evaluation quality and transparency matters:

Evaluation Quality: We welcome the opportunity to share the proposed methods with independent evaluation experts. It allows them to assess our evaluation design and delivery and means that the evaluations have gone through structured evaluation design and delivery process. This ensures high standards, independence and transparency on methods and findings. The excellent scores we received from EQuALS validate the end-to-end processes which has taken place over several months.

Evaluation Transparency: Transparency of evaluation, including dissemination of both the methods and findings are critical steps in meeting evaluation good practice. By publishing the evaluation design and evaluation report, the client (in this case the FCDO) increases the likelihood that the evaluation findings are utilised and increases the impact of the evaluation.

Our lessons on evaluation transparency include:

• Share the evaluation design and methods: Evaluation designs are not fixed and work best when tailored carefully to suit the purpose of client, the programme needs and stage of programme delivery. Sharing the evaluation design, approach and methods is as important as sharing the results. It helps to understand the findings and allows others to critique the methods used, as well as learn from and replicate them at a later point.
• Be clear on evidence strengths and weaknesses: Throughout the evaluation report, be transparent about the strength (or weakness) of the evidence supporting the findings. By being clear about what evidence a finding drew on, and what that did (or did not) means that we have ensured that we were fair and that the evidence would be used appropriately. For example, we make the triangulation process clear by documenting the strength of evidence underpinning each sub-evaluation question, including how many sources were used, how consistent these sources were, and to what extent we felt we had uncovered all views and perspectives (saturation).
• Publish to increase the reach of evaluation impact: By publishing the evaluation report, the client is enabling programme implementers and wider stakeholders to make use of the evaluation findings. This extends the reach of the evaluation’s impact, enabling it to support other technical assistance and programming. We are supportive of all efforts to make data collection and analysis methods transparent. While the direct users of a lot of evaluation outputs will often not need to know the details of how the findings have been generated, this information is still an important part of the full record.

Here is a quick summary of and links to the three recent Integrity evaluations all with excellent EQuALS scores:

Evaluation of the Humanitarian Emergency Operations & Stabilisation (HEROS) Programme – £80m

EQuALS Score – Excellent (97%)

• Programme: The HEROS programme provides capacity, advisory and operational support to the UK’s global humanitarian emergency response and stabilisation efforts (2017-25).
• Evaluation Type – Summative (2017-22): The evaluation assessed how, and how well, the HEROS programme has worked; the primary purpose was learning, though it also supported accountability.
• Evaluation Publication: https://devtracker.fcdo.gov.uk/projects/GB-1-205176/documents odt (live.com)

Evaluation of the Better Assistance in Crises (BASIC) Programme – £20.5m

EQuALS Score – Excellent (90%)

• Programme: BASIC aims to provide (1) technical assistance, (2) research, and (3) knowledge management and learning services to the FCDO and its partners to improve the use of social protection systems and approaches during crises (2018-25).
• Evaluation Type – Midline (2021-22): The evaluation seeks to provide (1) timely evidence to improve BASIC delivery, (2) evidence on how technical assistance and research services can support the use of social protection during crisis, and (3) guidance on the design of future centrally managed programmes.
• Evaluation Publication: BASIC Performance Evaluation: Midline Report | socialprotection.org
• https://iati.fcdo.gov.uk/iati_documents/D0002984.pdf

Evaluation of the Climate Public Private Partnership (CP3) Programme – Phase II – £130m (FCDO and BEIS)

EQuALS Score: Excellent (84%) 

• Programme: CP3 programme aims to increase low carbon investment in renewable energy, water, energy efficiency and forestry in developing countries by demonstrating the commercial viability of these sectors.
• Evaluation Type – Mid point (2018-22) A strategic evaluation at the mid-point of the second phase of the programme seeks to gather evidence to test the plausibility of the programme and its assumptions. The evaluation was conducted by the CP3 MEL team, a consortium led by Integrity in partnership with NIRAS and the Climate Policy Initiative Global.
• Evaluation Publication: DevTracker Programme GB-GOV-13-ICF-0010-CP3 Documents (fcdo.gov.uk) Awaiting upload.