Integrity Insights: How many evaluators does it take to turn on a lightbulb?

Adrian Bannister, Senior Learning Expert

During a recent session at the 2022 European Evaluation Society (EES) Conference in Copenhagen, Integrity Senior Learning Expert Adrian Bannister asked delegates to consider the challenges of sharing Monitoring, Evaluation and Learning (MEL) knowledge and supporting evidence use in the international development sector.

Using the metaphor of a switched-on lightbulb to represent a functioning knowledge system that connects producers (including MEL teams) and consumers (such as programme managers and policymakers), Adrian posed the question ‘How many evaluators does it take to turn on a light bulb?’ The discussion helped participants to consider how knowledge supports decision making and how evaluators can draw on the principles of knowledge brokering to ensure that MEL knowledge is fully utilised.

Seeing the light and feeling the heat on MEL knowledge

Let’s think of a lightbulb as representing the international development sector’s knowledge system. It is imperative that the lightbulb is always switched on and powered by a reliable and sustainable power source, as the achievement of global development objectives depends on the timely flow of useful knowledge to decision makers. But what if, instead of burning brightly, the lightbulb only flickers intermittently or, in some circumstances, is not illuminated at all?

MEL knowledge should play a significant role in strengthening decision making in the sector. When asked their views, however, nearly all of the 30 EES session participants (from independent consultants to senior evaluation specialists in donor agencies and INGOs) felt that MEL knowledge is often not being used effectively in practice.

So, what does it say if MEL specialists themselves are concerned? And, by implication, what can we do differently to turn the lightbulb on and keep it shining brightly? Bringing credible and rigorous evidence to decision makers is not sufficient on its own to achieve this. Rather, for MEL knowledge to be fully utilised it needs to be ‘brokered’.

What exactly does brokering MEL knowledge mean?

There are two kinds of professionals in our sector: those who perhaps unwittingly play a knowledge brokering role; and those who intentionally and actively embed knowledge brokering into their practice. Though a gross simplification, there is some truth in this statement.

In a complex development sector in which many actors combine and collaborate to achieve success, we are forever pulling and pushing knowledge and ideas between us. While some development professionals identify as knowledge brokers, others could draw benefits from expanding skill sets, mindsets and behaviours by borrowing from knowledge brokering theory and practice.

Knowledge brokers are broadly defined as pivotal actors in networks where ideas and evidence are transferred. Knowledge brokering serves to bridge the gap between knowledge producers and users (including decision makers), who often inhabit separate worlds. The diverse cultures, drivers, values and practices in these separate spaces can prevent productive communication, meaning a facilitator is required. This is where knowledge brokers can help.

According to Karol Olejniczak et al, knowledge brokers can play multiple roles, ‘spanning from political coalition builder to generator of innovations, and from disseminator of knowledge to capacity-builder for evidence-informed decision making.’ They highlight that, in practical terms, knowledge brokers commonly undertake a range of activities, including:

• identifying and targeting key actors
• assisting knowledge producers
• translating and adapting evidence
• reaching out and networking across policy influence spaces
• negotiating around behaviour change
• building capacity to use – and demand for – knowledge.

MEL specialists already perform some of these activities. For example, to help time-poor decision makers engage with our findings we are increasingly delivering less weighty and easier-to-digest products (such as briefing summaries and audio-visual content), in addition to more traditional long-form MEL outputs. That said, although better packaging is important, it does not in itself ensure utilisation.

Strengthening relationships to support use of MEL knowledge

While the mandate MEL specialists are given does not always create space to undertake broader knowledge brokering functions, our own sense of identity and purpose can also be a constraint. We should therefore look for opportunities to better understand knowledge brokers, collaborate directly or work closely alongside them, and borrow more from knowledge brokering in our own practice. We should also seek out reflective spaces, such as the EES conference, in which to share experiences and strengthen relationships with knowledge brokers.

Conversations at the 2022 EES conference highlighted three avenues worth exploring further:

• How the expertise of knowledge brokers can help MEL specialists navigate opaque decision-making processes: Drawing on their understanding of the psychological and institutional factors that underpin what informs and influences decisions, knowledge brokers can help MEL specialists identify and package knowledge during windows of opportunity to respond to the needs of decision makers.
• Why knowledge brokering interventions can enhance belief in and thirst for MEL knowledge: By curating bodies of MEL knowledge, stimulating demand for evidence and strengthening information appraisal skills, knowledge brokers can help decision makers to become ‘better customers’ who are more curious and can sustain deeper engagement with MEL.
• When knowledge brokering approaches can help MEL specialists to be seen as critical friends: Adopting a strong explicit or implicit user focus in MEL design and delivery is key in building relationships with programme teams and helping them feel the benefits of engaging with processes to create and use MEL knowledge.

Thanks to Graeme Ramshaw (Westminster Foundation for Democracy), Louise Clark (Institute of Development Studies) and Olivia Geymond (Tony Blair Institute) for co-facilitating breakout discussion groups during this EES session in Copenhagen.