Clearing The Fog

Clearing the fog: new tools for improving the credibility of impact claims

Complex Policy Evaluation

Development actors facing pressure to provide more rigorous assessments of their impact on policy and practice need new methods to deliver them.

There is now a broad consensus that the traditional counterfactual analysis leading to the assessment of the net effect of an intervention is incapable of capturing the complexity of factors at play in any particular policy change.

We suggest that evaluations focus instead on establishing whether a clearly-defined process of change has taken place, and improve the validity and credibility of qualitative impact statements.

IIED research in Uganda shows that the methods of process tracing and Bayesian updating facilitate a dialogue between theory and evidence that allows us to assess our degree of confidence in ‘contribution claims’ in a transparent and replicable way.

Full article from CECAN's Barbara Befani

Global Challenges Require Cross-Cutting Solutions

Complexity in Policy Evaluation

New research led by the Tyndall Centre for Climate Change Research suggests that current UK policies on water, energy and food are too fragmented to effectively tackle global challenges. Issues such as climate change, resource constraints and the increasing population cut across several sectors and need similarly cross-sectoral policies. Future research must meet this challenge by focusing on the nexus between sectors, scales and timeframes.

To ensure resilience to complex global challenges, it is necessary to evaluate whether policies and underlying research are working effectively across a range of sectors. It is also important to analyse how existing policies affect the water-energy-food nexus at different scales (local, national, global) and times (short- and long-term). Our new research does exactly that, exploring how the policies could better function across the water, energy and food-related sectors of the economy.

Looking through the lens of ‘land-use’ allows us to capture some of the complex water-energy-food interactions.

Tools, tools everywhere and not a hammer in site!

Complexity in Policy Evaluation

By Pete Barbrook-Johnson (né Johnson)

Members of the Sociology department, alongside colleagues from across the University of Surrey, have been working on the ERIE project for the past six years. One of the main outputs of the project is the development of a suite of software tools designed to help anyone and everyone make decisions and think strategically. The tools have been designed with government, industry and charity sector activities in mind, but they can be applied to a very wide range of topics, from managing your health and exercise, to understanding how we might better collaborate with colleagues across the University, and even bringing together students to share and build knowledge.

Have a look, get involved…

You can read about and start using these tools at I encourage all readers to take a look, and have a quick think about how one or more of the tools could be helpful for you, whether you are a researcher, teacher, student, work in government, industry or the charity sector, or anywhere else!

CECAN - funded and supported by the following organisations