Contact:
Frances Rowe and Adam Hejnowicz
DEFRA and the Rural Development Programme for England: Evaluating complex policy at a time of dynamic change.
From 2016 to 2018 DEFRA worked with the CECAN case study team, led by Newcastle and York universities, to help inform and improve their approach to evaluating the Rural Development Programme for England (RDPE). Sitting under Pillar 2 of the Common Agricultural Policy (CAP) the EU’s rural development policy (RDR) is a £100 billion programme to help rural areas address “the wide range of economic, environmental and social challenges of the 21st century” (European Commission). The complexity of the policy presented particular challenges for evaluation, which were compounded by the UK’s decision to leave the European Union in 2019.
As a complex policy involving numerous stakeholders and beneficiaries, with multiple variables and measures across different scales, the RDPE presents a number of challenges for policy evaluation including:
- Identification of adequate counterfactuals: what would have happened without the policy?
- Establishing attribution and causality, given range of programme interventions and system variables.
- Capturing and measuring socio-economic and environmental interactions.
- Measuring impact against and broader range of market and non-market outcomes.
- Programme level versus scheme level evaluation.
- Designing and evaluating new policy to replace EU rural development policy (RDR) after the UK’s exit from the EU in 2019.
What activities were employed?
CECAN worked with DEFRA through an iterative process of co-production to design and implement activities through two broad work streams. These aimed to help DEFRA to:
- Identify the evaluation challenges inherent in the complexity of the RDPE.
- Expand policy analysts’ knowledge and awareness of complex systems thinking and approaches, and apply it to the current evaluation of RDPE.
- Embed and incorporate complexity at the early stages of designing new policy.
Activities included:
- An interactive one day workshop that introduced policy analysts to a novel Evaluation Methods Toolkit. Stakeholders included DEFRA and its next steps agencies (eg Natural England and the Forestry Commission) plus representatives from the devolved administrations.
- Exploring how the toolkit could be used to critically appraise complex evaluation questions and identify the most suitable and complexity-appropriate evaluation methods.
- Designing and facilitating a series of interactive workshops with DEFRA to co-produce and map the future agriculture, environment and rural ‘system’ (Animal Plant Health and Welfare; Environmental Land Management; Transition and Productivity, Risk and Resilience; and Rural Economies and Communities)