CPD Courses

The Centre for the Evaluation of Complexity Across the Nexus (CECAN) has produced supplementary guidance for the 2020 revision of the Magenta Book. The Magenta Book, published by HM Treasury, is the official government guide about policy evaluation.

Like the rest of the UK, Northern Ireland has experienced weak productivity growth since the 2008 Financial Crisis despite broader economic recovery. Consequently, boosting productivity has been a central goal of contemporary economic policy even to the extent that both UK and Northern Ireland industrial strategies have been described as productivity policies.

A demi-regularity, also known as a demi-reg, is a semi-predictable outcome pattern. The demi-reg concept has relevance to fundamental principles in realist evaluation (Pawson and Tilley, 1997) including the context-mechanism-outcome (CMO) configuration and middle-range theorizing.

The webinar views the challenge of creating complexity-informed evaluation by seeing it as a public management challenge. How can public management adopt a more complexity-informed approach? The session will outline an emerging complexity-informed approach to public management: the Human, Learning, Systems (HLS) approach.

Participation in defining elements or issues within a system is a critical requirement in defining that system adequately.  In many social policy spaces, hearing the voices of those who are affected, and who have a lived knowledge of the issues, is foundational. This is equally important in defining the elements of a complex system to anchor evaluation.

This webinar will present an overview of the System Effects methodology, an approach developed to understand the user experience of complex systems. System Effects draws on the methodological approaches of soft systems thinking, fuzzy cognitive mapping, and graph theoretical analysis.

Public policy questions are often complex. Decisions rely on both evidence and values. Where should these values come from: politicians, public officials, independent experts, or the public? 

This session briefly outlines the origins of theory of change, its purpose, and wide variety of uses. The main message is to show that theory of change as a tool, is instrumental in helping evaluators, intervention managers, and wider stakeholders, understand how change happens. In turn this contributes to better targeted and shaped interventions, and ultimately improved practice. 

Many policies are ‘complex’, that is, they have multiple objectives and multiple actors with possibly conflicting objectives, feedback loops, and policy outcomes that are dependent on the details of the policy history. Evaluating such policies is correspondingly difficult, because often the impact of the policy is unpredictable and the outcomes may be highly context dependent.  

The rise of data science has opened up a number of opportunities for government policy planning and complex evaluation. Although both data science and evaluation involve using data to better understand a particular issue, the benefit of using data science for policy evaluation has still to be established.

Participation of stakeholders and citizens is increasingly recognized as a helpful and, in some circumstances, crucial ingredient when managing complex, adaptive socio-ecological systems. The scale, interconnectedness and ongoing dynamics of these systems require new approaches bringing together stakeholder knowledge and problem ownership with new complexity-appropriate tools. Such approaches have much to offer both ex-post evaluation and ex-ante policy appraisal. 

The Food Standards Agency, a non-ministerial government department, is currently revising its risk analysis process as part of the preparatory work for EU Exit. The food and feed safety risk analysis process contributes to the FSA objective to ensure that the high standard of food safety and consumer protection we enjoy in the UK is maintained when the UK leaves the EU.

Helen, Dione and Martha are all part of a CECAN team that has been developing guidance for evaluation practitioners and commissioners struggling with complex interventions – and particularly those working in complex policy areas. The guidance was requested by the government team of evaluators who are revising the Magenta Book (Cross government guidance on evaluation) with the idea of having an annex on complex evaluation ready for when this is published.