Blog

Policy evaluation is about assessing the value of policies, but too little attention, it seems to me, is paid to the meaning of “value” in all this.  The English word “value”, of course, has multiple meanings that include numerical (e.g. “a parameter value”), economic (e.g. “good value”) and ethical (e.g. “a value judgment”). 

On the 23rd and 24th November 2017 members of the CECAN team, Pete Barbrook-Johnson, Clare Twigger-Ross and myself attended the EEEN conference in the beautiful surroundings of the Royal Society in Edinburgh. Facilitated by the Scottish Environmental Protection Agency (SEPA), civil servants from environment agencies, consultants, practitioners and academics came together to share experiences and ideas for the future of environmental policy evaluation.

As we all know CECAN is pioneering, testing and promoting innovative policy evaluation approaches and methods across Nexus domains through a series of “real-life” case study projects – it’s about measuring best value in complex, inter-connecting systems.

I first heard of Qualitative Comparative Analysis from Prof David Byrne at an early CECAN meeting and it seemed to address key issues we faced when evaluating 13 very different interventions across England designed to improve community resilience to flooding.

As the first step in developing my CECAN research project titled ‘The Visual Representation of Complex Systems: A Typology of Visual Codes for Systemic Relations’ I was thrilled to have the opportunity to engage with the Relating Systems Thinking and Design community last month. 

It’s been a month since we launched ‘A Whole New World: Funding and Commissioning in Complexity’ and the response has been incredible. Over 120 people came to the two launch events, and more than 1000 people read the report online in the first week. We’ve also started to have some excellent conversations about how to take this work forward.

It would be naive and, potentially, ill-advised to have evaluation solely drive policy direction. Good, open, evidence-backed policy, however, does need to be informed by evaluation results and insights.  My CECAN Fellowship provided me with a rare opportunity to investigate how evaluation is applied in real life within a government department – in this case Defra – and how it can be used to plan for future policymaking. 

Pete Barbrook-Johnson, Research Fellow at the Policy Studies Institute at the University of Westminster & Knowledge Integrator Research Fellow at the Centre for the Evaluation of Complexity Across the Nexus (CECAN) at the University of Surrey. 

It is over two years since Ofwat set increasing trust in water as its key objective for its 2015 five year business plan.To establish what has happened since, I interviewed 19 senior and influential figures across all parts of the water sector. I am seeking wider perspectives via a survey (click HERE to participate) and hope this article will stimulate you to take part.

A couple of days ago, a DEFRA policy official told me that the uncertainty over EU exit was creating a fertile environment for evaluation, as champions try to ensure their favoured policies have a place in the forthcoming landscape, post Brexit. This struck me as interesting, and I made a note of it.

Sustainable Development and its relative Sustainability, concepts which have a rich history of appeal and animosity, have nevertheless become the dominant conversation framing environment-development policy in recent decades.

To say that we live in a complex world is, in a very general sense, rather banal and uninteresting being neither particularly illuminating nor especially profound. But, scratch beneath the surface, and an acknowledgement of that complexity can be revelatory.

Network analysis is the method of the future. That is not only – certainly not primarily – because we are ever more connected in some superficial social-media driven internet sort of way. All of that may be fascinating (and certainly can be analysed using network analysis), but it is not fundamental to our existence as humans – we existed before Facebook, we will exist after it is gone

The first of two blogs following this event, from the perspective of the lead facilitator, Dr Paul Brand

I have only recently joined the small economic research consultancy Simetrica. Before this I spent 16 years in the Government Economic Service, starting as an economic advisor in DTI in 2000 (now known as BEIS). I first worked on employment policy and one of my main tasks was to produce Impact Assessments for new employment regulation using the tools of Cost Benefit Analysis (CBA).

CECAN is exploring how evaluation of policy can better inform the impact those policies have and assess the extent to which these have been successful. In order to do this, access to data is crucial, yet can at times be problematic. CECAN’s Knowledge Integrator, Candice Howarth met Emma Uprichard and Robert MacKay from the Centre and based at the University of Warwick and asked them over a series of emails to explain what the implications of some of these challenges are.

As the world changes in complex and unpredictable ways, Government is changing too.  As it does so, the need grows for policy-making and the evidence that informs it to be alive and responsive to the increasing pervasiveness of complexity.  In public service systems the increase in complexity often means that  no single institution is ever ‘in charge’ or has direct control over how changes unfold. 

Maastricht was the location of this year’s European Evaluation Society (EES) conference over a sunny week in late September. At the end of first day, we were treated to a civic reception in the building in which the Maastricht treaty was signed in 1992, bringing up mixed emotions for some of us.

The scope of the CECAN project runs wide as well as deep; complexity in the energy, water, environment and food domain would most immediately be thought to arise from the physical systems at the nexus core. Yes, complexity in weather systems, biological populations spring quickly and easily to mind as do the ‘user level’ human interactions with these systems and other local and global physical systems.

By Roger Highfield, Director of External Affairs of the Science Museum, member of the Royal Society’s Science Policy Advisory Group. To tackle climate change, ecosystem destruction and the many daunting issues facing humanity we need not only to draw on science and engineering but also develop policies that can change the behaviour of 7.5 billion people.

This post is by Andy Jordan, Charlotte Burns and Viviane Gravey.  They recently co-led an expert reviewof the environmental implications of Brexit funded by the UK in a Changing Europe Initiative.

What can academics learn from how civil society organisations and NGOs approach policy impact? Julia Himmrich argues that academics have a lot to gain from embracing the practices of long-term advocacy. Advocacy is about establishing relationships and creating a community of experts both in and outside of government who can give informed input on policies.

When I turned up at the CECAN Evaluation and Complexity workshop this week it was my first day back at work after a holiday in Crete, an experience that I thought might have put me in the right frame of mind.   Being the non-scientist in a roomful of scientists often seems like being in a foreign country where I only speak a few words of the language.

UK social scientists feel a growing pressure to achieve policy change. In reality, this process is more complex than it sounds. James Lloyd looks at six reasons that limit the impact research can have on policy change. None of this should suggest that academic researchers shouldn’t seek to influence policymaking. But more consideration is needed on how best academic evidence can leverage the real-world nature of policymaking.

Governments all over the world invest large sums of public money into producing knowledge that helps them understand their countries’ complex socioeconomic issues. This knowledge, in the form of research, can be used to formulate potential solutions through public policies and programmes.

Development actors facing pressure to provide more rigorous assessments of their impact on policy and practice need new methods to deliver them. There is now a broad consensus that the traditional counterfactual analysis leading to the assessment of the net effect of an intervention is incapable of capturing the complexity of factors at play in any particular policy change.

New research led by the Tyndall Centre for Climate Change Research suggests that current UK policies on water, energy and food are too fragmented to effectively tackle global challenges. Issues such as climate change, resource constraints and the increasing population cut across several sectors and need similarly cross-sectoral policies. Future research must meet this challenge by focusing on the nexus between sectors, scales and timeframes.

Members of the Sociology department, alongside colleagues from across the University of Surrey, have been working on the ERIE project for the past six years. One of the main outputs of the project is the development of a suite of software tools designed to help anyone and everyone make decisions and think strategically.