A Pluralistic Evaluation Framework

Richard Gunton
Richard Gunton, CECAN Fellow
I wrote at the start of this year about my work on articulating values in evaluation.  This has crystallised into a pluralistic evaluation framework, which I will shortly be presenting to policymakers from Defra and other agencies (in London on 12 June and in York on 3 July).  This workshop is intended to offer fresh ideas and a simple template to assist at all stages of the policy cycle - from appraisal and impact assessment through to full evaluation.

Value pluralism is the technical term for an ethical approach that recognises multiple kinds of values that cannot be collapsed down to a single value.

Looking Through a Glass Darkly: Public Goods and Agricultural Policy

Brexit & Environment

From Brexit & Environment, 9th March 2018

By Dr Adam Hejnowicz and Prof. Sue Hartley

This blogpost highlights some of the points raised in a policy brief, New Directions: A public goods approach to agricultural policy post-Brexit.

Following the Brexit referendum, the mantra “public monies for public goods” has been increasingly heard, especially in relation to agricultural and environmental policy. This public goods agenda represents, in part, a reaction against the well described failings of the Common Agricultural Policy (CAP), which despite greening reforms continues to oversee declines in European environmental quality.

Humility and courage: strategy and evidence in our complex world

Humility and courage

Stuart Astill, IOD Parc

Here in CECAN, a lot of time is spent considering and analysing policies, projects and interventions. We also think about methodologies, approaches and systems that we can adopt to do this analysis in the best way possible.

At the same time, we all recognise that there is a particular way of thinking that is valuable when working in the complex world – and certainly other ways of thinking that can trip you up quickly.

It is clear, if we take a sideways look at much of what CECAN does, that having the mindset of an explorer, a navigator, an illuminator, a learner and a satisfier is valuable.

Trying to be a ‘finisher’ or seeker of certainty would be inadvisable; being a ‘taker’ of ‘results’ rather than a learner would be foolish and to be a ‘maximiser’ would lead to a lifetime spent collecting information that still turned out to be incomplete. By definition.

Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes


Courtesy of Brian Castellani's 'Sociology & Complexity Science' Blog.

Phil Haynes is Professor of Public Policy and researches and teaches public policy and management, as applied to a variety of contemporary circumstances. His research focuses on the application of complex systems theory, often using applied statistical methods. His research has been funded by the ESRC and the government and voluntary sector. He has published in a wider variety of journals including Social Policy and Administration and Public Management Review.  He is author of several books including Managing Complexity in the Public Services (2015) now in its second edition.

His most recent book, which is part of our complexity in social sciences series at Routledge, is aptly titled, SOCIAL SYNTHESIS: Finding Dynamic Patterns in Complex Social Systems.

Finding The Common Ground

Finding the common ground

Along with several lead and co-authors, as part of a British Ecological Society Agricultural Ecology Group convened workshop held in December 2017, a report has been produced entitled "Finding the Common Ground", which sets out an ecological perspective on how future agricultural policy should develop as a consequence, and in relation to, Brexit.

The following blog by Steve Peel, Independent Eco-Agronomist sets out the main arguments:

Brexit is going to influence many aspects of life in the UK, few more so than farming and the environment.

Carillion may have collapsed, but public-private partnerships can be so much more...


public private partnerships

Pete Barbrook-Johnson @bapeterj

Last month, Carillion, one of the largest companies in the UK which regularly entered into contracts with government to deliver public infrastructure and services, went into liquidation. Since then, public-private partnerships (PPP), and their pantomime villain superstars - private finance initiatives (PFI) - have received an unprecedented level of criticism. The Guardian Opinion section – and my love-hate relationship with it - has gone into overdrive!

Teaching Evaluation of Complex Policy and Programmes

CECAN Syllabus

By Corinna Elsenbroich, CECAN Research Fellow

CECAN develops, tests and enhances methods to deal with complexity in policy evaluation, trying to advance research, policy and evaluation practice. To ensure that these methods will influence an ever widening audience, CECAN has now launched a syllabus for building capacity and supporting the application of complexity sensitive evaluation nationally and internationally.

Putting Values Back in Evaluation.

putting values back in evaluation

By Richard Gunton, CECAN Research Fellow

Policy evaluation is about assessing the value of policies, but too little attention, it seems to me, is paid to the meaning of “value” in all this.  The English word “value”, of course, has multiple meanings that include numerical (e.g. “a parameter value”), economic (e.g. “good value”) and ethical (e.g. “a value judgment”).  And herein lies an interesting problem, because this multiplicity of meanings makes it easy to present questions having an ethical component as if they were objective problems that might be solved in purely mathematical or economic terms.  Evaluating the National Curriculum’s mathematics programme is not at all like evaluating x-squared when x equals a half!  The ultimate question that evaluators ask of policies is: how good is this policy?  And that is primarily a question of value in the ethical sense, because goodness is a normative concept.

Building Collaborative Narratives and Developing Trust in Policy and its Evaluation.

dont steer blind


Reflections from the 6th European Environmental Evaluators Network forum: Betheney Wills, CECAN PhD Researcher.

On the 23rd and 24th November 2017 members of the CECAN team, Pete Barbrook-Johnson, Clare Twigger-Ross and myself attended the EEEN conference in the beautiful surroundings of the Royal Society in Edinburgh. Facilitated by the Scottish Environmental Protection Agency (SEPA), civil servants from environment agencies, consultants, practitioners and academics came together to share experiences and ideas for the future of environmental policy evaluation.

The conference’s theme - Evaluating Innovation in Environmental Protection and Sustainability - considered whether evaluation is evolving in line with the societal and environmental challenges policy faces. Participants were asked to keep the following questions in mind over the two days;

How do we evaluate innovation and how do we innovate in evaluation?

There was a variety of talks and subsequent discussions around topics including the circular economy, EU laws, the use of economic mechanisms, government policy, low-carbon incentives and innovation.

An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!

Evaluating Christmas

Anne Liddon, Science Communications Manager / Fran Rowe, Research Assistant / Amy Proctor, Research Associate, Centre for Rural Economy, Newcastle University

As we all know CECAN is pioneering, testing and promoting innovative policy evaluation approaches and methods across Nexus domains through a series of “real-life” case study projects - it’s about measuring best value in complex, inter-connecting systems.  And what could be more complex and “real-life” than Christmas?  We are all busy people and we want to get the best out of this annual festivity but what are the best methods for evaluation that can help us to do that?  We have been consulting with the most important actors involved (elves, reindeer, over-excited children, and, of course, Santa Claus himself) to consider the challenges and opportunities.

CECAN’s ‘The Visual Representation of Complex Systems’ at Environment, Economy, Democracy: Flourishing Together #RSD6.

Visual Representation of Complex Systems

By Dr Joanna Boehnert, CECAN Fellow

As the first step in developing my CECAN research project titled ‘The Visual Representation of Complex Systems: A Typology of Visual Codes for Systemic Relations’ I was thrilled to have the opportunity to engage with the Relating Systems Thinking and Design community last month. The Environment, Economy, Democracy:  Flourishing Together RSD6 conference (at the Oslo School of Architecture and Design, Oslo, Norway, October 18-20, 2017) was an ideal place to collect ideas from designers, academics and sustainability practitioners with expertise in systems mapping, design and the visualisation of complexity.

A Whole New World – What Next?




Toby Lowe, Newcastle University

It’s been a month since we launched ‘A Whole New World: Funding and Commissioning in Complexity’ and the response has been incredible. Over 120 people came to the two launch events, and more than 1000 people read the report online in the first week. We’ve also started to have some excellent conversations about how to take this work forward.

In this post I want to briefly outline the key ideas, and talk about what we might do next.

‘New Public Management’ is dying – about time too

New Public Management (NPM) has been the dominant paradigm for public services for the last 40-odd years.  Its worldview is based on the idea that public servants cannot be trusted to organise and run public services, and so must be extrinsically motivated to perform well – by means of competitive markets and performance targets.  It seems that an increasing number of people recognise that this way of funding and commissioning public services, and other social interventions, is no longer helpful.