The Story of the Pluralistic Evaluation Framework

Pluralistic Framework

By Richard Gunton, CECAN Fellow

19th November 2018


The pluralistic evaluation framework is a new tool for policymakers that has gradually taken shape during the last 12 months of my CECAN fellowship. It is now ready to be presented at a webinar on 20 November, where I will be explaining the rationale in the space of 45 minutes. Here I want to share a little of the journey that it has been on, building on what I wrote here last January and June


Power and Hope: Using Evaluation in Volatile Contexts


By Caroline Oliver, Reader in Sociology, Roehampton University and Research Affiliate at the Centre on Migration, Policy & Society, University of Oxford

24th October 2018


As a social scientist, I find myself increasingly caught up in the ‘impact’ agenda, since universities are audited in the Research Excellence Framework on how the work of researchers has economic and social benefits. While there is merit in looking critically at this agenda (Holmwood 2011) it also provokes researchers to actively imagine a role for the social sciences in influencing social policy and practice, informing social reform through using robust evidence and social theory (Pawson 2000; Reed and Chubb 2018). The need for this is great; as Brian Castellani put it in his recent blog, we are witnessing an increasingly widespread ‘culture of cruelty’ and backlash against the establishment of a more just and equitable civil society. Engaged governance and just social policies become important means of challenging negative psychologies based on escalating fears, conflicts and resentments, while knowledge from research and evaluation provides vital ingredients in the search for something different.


The Nexus: A New Approach to Sustainability Transformations – What, Why and How

Pollution photo

By Adam Hejnowicz (CECAN, University of York), Pete Barbrook-Johnson (CECAN, University of Surrey), Kirsty Blackstock (The James Hutton Institute), and Chad Staddon (University of West England).

12th September 2018

This blog reports on recent presentations and discussions held at the Royal Geographical Society annual conference in Cardiff, at a special session on adaptive management and governance of the food-energy-water-environment nexus. The session was jointly organised by researchers from CECAN and the James Hutton Institute.


What is a Nexus Approach?

A Pluralistic Evaluation Framework

Richard Gunton
Richard Gunton, CECAN Fellow
I wrote at the start of this year about my work on articulating values in evaluation.  This has crystallised into a pluralistic evaluation framework, which I will shortly be presenting to policymakers from Defra and other agencies (in London on 12 June and in York on 3 July).  This workshop is intended to offer fresh ideas and a simple template to assist at all stages of the policy cycle - from appraisal and impact assessment through to full evaluation.

Value pluralism is the technical term for an ethical approach that recognises multiple kinds of values that cannot be collapsed down to a single value.

Looking Through a Glass Darkly: Public Goods and Agricultural Policy

Brexit & Environment

From Brexit & Environment, 9th March 2018

By Dr Adam Hejnowicz and Prof. Sue Hartley

This blogpost highlights some of the points raised in a policy brief, New Directions: A public goods approach to agricultural policy post-Brexit.

Following the Brexit referendum, the mantra “public monies for public goods” has been increasingly heard, especially in relation to agricultural and environmental policy. This public goods agenda represents, in part, a reaction against the well described failings of the Common Agricultural Policy (CAP), which despite greening reforms continues to oversee declines in European environmental quality.

Humility and courage: strategy and evidence in our complex world

Humility and courage

Stuart Astill, IOD Parc

Here in CECAN, a lot of time is spent considering and analysing policies, projects and interventions. We also think about methodologies, approaches and systems that we can adopt to do this analysis in the best way possible.

At the same time, we all recognise that there is a particular way of thinking that is valuable when working in the complex world – and certainly other ways of thinking that can trip you up quickly.

It is clear, if we take a sideways look at much of what CECAN does, that having the mindset of an explorer, a navigator, an illuminator, a learner and a satisfier is valuable.

Trying to be a ‘finisher’ or seeker of certainty would be inadvisable; being a ‘taker’ of ‘results’ rather than a learner would be foolish and to be a ‘maximiser’ would lead to a lifetime spent collecting information that still turned out to be incomplete. By definition.

Dynamic Pattern Synthesis for Modeling Complex Systems. An Interview with Phil Haynes


Courtesy of Brian Castellani's 'Sociology & Complexity Science' Blog.

Phil Haynes is Professor of Public Policy and researches and teaches public policy and management, as applied to a variety of contemporary circumstances. His research focuses on the application of complex systems theory, often using applied statistical methods. His research has been funded by the ESRC and the government and voluntary sector. He has published in a wider variety of journals including Social Policy and Administration and Public Management Review.  He is author of several books including Managing Complexity in the Public Services (2015) now in its second edition.

His most recent book, which is part of our complexity in social sciences series at Routledge, is aptly titled, SOCIAL SYNTHESIS: Finding Dynamic Patterns in Complex Social Systems.

Finding The Common Ground

Finding the common ground

Along with several lead and co-authors, as part of a British Ecological Society Agricultural Ecology Group convened workshop held in December 2017, a report has been produced entitled "Finding the Common Ground", which sets out an ecological perspective on how future agricultural policy should develop as a consequence, and in relation to, Brexit.

The following blog by Steve Peel, Independent Eco-Agronomist sets out the main arguments:

Brexit is going to influence many aspects of life in the UK, few more so than farming and the environment.

Carillion may have collapsed, but public-private partnerships can be so much more...


public private partnerships

Pete Barbrook-Johnson @bapeterj

Last month, Carillion, one of the largest companies in the UK which regularly entered into contracts with government to deliver public infrastructure and services, went into liquidation. Since then, public-private partnerships (PPP), and their pantomime villain superstars - private finance initiatives (PFI) - have received an unprecedented level of criticism. The Guardian Opinion section – and my love-hate relationship with it - has gone into overdrive!

Teaching Evaluation of Complex Policy and Programmes

CECAN Syllabus

By Corinna Elsenbroich, CECAN Research Fellow

CECAN develops, tests and enhances methods to deal with complexity in policy evaluation, trying to advance research, policy and evaluation practice. To ensure that these methods will influence an ever widening audience, CECAN has now launched a syllabus for building capacity and supporting the application of complexity sensitive evaluation nationally and internationally.

Putting Values Back in Evaluation.

putting values back in evaluation

By Richard Gunton, CECAN Research Fellow

Policy evaluation is about assessing the value of policies, but too little attention, it seems to me, is paid to the meaning of “value” in all this.  The English word “value”, of course, has multiple meanings that include numerical (e.g. “a parameter value”), economic (e.g. “good value”) and ethical (e.g. “a value judgment”).  And herein lies an interesting problem, because this multiplicity of meanings makes it easy to present questions having an ethical component as if they were objective problems that might be solved in purely mathematical or economic terms.  Evaluating the National Curriculum’s mathematics programme is not at all like evaluating x-squared when x equals a half!  The ultimate question that evaluators ask of policies is: how good is this policy?  And that is primarily a question of value in the ethical sense, because goodness is a normative concept.

Building Collaborative Narratives and Developing Trust in Policy and its Evaluation.

dont steer blind


Reflections from the 6th European Environmental Evaluators Network forum: Betheney Wills, CECAN PhD Researcher.

On the 23rd and 24th November 2017 members of the CECAN team, Pete Barbrook-Johnson, Clare Twigger-Ross and myself attended the EEEN conference in the beautiful surroundings of the Royal Society in Edinburgh. Facilitated by the Scottish Environmental Protection Agency (SEPA), civil servants from environment agencies, consultants, practitioners and academics came together to share experiences and ideas for the future of environmental policy evaluation.

The conference’s theme - Evaluating Innovation in Environmental Protection and Sustainability - considered whether evaluation is evolving in line with the societal and environmental challenges policy faces. Participants were asked to keep the following questions in mind over the two days;

How do we evaluate innovation and how do we innovate in evaluation?

There was a variety of talks and subsequent discussions around topics including the circular economy, EU laws, the use of economic mechanisms, government policy, low-carbon incentives and innovation.

An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!

Evaluating Christmas

Anne Liddon, Science Communications Manager / Fran Rowe, Research Assistant / Amy Proctor, Research Associate, Centre for Rural Economy, Newcastle University

As we all know CECAN is pioneering, testing and promoting innovative policy evaluation approaches and methods across Nexus domains through a series of “real-life” case study projects - it’s about measuring best value in complex, inter-connecting systems.  And what could be more complex and “real-life” than Christmas?  We are all busy people and we want to get the best out of this annual festivity but what are the best methods for evaluation that can help us to do that?  We have been consulting with the most important actors involved (elves, reindeer, over-excited children, and, of course, Santa Claus himself) to consider the challenges and opportunities.