Blog

Building Collaborative Narratives and Developing Trust in Policy and its Evaluation.

dont steer blind

 

Reflections from the 6th European Environmental Evaluators Network forum: Betheney Wills, CECAN PhD Researcher.

On the 23rd and 24th November 2017 members of the CECAN team, Pete Barbrook-Johnson, Clare Twigger-Ross and myself attended the EEEN conference in the beautiful surroundings of the Royal Society in Edinburgh. Facilitated by the Scottish Environmental Protection Agency (SEPA), civil servants from environment agencies, consultants, practitioners and academics came together to share experiences and ideas for the future of environmental policy evaluation.

The conference’s theme - Evaluating Innovation in Environmental Protection and Sustainability - considered whether evaluation is evolving in line with the societal and environmental challenges policy faces. Participants were asked to keep the following questions in mind over the two days;

How do we evaluate innovation and how do we innovate in evaluation?

There was a variety of talks and subsequent discussions around topics including the circular economy, EU laws, the use of economic mechanisms, government policy, low-carbon incentives and innovation.

An Extra Claus for the Santa Book? Evaluating Christmas the CECAN Way!

Evaluating Christmas

Anne Liddon, Science Communications Manager / Fran Rowe, Research Assistant / Amy Proctor, Research Associate, Centre for Rural Economy, Newcastle University

As we all know CECAN is pioneering, testing and promoting innovative policy evaluation approaches and methods across Nexus domains through a series of “real-life” case study projects - it’s about measuring best value in complex, inter-connecting systems.  And what could be more complex and “real-life” than Christmas?  We are all busy people and we want to get the best out of this annual festivity but what are the best methods for evaluation that can help us to do that?  We have been consulting with the most important actors involved (elves, reindeer, over-excited children, and, of course, Santa Claus himself) to consider the challenges and opportunities.

CECAN’s ‘The Visual Representation of Complex Systems’ at Environment, Economy, Democracy: Flourishing Together #RSD6.

Visual Representation of Complex Systems

By Dr Joanna Boehnert, CECAN Fellow

As the first step in developing my CECAN research project titled ‘The Visual Representation of Complex Systems: A Typology of Visual Codes for Systemic Relations’ I was thrilled to have the opportunity to engage with the Relating Systems Thinking and Design community last month. The Environment, Economy, Democracy:  Flourishing Together RSD6 conference (at the Oslo School of Architecture and Design, Oslo, Norway, October 18-20, 2017) was an ideal place to collect ideas from designers, academics and sustainability practitioners with expertise in systems mapping, design and the visualisation of complexity.

A Whole New World – What Next?

 

uncertainty

 

Toby Lowe, Newcastle University

It’s been a month since we launched ‘A Whole New World: Funding and Commissioning in Complexity’ and the response has been incredible. Over 120 people came to the two launch events, and more than 1000 people read the report online in the first week. We’ve also started to have some excellent conversations about how to take this work forward.

In this post I want to briefly outline the key ideas, and talk about what we might do next.

‘New Public Management’ is dying – about time too

New Public Management (NPM) has been the dominant paradigm for public services for the last 40-odd years.  Its worldview is based on the idea that public servants cannot be trusted to organise and run public services, and so must be extrinsically motivated to perform well – by means of competitive markets and performance targets.  It seems that an increasing number of people recognise that this way of funding and commissioning public services, and other social interventions, is no longer helpful.

Complexity Settings to the Rescue: A New Lease of Life for Evidence-Based Policy?

complexity science

 

CECAN Fellow Sara Giorgi shares her perspective on some of the key insights from her research.

It would be naive and, potentially, ill-advised to have evaluation solely drive policy direction. Good, open, evidence-backed policy, however, does need to be informed by evaluation results and insights.  My CECAN Fellowship provided me with a rare opportunity to investigate how evaluation is applied in real life within a government department – in this case Defra – and how it can be used to plan for future policymaking. 

Models as ‘Interested Amateurs’

models as interested amateurs

From 'i2insights.org' 24th January 2017

Pete Barbrook-Johnson, Research Fellow at the Policy Studies Institute at the University of Westminster & Knowledge Integrator Research Fellow at the Centre for the Evaluation of Complexity Across the Nexus (CECAN) at the University of Surrey. 

How can we improve the often poor interaction and lack of genuine discussions between policy makers, experts, and those affected by policy?

Taking The Temperature of Trust - Henry Leveson Gower

taking the temperature of trust

Henry Leveson-Gower shares emerging findings of in-depth interviews he has conducted as part of his ongoing research project on trust in water.

It is over two years since Ofwat set increasing trust in water as its key objective for its 2015 five year business plan.To establish what has happened since, I interviewed 19 senior and influential figures across all parts of the water sector. I am seeking wider perspectives via a survey (click HERE to participate) and hope this article will stimulate you to take part.

One Researcher’s Anecdote is Another Researcher’s Data

one researchers anecdote

Frances Rowe, Newcastle University

A couple of days ago, a DEFRA policy official told me that the uncertainty over EU exit was creating a fertile environment for evaluation, as champions try to ensure their favoured policies have a place in the forthcoming landscape, post Brexit. This struck me as interesting, and I made a note of it. While some may call this anecdote, for a qualitative researcher this is data: incomplete, uncorroborated, yes, but data nonetheless.  It might in a future analysis about evaluation uptake prove to be gold dust, an insight that unlocks others, a necessary factor in assessing evaluation effectiveness. Who knows?

That Way Lies Prosperity: Sustainability and the Nexus

sustainability and the nexus

Dr Adam Hejnowicz, University of York

Sustainable Definitions

Sustainable Development and its relative Sustainability, concepts which have a rich history of appeal and animosity, have nevertheless become the dominant conversation framing environment-development policy in recent decades.

The most recent version of Sustainable Development adopted by the UN General Assembly in 2012 reads as follows:

“We also reaffirm the need to achieve sustainable development by: promoting sustained, inclusive and equitable economic growth, creating greater opportunities for all, reducing inequalities, raising basic standards of living; fostering equitable social development and inclusion; and promoting integrated and sustainable management of natural resources and ecosystems that support inter alia economic, social and human development while facilitating ecosystem conservation, regeneration and restoration and resilience in the face of new and merging challenges.”

Agriculture Fit For A Complex World

complex agriculture

Adam Hejnowicz, Postdoctoral Research Associate - Environmental Policy Evaluation (CECAN) - University of York

Complex Realities

To say that we live in a complex world is, in a very general sense, rather banal and uninteresting being neither particularly illuminating nor especially profound. But, scratch beneath the surface, and an acknowledgement of that complexity can be revelatory.

In a complex world the choices and decisions we make, and the consequences that result from the actions taken in response to those decisions, do not hold clear and predictable outcomes. This is especially true where the context and conditions in which we live, and in which those decisions are made, frequently and rapidly change.  The recent changes in the political circumstances of the UK and USA are good examples.

This is the world, not just of reality, but also of policy. If we fail to recognise these realities then the policies we develop to deal with the global challenges we face will be poor and, at best, so will the outcomes of those policies. At worst, the results of poorly devised and implemented policies could prove catastrophic to both our future wellbeing and the sustainability of the Earth System.

Why We Need Network Analysis to Understand the Future of Economics

Stuart Astill

Stuart Astill, IOD Parc - From 'OpenDemocracy.Net' 2nd November 2016

What it is, what it’s not

Network analysis is the method of the future. That is not only – certainly not primarily – because we are ever more connected in some superficial social-media driven internet sort of way. All of that may be fascinating (and certainly can be analysed using network analysis), but it is not fundamental to our existence as humans – we existed before Facebook, we will exist after it is gone.

Entirely fundamental though are the complex linkages between humans, problems and resources. And those linkages are just as important as the humans, problems and resources themselves. Analysing the links, not just the elements in isolation, requires network analysis.

The problem

In environmental, human and, therefore, long-run economic terms the models we use to describe the world currently find false optimal flight-paths towards unsustainable monolithic solutions. And don’t forget what an important and multi-faceted word unsustainable is – not just environmental concerns, but also the physical and mental health of populations, poverty and income divergence, political and societal fractures.

CECAN Complexity in Evaluation Workshop: What we Did and What we Learned

complexity workshop blog

 

The first of two blogs following this event, from the perspective of the lead facilitator, Dr Paul Brand

‘The Complexity in Evaluation Workshop: What we did and what we learned’

This was my first experience of facilitating for CECAN.   I’d heard about it from colleagues who are involved and it all sounded fascinating, but perhaps a little ‘technical’ – possibly even a tad ‘dry’.

This event was publicised as:

“This 2 day residential workshop, conducted under the Chatham House Rule, will bring together evidence teams, policy makers, policy analysts, complexity scientists, evaluation experts and experts in Nexus subjects.”

Fascinating I thought, but not necessarily a hotbed of creativity and passion.  How wrong I was!

The post event feedback included comments like:

‘A journey’, ‘worthwhile’, ‘enjoyable’, ‘dynamism’, ‘energy’ and ‘larding’ (the latter I will explain later).

Why Carry Out Economic Evaluation?

Impact blog

Dr Ulrike Hotopp, Chief Economist, Simetrica and member of the Council of the UK Evaluation Society

I have only recently joined the small economic research consultancy Simetrica. Before this I spent 16 years in the Government Economic Service, starting as an economic advisor in DTI in 2000 (now known as BEIS). I first worked on employment policy and one of my main tasks was to produce Impact Assessments for new employment regulation using the tools of Cost Benefit Analysis (CBA). These can also be described as ex-ante evaluation – forecasting the impact of new regulation.

Having done a few of these, an important question is: how realistic was the assessment? Was it really true that improved maternity rights had no impact on female employment but instead improved the rate of women returning to work after having a child? Were there any wider micro and macro economic effects? On other groups in the labour market, company profits?

Access to Data is Crucial

data

Emma Uprichard and Robert MacKay, University of Warwick

Interviewed by Candice Howarth, University of Surrey

CECAN is exploring how evaluation of policy can better inform the impact those policies have and assess the extent to which these have been successful. In order to do this, access to data is crucial, yet can at times be problematic. CECAN’s Knowledge Integrator, Candice Howarth met Emma Uprichard and Robert MacKay from the Centre and based at the University of Warwick and asked them over a series of emails to explain what the implications of some of these challenges are.

Why is data needed in evaluations?