A Pluralistic Evaluation Framework

Richard Gunton
 
Richard Gunton, CECAN Fellow
 
I wrote at the start of this year about my work on articulating values in evaluation.  This has crystallised into a pluralistic evaluation framework, which I will shortly be presenting to policymakers from Defra and other agencies (in London on 12 June and in York on 3 July).  This workshop is intended to offer fresh ideas and a simple template to assist at all stages of the policy cycle - from appraisal and impact assessment through to full evaluation.
 

Value pluralism is the technical term for an ethical approach that recognises multiple kinds of values that cannot be collapsed down to a single value. It contrasts with approaches that posit an ultimate kind of goodness such as happiness, or economic wealth.  Having a single objective allows a calculus of consequences which allows an algorithm for decision-making that might be dubbed “objective” - but considering multiple, incommensurable kinds of goodness is surely more like our natural approach to decision-making.  Given that goodness is ultimately a concept derived from experience, an ethical framework may reasonably be judged by how far it reflects a natural human approach and facilitates the exercise of judgment. Considering multiple different values at once is an important characteristic of human judgment - and part of the classic challenge of good government and policymaking.

What policy evaluation needs, therefore, is tools that can help us handle multiple values and the complexity associated with them.  Indeed, in a culture where great emphasis is placed on technical solutions, scientific analyses and artificial intelligence, it may be important to provide tools that are designed to facilitate judgment and clarify the limits of what algorithms can achieve.  That is the nature of the framework that I am working on.

I’ll say more in detail about this pluralistic evaluation framework in my next post.  By then I hope to have learned a lot from participants at these two workshops, and from conversations with my co-presenter and mentor Dr Ian Christie, and a wider team working on these topics with me.