By Caroline Oliver, Reader in Sociology, Roehampton University and Research Affiliate at the Centre on Migration, Policy & Society, University of Oxford
24th October 2018
As a social scientist, I find myself increasingly caught up in the ‘impact’ agenda, since universities are audited in the Research Excellence Framework on how the work of researchers has economic and social benefits. While there is merit in looking critically at this agenda (Holmwood 2011) it also provokes researchers to actively imagine a role for the social sciences in influencing social policy and practice, informing social reform through using robust evidence and social theory (Pawson 2000; Reed and Chubb 2018). The need for this is great; as Brian Castellani put it in his recent blog, we are witnessing an increasingly widespread ‘culture of cruelty’ and backlash against the establishment of a more just and equitable civil society. Engaged governance and just social policies become important means of challenging negative psychologies based on escalating fears, conflicts and resentments, while knowledge from research and evaluation provides vital ingredients in the search for something different.
Nowhere is this more evident than in the field of international migration, an area in which I have worked as a scholar for twenty years, and now, as an evaluator (see here). However, as in the nexus areas of CECAN’s brief, in the field of international migration, there is no agreement between stakeholders and beneficiaries on what a ‘just social policy’ looks like. The diversity of opinion is compounded by many layers of governance, where the ‘solutions’ arrived at by national governments look often very different from the perspectives of cities dealing with challenges of migration on their doorsteps. These ‘solutions’ may even look like ‘problems’ when viewed from the perspective of local populations or refugee charities (although even then, for quite different reasons). Deliberations about new initiatives and new responses are often highly normative, and occur in dynamic and even volatile environments, where governments respond to changing expectations wrought by international politics and national political currents, sometimes even overnight.
In such volatile contexts, CECAN’s key messages for complex evaluation offer vital lessons to those seeking to build just social policies addressing this and other similar topics. The willingness to embrace uncertainty in the development of flexible and adaptive responses, learn through feedback loops, encourage teams to talk freely and openly about evidence and reflect on mistakes in a dynamic process, are important messages to harness. However, equally crucial is the need for critical unpacking of broader assumptions and recognition of power operating within and without the contexts of an initiative operating in this field. We are often well acquainted at recognising power dynamics within the evaluation relationship, as evaluators make choices and wield some form of judgement over an initiative, speak back to policymakers, or ensure that others with less power have the opportunity to make their voices heard and even count. However, recognition of power dynamics operating in the broader contexts of an initiative itself is also vital.
In the migration field, in particular, this recognition requires a broad understanding of ‘context’, which goes beyond the influences of local geography and social makeups of cities in which an initiative operates. It means acknowledgement of the power of international and national politics, including constraints at these levels which might act as crucial factors in what an initiative is about to achieve (Geuijen, Oliver and Dekker 2019, forthcoming). It also means consideration of social currents, including increasing populism, in which an initiative makes its way. This broad context is not just an external backdrop, for example as in a PESTLE analysis, but as influences that are present within an endeavour and have an impact within the daily interactions of publics affected. Recognition of the importance of context in this sense might ask of us to (re)consider throughout an initiative what success looks like. For example, even small steps in achieving local consent for an initiative might be half the battle (initial responses to the opening of the centre we are evaluating, from some quarters in the neighbourhood were desires to set it on fire). Important critical and transparent deliberation needs to occur over questions of value: for who? For how many? Over how long?
The exercise of evaluation in such volatile contexts also entails untangling how action and capacity for change within an initiative is shaped, or constrained dynamically by different interest groups. As such, it also requires reflexivity about how, within such contexts, evaluation evidence is highly significant as a tool in multiple actors’ positioning for the next move. The media becomes a willing ideological battleground in which an innovation needs to prove its mettle immediately amid a field of dissenting and critical voices. In such contexts, discussing errors for greater learning can be a very sensitive matter indeed, placing even greater pressure on evaluation to reduce evidence to ‘simple’ answers without caveats.
In negotiating this terrain, a key responsibility for evaluators is to respond nimbly, yet also safeguard time for reflection on how and by whom power and control is wielded as an initiative takes shape, building in social theory about these processes. Such practice will help expose the power dynamics and broader societal contexts that are also in operation and shape an initiative. This would ultimately facilitate evaluation to transform understandings, to guide policies in the right direction, and ultimately support engaged governance to gain traction in highly contested terrains. By this, one would hope, we might make a real impact through our research.
Chubb, J. Reed, M.S. ‘The politics of research impact: academic perceptions of the implications for research funding, motivation and quality’. British Politics. 13 (3): 295-311.
Geuijen,K., Oliver, C. & Dekker, R. 2019 ‘Local innovation in Asylum seeker reception’, in Glorius, B. & Doomernik, J. 2019, forthcoming. Geographies of Asylum in Europe and the Role of the Local. Amsterdam: IMISCOE Springer.
Holmwood, J. 2011. ‘Viewpoint – The Impact of ‘Impact’ on UK Social Science’. Methodological Innovations Online. 6(1) 13-17
Pawson, N. 2000. Realistic Evaluation: An overview.