Dr Ulrike Hotopp, Chief Economist, Simetrica and member of the Council of the UK Evaluation Society
I have only recently joined the small economic research consultancy Simetrica. Before this I spent 16 years in the Government Economic Service, starting as an economic advisor in DTI in 2000 (now known as BEIS). I first worked on employment policy and one of my main tasks was to produce Impact Assessments for new employment regulation using the tools of Cost Benefit Analysis (CBA). These can also be described as ex-ante evaluation – forecasting the impact of new regulation.
Having done a few of these, an important question is: how realistic was the assessment? Was it really true that improved maternity rights had no impact on female employment but instead improved the rate of women returning to work after having a child? Were there any wider micro and macro economic effects? On other groups in the labour market, company profits?
Of course it takes a long time for the effects to feed through the labour market and society, and a lot of other things happen, which also affect the employment rate of women of child-bearing age, but the need for ex post evaluation, i.e. checking out whether a policy has actually achieved what it set out to achieve, is obvious. However, this obvious relationship between economics and evaluation is not always recognised by professional economists.
This need for curiosity into impact is not just the case for central government policy making. Companies and charities also feel the need to find out whether their initiatives had the expected effects. Has a marketing campaign raised turnover of a firm or a charitable project reduced the economic pressures in a struggling community, here in the UK or further afield?
There are many techniques, mainly but not only used by economists which provide rigour to the analysis, point out unintended consequences, successes and failures. These techniques and methods are robust. You can find many articles in the peer reviewed economic journals across the profession, including the Royal Economic Society’s Economic Journal.
I would like to give one example: In my new job at Simetrica I conducted an evaluation for the charity C2 (Connecting Communities). They had intervened in a deprived community in Cornwall. Their main intervention was to give people a voice, allowing them to formulate their wishes for a clean environment, warm homes and a safe place to walk around. My approach was to use difference in difference method, compare data before and after the intervention within the community and with the developments in the rest of the country.
As a “proper” economist I took with a significant pinch of salt all the claims made by C2. Despite my critical viewpoint it turned out that the work of C2 had a positive and lasting effect on people. And, from a value for money point of view, it was good business for C2 to invest in Simetrica’s evaluation. They managed to use it to raise almost twice as much as the evaluation costs and they now spend this on other projects.
Important elements of economic evaluations are a focus on quantification of inputs (costs) and outcomes (benefits), ensuring that all values are compared in a consistent time frame (Net Present Value). The boundaries of what can be quantified and monetised have been pushed forward and include well-being and life satisfaction. Using ethical and philosophical approaches economics allows a comparison between people and time, providing objective analysis of the effectiveness of policy and other actions. In addition economic evaluation, if conducted comprehensively, can show the unintended consequences which may be caused by distortions in markets and decision making.
I hope that CECAN’s work will attract more economists to apply their toolkit and insights not just to the forward looking impact assessment style work but also to the systematic evaluation of policy and other interventions. In addition, this will bring the pleasure of working with other sciences – social and natural – but that is for another blog.
Dr Ulrike Hotopp, Chief Economist, Simetrica and member of the Council of the UK Evaluation Society (www.evaluation.org.uk)