Whose evidence counts

The second of my five from the State of Evaluation report, and the one that goes to what counts as evidence, and who gets to decide.

The report found that when evaluators define what they do, they converge on three words: judgement, value and learning.

In arts, culture and heritage they are exactly the three that are often the hardest to make room for. With so many contingent pressures bearing down on the work, the space and time to learn is hard to find (learning), the room to respond rather than react can be harder still (judgment), and the question of what value actually means, which registers of value really work for us (value), is one that I have seen asked repeatedly, and answered in myriad pragmatic to ideological ways.  

The pressure I see, across funders, is to translate cultural and community outcomes into economic proxies. Footfall, spend, jobs, a Gross Value Added or ROI figure that travels well in a business case. None of these is wrong, but even together they are a thin description of what a place or a piece of work actually did for the people in it.

This is not an argument against metrics. Good evaluation needs them, and I want more of them, not fewer. The argument is for the right ones. A metric is only as good as its fit to what the work was actually trying to do, and too often we reach for the numbers that are easy to gather and travel well rather than the ones that tell us something true. Judgement and value are harder to count and easier to feel. The risk is that we measure what is convenient and then call it what matters.

It helps here to be clear about what evaluation is, and is not. It is close kin to research, and it borrows research's rigour, its methods and its honesty about evidence. But it is not the same job. Research asks what is true. Evaluation asks, against this, was this good, for whom, and what do we do next. It is research in service of a judgement, and that judgement is the point. Forget the difference and you get studies that are methodologically tidy and useless to the people who have to decide something.

Holding the line on cultural value is part of the job, not a softer alternative to it.

#StateOfEvaluation #Evaluation #UKEvaluationSociety #CulturalValue #Heritage #SocialValue #ArtsAndCulture #ImpactMeasurement