The pluralism we are quietly squeezing
The fourth of my five from the State of Evaluation report is the finding I found most cheering and most troubling at once.
UK evaluation draws on more than twenty disciplinary backgrounds and is full of portfolio careers and fluid identities. The report calls this pluralism a strength, and it is. It is what makes evaluation adaptive and contextually sensitive. In place work, that sensitivity is not a nice-to-have. Every place is particular, and the methods worth using are the ones that fit it.
But there is a quiet pressure pulling the other way. Funders, reasonably enough, want metrics that compare across a portfolio. Comparability favours standardisation, and standardisation flattens the particular. The very thing that lets evaluation speak to a place is the thing a comparable dataset asks it to give up.
And standardisation is rarely neutral. A common framework usually arrives as a blueprint, set elsewhere and rolled out the same each time, against which a place is asked to make itself legible. That is a question of power as much as method. The funder defines what will be counted, and so quietly defines what is allowed to matter, and the funded organisation shapes itself to fit, because the next grant depends on it. The same imbalance runs all the way through to the participant whose experience is collected against categories an evaluator chose, in language the evaluator brought. The metrics decide whose account of the work counts as evidence. An agile, adaptable approach can hand some of that definitional power back, letting a place set its own terms. A blueprint tends to keep it at the top.
I do not think we have to choose between them every time – I come with a blueprint of values, ethics, and practices honed over years; equally, I am ready to start from scratch in any new project. And too, its is heartening and invigorating to see what questions funders are asking of themselves these days.
But we should be honest that the pull towards one voice and the need to honour the particular are in real tension, and be clear which one we are serving.
#StateOfEvaluation #Evaluation #UKEvaluationSociety #EvaluationMethods #MixedMethods #Interdisciplinary #Placemaking #ContextMatters