John Holden is an associate at the thinktank Demos and a visiting professor at City University London; Alex Walker is head of arts and heritage at Preston City Council
Dear Alex:
In April 2013, culture secretary Maria Miller said: “It goes without saying that culture’s value to our society is immeasurable.”
So why do we spend so much time, energy and money on measurement? Partly to satisfy the myth that politicians take decisions based on evidence, when most of the time they clearly do not.
Of course, evidence has a place in improving practice and decision-making – but it has become fetishised. Evidence is not a cure-all, as it always needs interpretation, no matter how good it is – and often it is not very good at all.
John
Dear John:
In the same speech, she referred to things that clearly had been measured – whether it be the percentage of tourists coming to the UK for cultural experiences or the economic impact of spending associated with cultural attractions.
More recently, she has spoken about “what makes our hearts sing”. That, I agree, is immeasurable.
Or do we mean unmeasurable? The key is deciding what and why we are measuring and what we are going to do with the results
Alex
Dear Alex:
We have to recognise three things. The first is the methodological problems: the difficulty of showing cause and effect; the problem of experience in one context not being transferable to another; the issue of advocacy disguised as research; and the changes in policy goals that subvert the possibility of getting sound longitudinal data.
The second is the distorting effect that data-gathering has on the operations of cultural organisations. The third is that, however good the evidence is, some decision-makers will choose to disbelieve it.
John
Dear John:
Participation in a cultural activity isn’t like participating in a sport, where changes in weight or blood pressure can be investigated.
Asserting that we believe or even “know” that culture has a range of instrumental impacts is not enough. Perhaps we should be confident enough to say that although instrumental benefits are important, they are not the main point.
But in times when we all need to win funders’ support – and when funders need to be able to defend their decisions –this might feel risky.
Alex
Dear Alex:
What worries me is that funder demands for evidence bring with them ever-increasing micro-management, so we could end up like the education sector: demoralised, and terrified of Ofsted-like inspections.
Evidence gathering has two useful roles – first in creating a general “climate” that influences decision-making and second, if it’s done honestly, as a management tool to help improve practice.
What I am against is the kind of evidence people produce to justify their next round of funding. Too much evaluation falls into that category.
John
Dear John:
Maybe the debate should be about the minimum evidence that funders need, the standard and methodology of collection. Given their own stretched resources to analyse data, it is in no one’s benefit to collect information that cannot be used.
Some of the most compelling evidence for politicians and others are the real-life stories of “museums changing lives” and being able to see with their own eyes the wonder of children on their first museum visit, or the poem written by a young person on probation dedicated to the access worker who introduced him to creative activity.
Alex
Dear Alex:
In April 2013, culture secretary Maria Miller said: “It goes without saying that culture’s value to our society is immeasurable.”
So why do we spend so much time, energy and money on measurement? Partly to satisfy the myth that politicians take decisions based on evidence, when most of the time they clearly do not.
Of course, evidence has a place in improving practice and decision-making – but it has become fetishised. Evidence is not a cure-all, as it always needs interpretation, no matter how good it is – and often it is not very good at all.
John
Dear John:
In the same speech, she referred to things that clearly had been measured – whether it be the percentage of tourists coming to the UK for cultural experiences or the economic impact of spending associated with cultural attractions.
More recently, she has spoken about “what makes our hearts sing”. That, I agree, is immeasurable.
Or do we mean unmeasurable? The key is deciding what and why we are measuring and what we are going to do with the results
Alex
Dear Alex:
We have to recognise three things. The first is the methodological problems: the difficulty of showing cause and effect; the problem of experience in one context not being transferable to another; the issue of advocacy disguised as research; and the changes in policy goals that subvert the possibility of getting sound longitudinal data.
The second is the distorting effect that data-gathering has on the operations of cultural organisations. The third is that, however good the evidence is, some decision-makers will choose to disbelieve it.
John
Dear John:
Participation in a cultural activity isn’t like participating in a sport, where changes in weight or blood pressure can be investigated.
Asserting that we believe or even “know” that culture has a range of instrumental impacts is not enough. Perhaps we should be confident enough to say that although instrumental benefits are important, they are not the main point.
But in times when we all need to win funders’ support – and when funders need to be able to defend their decisions –this might feel risky.
Alex
Dear Alex:
What worries me is that funder demands for evidence bring with them ever-increasing micro-management, so we could end up like the education sector: demoralised, and terrified of Ofsted-like inspections.
Evidence gathering has two useful roles – first in creating a general “climate” that influences decision-making and second, if it’s done honestly, as a management tool to help improve practice.
What I am against is the kind of evidence people produce to justify their next round of funding. Too much evaluation falls into that category.
John
Dear John:
Maybe the debate should be about the minimum evidence that funders need, the standard and methodology of collection. Given their own stretched resources to analyse data, it is in no one’s benefit to collect information that cannot be used.
Some of the most compelling evidence for politicians and others are the real-life stories of “museums changing lives” and being able to see with their own eyes the wonder of children on their first museum visit, or the poem written by a young person on probation dedicated to the access worker who introduced him to creative activity.
Alex
