The Menace of Measurement: A discussion about arts indicators
SAA commissioned Marnie Badham to write a position paper on arts metrics. The Abstract and Executive Summary are posted on this website. For a copy of the complete report, please contact the SAA.
Abstract
The arts are multifaceted, complex, and have both intrinsic and instrumental values. Intrinsic values have little language to articulate, therefore we typically decline trying to articulate these and rely on evaluating the arts sector through instrumental impacts that are much easier to qualify. Therefore, the arts are measured through economic and social benefits and the true nature of the arts, the aesthetic, communicative and cognitive development roles, remain unarticulated. We resist measurement because we are asked to measure the wrong things and we feel little resources should be spent on the creation and dissemination of the arts.
Arts indicators are not evaluation. They are often misunderstood as social or economic indicators, because they are measuring wealth or service provision. Arts indicators, as a branch of cultural indicators, measure changing social values over time, and as such, should not be used to compare locations or measure performance of isolated individual phenomena (like economics) within the arts sector. Indicators require a conceptual handshake between practice and policy, and can provide better understanding of the arts by observing the changing public perceptions. This data, embedded in a larger theoretical framework, can communicate to the public about the value of the arts and, as importantly, provide evidence to decision makers for policy change.
This discussion paper provides an overview of the argument about measurement and the arts, a detailed theoretical review of international arts and cultural indicators, and proposes some thoughts for steps forward for arts indicators in Saskatchewan.
Executive Summary
In Saskatchewan, the arts and cultural industries are an important part of the province. To maintain their lively existence and continued growth, the Saskatchewan arts community, including artists, patrons, policy makers and those who benefit need to do a better job of articulating the value of the arts. We need to know more about art and how it relates to our lives. We need to know how the public feels about the arts. We need to gather, develop and distribute more knowledge about the arts: improved statistics and indicators (top down), map the existing local arts and cultural activities (bottom up) and translate this into relevant policy language. Arts indicators may be one tool to help us.
The recent global resurgence of social indicators has governments and communities interested in new ways of measuring progress. Indicators typically measure things that are easily quantifiable, however, we know the arts are complex, multifaceted and difficult to quantify through instruments of social science. As a result, measurement and value has typically been focused on instrumental, social, and economic arts outcomes. Therefore, the intrinsic qualities of the arts: aesthetic merits, communicative qualities and contributions to the public sphere – are seldom articulated, regularly overlooked, and often undervalued. We ask the wrong questions and do not collect the right data. This does not help to account for public monies or help us understand the value of the arts. The state of Saskatchewan cultural metrics is generally weak.
Indicators have evolved from measuring the economic health of systems (how much) to social provision (how good) to understanding broader values and cultural change over time. Indicators are used to measure performance and predict systems, inform evidence-based policy, and promote democratic engagement around important local issues. Arts indicators are often confused for economic indicators (the economic performance of the arts industry) or social indicators (the social provision and access to subsidized programming). Part of this confusion is due to the vagueness of cultural policy where the terms art and culture are used almost interchangeably. This discussion paper explains that arts indicators are part of the broader category of cultural indicators: tools used to better understand public perception and social values towards particular phenomena (art) over time. The field is under-theorized, lacks interagency coordination, data lacks quality, existing frameworks are unwieldy, and many government reports on indicators sit gathering dust on shelves. To consider an arts indicator framework, this discussion paper details six approaches to arts and cultural indicators to better understand the policies and theories at play:
1. Culture as a way of life
2. Culture as a resource
3. High culture
4. Cultural vitality
5. Creative vitality
6. Cultural industries
Many practical lessons are learned through the review of the literature: from international policy in cultural industry frameworks, from economists and marketing experts on creativity indexes, to practical measures of local neighbourhood activities. Each framework brings its own set of goals, local context, and policy frameworks, and therefore are not easily compared, so this paper appraises each on their own merits. Arts indicators should be measured over time and be developed within a strong conceptual and policy framework, linked to relevant contexts.
A draft arts indicator framework is then presented for discussion combining the strengths of three approaches. The hybrid draft includes the Urban Institute’s Cultural Vitality Indicators (presence, participation and support) and Mercer’s Cultural Industries Indicator Floorplan (creation, production and reproduction, promotion and knowledge, dissemination and circulation and consumption and usage), alongside additional public value research interests from Throsby in Australia (public perception). Example indicators relevant to the Saskatchewan context are listed.
Finally, some practical advice is offered, suggesting a coordinated approach is required for the development of an arts indicator framework. Clear goals are imperative: advocacy, research, or policy, as these goals can be in conflict. Strong leadership, resources and political will are essential, therefore leaders from the arts, policy, and the private sector, alongside individuals with research capacity should be invited to the table early on. A narrow strategic range of indicators is much more powerful than a long sprawling list, but to get started, a generative process of mapping the arts sector is encouraged. Selection of the final indicators will depend on a range of factors: who is the target audience, what data is available, the regularity of reporting, and many concerns. Each of these of these considerations is included for reflection before commencement.
