The minefield of cultural measurement

First published at artsHub, 14 August 2013

Arts strategist Julianne Schultz says we need to start measuring the value of culture, an important but risky venture.

In a speech as part of the Currency House Art and Public Life series in Sydney today, Professor Schulz called on the arts sector to be more ambitious in measuring the intrinsic value of the work produced by artists.

Schultz, who chaired the Reference Group on the National Cultural Policy, said the arts should take advantage of new measuring tools.

For very good reasons we have been diffident about measuring the value of culture. It feels wrong in many ways. We have accepted as a mantra that there are some things that are so intrinsically valuable that they defy quantification. I am not unsympathetic to that view. Robert Kennedy expressed it with great eloquence many years ago, when he decried that we know measure the value of everything except that which is most valuable. In the intervening years new ways of measuring some of these things of value have been created – and that is important. In the arts and culture sector we have opted for a proxy measurement of value, the number of tickets sold, the number of visitors and their multipliers, the profitability of organisations and so on. These are important tools, but not sufficient to capture the public value that accrues from engagement in cultural activities. Thirty years ago environmental value was not something that was measured, now it is. We have to be more ambitious and smarter in finding a way to measure the public value of culture. We know it exists, but we have not yet found the right way to measure it. We have to be more ambitious in measuring the intrinsi’c value of the work produced by artists and the costs that fall disproportionately on them and their families because their work is not properly valued; the institutional in terms of a national ethos which draws visitors or inspires productive innovation; the instrumental value, like the well documented legacy for children of exposure and involvement in arts and culture to successful and engaged lives, and the commercial value which is contributing more to the national economy than many other sectors.

 

It’s not easy being an arts policy nerd. As policy fields go, the area is a lot smaller and less influential than key political battlegrounds like economics, climate change or asylum seekers. Public intellectuals with clout in the field are few in number, and there is no high profile think-tank with easy access to media outlets, like a Climate Institute or an Institute for Public Affairs. Even the vested interests are not particularly organized: industry bodies such as APRA or SPAA don’t strike quite the same fear into a minister’s heart as the Minerals Council of Australia, the Pharmacy Guild or the AI Group.

The thin and patchy nature of cultural policy debate has a number of consequences. One is that discussion tends to languish for long periods. The fitful progress of the national cultural policy towards its eventual outcome in Creative Australia is a good example. First mooted by Peter Garrett as opposition Arts spokesman in 2006, the policy was finally delivered this year.

But the threadbare nature of the cultural policy discussion can have positive consequences too. One is that prominent artists and intellectuals have an unusually strong influence. While the big debates about economic policy are fiercely contested by powerful players, and correspondingly crowded with talking heads, cultural policy is comparatively empty. The few players of significance that do take the field have unusual freedom to move.

Amongst this small coterie, one figure has reached an unquestionable position of influence: Julianne Schultz. From her seemingly peripheral position as the editor of a small but respected magazine, Schultz has spun a web of influence that places her firmly at the centre of the Australian artistic and cultural debate. A key consultant to a succession of Labor arts ministers, Schultz co-chaired the creative stream of the 2020 Summit and went on to lead the reference group for the Creative Australia policy. She’s on the board of the ABC and the Grattan Institute, chairs the Australian Film Television and Radio School, as well as wearing a haberdashery of other hats. When The Australian’s Matt Westwood profiled her last year, she described her intellectual background as ‘broadly cultural, but … from a journalism-media background.’

This breadth of interests and networks makes Schultz a voice worth listening to, especially when, as she did this morning, she advances a bold new policy agenda to build on Creative Australia.

The take-home message of Schultz’ speech this morning concerns the need for an expanded Ministry of Culture. This new super-department would bring together existing federal programs and agencies in a cabinet-level Department. The precedents are strong for such a body overseas: France’s Socialist government of the 1980s was famous for its swashbuckling culture minister, Jack Lang. As Schultz observed today:

At the moment not even all the national collecting institutions answer to the same minister, heritage is in environment, cultural diplomacy and UNESCO are in DFAT, industry assistance for the creative industries is in innovation and climate change, tourism and sport are elsewhere, trade is not linked in any consistent way, broadcasting is in broadband and the digital economy, there are programs in education and health, and regional affairs funds the building facilities and gives prizes for regional arts.

[…]

Such a department would be able to address the cultural sector as a whole, bring a fresh and critical perspective to the sustainability of the component parts with rigorous economic analysis by taking the lead on developing the tools to measure public value.  Its ethos would be sympathetic to cultural potential. It would complement not replicate agencies, like the Australia Council and Screen Australia that allocate funds – so that the arm’s length relationship between cultural production and government, which is so highly valued would be maintained.

Schultz’s proposal is both bold and sensible, and echoes my thinking on the subject; in 2010, I proposed a similarly structured portfolio.

Other aspects of Schultz’ speech are just as interesting, though they will perhaps receive less attention. One argument she makes that could potentially be a game-changer is the need for a much broader and deeper set of cultural statistics and indicators.

‘Thirty years ago environmental value was not something that was measured,’ she points out. ‘Now it is.’

‘We have to be more ambitious and smarter in finding a way to measure the public value of culture. We know it exists, but we have not yet found the right way to measure it.’

There’s no doubt that measurement drives public policy, as the long-running evidence-based policy debate inside the public service amply demonstrates. The ‘poor cousin’ status of the arts and culture within government agencies stems, in part, from the fact that it remains very hard to measure the community value of a beautiful artwork or a provocative documentary. As former top bureaucrats like Leigh Tabrett have told us, the all-powerful central agencies of government – especially Treasury and Finance – are still highly skeptical of the value of the arts and culture, seeing it as warm and fuzzy window-dressing compared to the serious stuff of roads, schools and hospitals.

Echoing an important stream of the academic debate about measuring culture, Schultz says there should be much more effort devoted to measuring the so-called ‘intrinsic’ value of the arts, for instance by using sophisticated tools from economics to measure the ‘contingent valuation’ of the arts by ordinary citizens. So, for instance, the public could be polled and asked what they would be willing to pay for a new art gallery in a regional city, or whether they’d like to spend more on public broadcasting than the ABC’s famous “eight cents a day’. When such exercises have been tried I other countries, they have consistently yielded answers in excess of current government funding levels.

Schultz also says there are a range of other measurements that could better capture the value of the arts, including instrumental value, ‘like the well documented legacy for children of exposure and involvement in arts and culture to successful and engaged lives,’ as well as economic factors, ‘which is contributing more to the national economy than many other sectors’.

It all sounds very useful, and arts advocates would no doubt love extra arguments with which to persuade skeptical razor gangs in Finance and Treasury. But by stepping into the minefield of cultural measurement, Schultz – and Australian culture in general – will need to tread carefully. The most recent attempt to develop such measures in the UK, for instance, developed detailed proposals to measure the value of the arts in Britain using contingent valuation [pdf]. Sadly, they were little help when faced with George Osborne’s austerity drive.

Not all metrics are created equal. Just yesterday, for instance, Essential Research released an opinion poll in which those surveyed said they were unhappy about Australia’s ongoing budget deficit, and would like to see cuts to arts funding to help pay for it. And if there’s one measurement every politician understands, it’s a poll.

Advertisements

The “three faces of time” in arts participation

Andries van den Broek has a really cool new paper in Cultural Trends this year. It’s entitled “Arts participation and the three faces of time: A reflection on disentangling the impact of life stage, period and socialization on arts participation, exemplified by an analysis of the US arts audience

It’s a really neat way of thinking about the temporal aspects of culture, and completely original as far as I know (though van der Broek points out that analysis of generational cohorts goes back to Comte).

Here’s a taste of his argument:

This is the history of the arts participation of a fictitious character, Pete. At the end of 2013, he’ll be 50 years of age, which implies he was born in 1963. He is not particularly keen on visual arts or theatre, though he visits the odd exhibition and performance. He is more into rock concerts, but also attends the occasional classical music concert and art house movie.

How come his cultural repertoire is like that? Is this typical of his being 50? (Do other people at the same life-stage typically display a pattern like that?) Or, is this typical of 2013? (Does it reflect what is the cultural offer that year?) Or, is it typical of someone who grew up in the 1970s? (Does it relate to a taste pattern acquired in that era?) It’s probably the case that Pete’s cultural repertoire is affected by all three (and, of course, by many other factors too). But, which aspect of Pete’s cultural repertoire can be attributed to the fact that he is 50; which aspect relates to it being 2013, and which aspect to his having grown up in the 1970s?

van der Broek goes on to do some stats on the effects of these three frames, using US data from the NEA’s Survey of Public Participation in the Arts. This allows him to tease out the differences between, say, the formative cultural experiences of generational cohorts from, say, the effects of their life-cycle in determining their participation patterns. Overall, he finds that people are not participating in as much culture as they used to, and that the composition of artforms does change.

And what is that change? One of the main ones is that fewer people are interested in classical music. Younger generations are not replacing the cohorts of classical lovers that are slowly dying.

Most importantly, though, van der Broek finds that arts participation (at least as measured by the NEA) is declining in the US. “All in all, the upshot is that the future of arts participation is not threatened by the cultural behaviour of recent (or future) as compared to earlier cohorts, but by a general decline in arts participation irrespective of cohort (and of age).”

In summary, a really interesting paper and one that I expect I will be returning to.

Australian federal budget 2011: wrap-up of arts and cultural funding

The following article appeared in Crikey on Friday May 13th 2011. 

The 2011 federal budget contained some modest announcements for the arts and culture.

In the Arts portfolio, the government delivered on its 2010 election promise for $10 million over five years in new grants for artists to create work. The funding will support “up to 150 additional artistic works, presentations and fellowships over the next five years through the New Support for the Arts program.”

As well, $400,000 has been found for the federal government’s Contemporary Music Touring Program, a successful program which supports popular mid-level contemporary music acts to tour regional areas.

In broadcasting, $12.5 million has been provided for the proverbially penurious community radio sector, an increase of 25% for a critical area of broadcasting that generally receives very little government support

There was also a package for the screen industry, with a headline figure of $66 million (as we will see, it is actually less than this). Much of the extra money goes to production subsidies through the tax system in the form of lower qualifying thresholds for the Screen Production Incentive. According to Screen Australia, the changes include:

  • Lowering the threshold for Producer Offset eligibility from $1 million to $500,000, for features, TV and online programs

  • Replacing the Producer Offset for low-budget docos with a Producer Equity payment

  • Converting the 65 episode cap to 65 commercial hours for TV

  • Exempting documentaries from the 20% above-the-line cap

  • A reduction in qualifying Australian production expenditure thresholds, and allowances for a broader range of expenses to be eligible for QAPE.

Some really good news is the restoration of the Australian Bureau of Statistics’ screen industry survey, which provided gold-standard data on the state of the industry and which hasn’t been performed since 2007-08 (shortly before the Rudd government slashed funding to the ABS in its first budget).

But how much new money for screen is really here? Go to Budget Paper 2 and you will find that the total extra funding is only $8 million. This is because, quoting from the budget papers, “these changes will be partly offset by $48 million in savings over four years from 2011-12 by removing the Goods and Services Tax (GST) amounts from [qualifying production expenditure] for the film tax offsets and increasing the minimum expenditure thresholds for documentaries to $500,000 in production (from the current threshold of $250,000).”

Money is also being clawed back from cultural agencies through the increased efficiency dividend. Rising to 1.5% in future years, the efficiency dividend hits smaller agencies much harder than big ones. And everything in the arts is small.

The efficiency dividend measures mean the Australia Council is being asked to save $3.3 million over the forward estimates, the Australian Film Television and Radio School will have to find $1 million, the National Film and Sound Archive $1.1 million, the National Gallery $1.4 million, the National Library $2.1 million, the National Museum $1.7, and Screen Australia $759,000. That’s more than $12 million in funding cuts for cultural agencies over the forward estimates.

If we look a little closer at the portfolio budget statements, for instance from the Australia Council, we can see the effects of the efficiency dividend in falling support for artists and cultural organisations. This year there will be “a decrease of approximately $2.5 million in forecast grants expenses compared with 2010-11.” Australia Council grants funding will be only 2% above 2010 levels in 2014-15. But CPI is forecast to run at 3% annually, meaning Australia Council support for artists and organisations will fall in real terms — by perhaps as much as 10%.

In other words, the “New Funding for the Arts” money announced in this budget will be almost completely clawed back by the effects of static funding and the increased efficiency dividend on the Australia Council.

The one really big-ticket spending item in culture was of dubious policy value: the $376 million spend on helping pensioners and senior Australians to make the switch to digital TV. Opposition leader Tony Abbott has already pilloried the program as “Building the Entertainment Revolution”, while our own Bernard Keane and Glenn Dyer have pointed out “the political imperative of ensuring pensioners aren’t left without television as analog signals switch off”.

Personally, I’m sympathetic to the argument that television represents an important human service that allows older Australians to stay connected with the broader community. But the spending program should also be seen in the context of the broader budget, in which $211 million in spending is being “saved” from aged care itself. The government appears to be prioritising access to daytime television over places in aged-care facilities.

Money for art and culture is often spuriously disparaged by critics as diverting resources away from the critical services that governments provide. In reality, of course, the numbers are tiny compared to the investments annually in roads, schools and hospitals. But in this case it really does seem as though the owners of television networks are getting a subsidy at the expense of much-needed investment in aged care infrastructure.


The diffusion of the printing press in Europe, 1450-1500

These maps are just too pretty not to re-post. They come from Jeremiah Dittmar’s fascinating new paper, Information Technology and Economic Change: The Impact of the Printing Press.

The diffusion of the printing press, 1450-1500. Source: Jeremiah Dittmar.

There’s a good summary of the paper at Vox, but the take-home message is probably in two parts. Firstly:

  • First, the printing press was an urban technology, producing for urban consumers.
  • Second, cities were seedbeds for economic ideas and social groups that drove the emergence of modern growth.
  • Third, city sizes were historically important indicators of economic prosperity, and broad-based city growth was associated with macroeconomic growth (Bairoch 1988, Acemoglu et al. 2005).

And secondly:

I find that cities in which printing presses were established 1450-1500 had no prior growth advantage, but subsequently grew far faster than similar cities without printing presses. My work uses a difference-in-differences estimation strategy to document the association between printing and city growth. The estimates suggest early adoption of the printing press was associated with a population growth advantage of 21 percentage points 1500-1600, when mean city growth was 30 percentage points. The difference-in-differences model shows that cities that adopted the printing press in the late 1400s had no prior growth advantage, but grew at least 35 percentage points more than similar non-adopting cities from 1500 to 1600.

Hollywood’s institutionalised sexism

Several media outlets including The Wrap have carried articles today about the “celluloid ceiling” – the long-term institutionalised sexism of Hollywood’s motion picture industry.

The data is sourced from Martha Lauzen’s recent report “The Celluloid Ceiling:Behind-the-Scenes Employment of Women on the Top 250 Films of 2009”. According to Lauzen, the percentage of women involved as directorss, producers, cinematographers and associated roles is low and declining:

In 2009, women comprised 16% of all directors, executive producers, producers, writers, cinematographers, and editors working on the top 250 domestic grossing films.  This represents a decline of 3 percentage points from 2001

The graph below says it all. As you can see, men still dominate Hollywood.

Historical Comparison of Percentages of Women Employed in Key Behind-theScenes Roles, 1998-2009. Source: Martha Lauzen

The Wrap carried an interview with high-grossing director Catherine Hardwicke, of Twilight fame.

Catherine Hardwicke …  directed 2008’s “Twilight,” the first in the hugely successful vampire franchise, but Hardwicke told The Wrap she was prevented from even pitching to direct “The Fighter.”

“I couldn’t get an interview even though my last movie made $400 million,” she said to The Wrap. “I was told it had to be directed by a man — am I crazy?” said Hardwicke, who also noted she liked what David O. Russell did on the film. “It’s about action, it’s about boxing, so a man has to direct it … But they’ll let a man direct “Sex in the City” or any girly movie you’ve ever heard of.”

 

Will Dawson maps #twitdef

Axel Bruns lab at QUT is really starting to turn out some quality research – as evidenced by the summer vacation project (!) of Will Dawson.

Dawson has mapped the posts, retweets and network connections of the #twitdef conversation. And some very pretty data he presents. Here, for instance, is a bar graph on the 15 most re-tweeted Twitter identities using the #twitdef hashtag:

The fifteen most retweeted users during the #twitdef controversy. Source: Will Dawson / Mapping Online Publics

Interstingly, the top tweeters and the most re-tweeted are quite different data sets. Dawson concludes that:

The types of users being retweeted also differ significantly from the most prolific tweeters in terms of plain numbers. Of the top 15 retweeted users, eight are journalists, four are  academics (including Posetti), two are regular citizens, and only one is the account of an official media outlet (@crikey_news, in yellow). Of the seven journalists, only one works for The Australian – Caroline Overington (@overingtonc).  The lack of prolific tweeters (in terms of numbers) in this list supports one of the more long-standing theories of twitter – that just because someone is tweeting a lot, doesn’t mean people are actually listening to them.

Why culture is bigger than the arts: Rebutting Christopher Madden part 2

The main thrust of Christopher Madden’s recent article in response to my criticism of the Australia Council can be summarised by his title, “An arts council by any other name.”

To quote Madden:

Eltham calls for the Australia Council to be ‘abolished’. Yet the agency described in his ‘new model’ – what I will refer to as a ‘culture council’ – looks suspiciously like the Australia Council with a new name and a different focus. The new agency is not described in detail. Is it at arm’s length like the Australia Council? Would it support all of culture, from Facebook to historic places? What powers would it have? How would its ‘radically strengthened and diversified’ peer review processes work? Whatever the details, I would caution that an arts council by any other name would smell as sweet.

Today, an online survey popped into the inbox of my partner Sarah-Jane which, I think, proves exactly why what I’m calling for is not “an arts council by any other name”. It’s a survey of emerging artists who have applied for or been funded by the Australia Council in the past twelve months. Sarah-Jane is a film-maker: a director and producer who has received funding through OzCo’s Story of the Future initiative (an initiative which has since been discontinued). And yet, what are the types of artforms that the survey asks about? Here they are:

A7i. Which types of artistic work have you been practising in over the last 12 months?
If you don’t think these options describe your work, please select the best options and add a comment in the box below.
Please click all that apply
Music
Theatre
Dance
Writing
Visual Arts
Crafts
Hybrid Arts
Community Arts
Other

Notice how there are no film related categories here? That’s because the Australia Council doesn’t fund film-makers – Screen Australia does. Nothing wrong with that.

But what is an issue is the result of that distinction: the fact that OzCo only collects statistics about the things it already funds. We saw exactly the same issue in the Australia Council’s recent research about artists occupations.

You can see the problem here. If OzCo doesn’t fund it, the Australia Council does not consider it an “artform”. The reason for this is not any coherent theory of cultural practice – which is probably just as well, because just exactly what makes jewellery more artistic than cinematography anyway? No, the distinction is all about bureaucratic demarcation.

The result is that things that OzCo doesn’t fund, like game design or film-making, are not part of the Australia Council’s statistics, its evidence base, or its policy advocacy. This alone demonstrates why Madden is wrong: a different council, by another name and with a different ambit of cultural practice, would be a very different organisation, with very different policy imperatives.

How to be a data journalist

On the Guardian’s amazing Data Blog, a post about the exploding field of data journalism.

Data journalism is huge. I don’t mean ‘huge’ as in fashionable – although it has become that in recent months – but ‘huge’ as in ‘incomprehensibly enormous’. It represents the convergence of a number of fields which are significant in their own right – from investigative research and statistics to design and programming. The idea of combining those skills to tell important stories is powerful – but also intimidating. Who can do all that?

The reality is that almost no one is doing all of that, but there are enough different parts of the puzzle for people to easily get involved in, and go from there.

Read the rest for the excellent primer.

 

Rebutting Christopher Madden: part 1

Recently I had a piece published Overland magazine calling for radical reform, perhaps even abolition, of the Australia Council for the Arts. This week, the Overland website carries a response by cultural policy analyst Christopher Madden.

I think Madden’s rebuttal misguided in several important respects and so today I’m going to unpick his piece item by item … but before I do that I think it’s worth saying that we agree on many things. More than that, I welcome this debate – it’s exactly what I hoped to provoke with the piece. Madden’s response to my article is robust, informed, detailed and well-intentioned. It’s also, I think, quite wrong. Continue reading

Is employment in the Australian cultural industries falling? John Black can’t tell us

In today’s Australian Financial Review, former Labor Senator John Black has an interesting opinion piece about Australian unemployment trends since 2008.

Black’s research company, Australian Development Strategies, has undertaken some economic modelling on the issue, published in a web paper entitled Australian Jobs Profile for 2010.

In the paper, repeated in his column for the AFR, Black makes the startling claim that:

The industry with the biggest loss in jobs – 25,000 – since November 2007 has been the media – which includes publishers, the music industry, television, the internet, web search providers, ISPs, data processors, telecommunications workers and librarians. These skilled jobs of particular interest to younger Australians have fallen by 11 percent since November 2007, despite the National Broadband Network.
I’m often in violent disagreement with Black’s political analysis, but, if true, his article uncovers an interesting point about unemployment in the Australian media and cultural industries that the gold standard Australian Bureau of Statistics data can’t capture (because it is based on Census data, held only every five years).
Unfortunately, because of the opaque nature of the report, it’s almost impossible to determine where Black has derived his figures from. There’s no methodology section to the report, and about the most detail that can be discovered is the following, buried in a paragraph on page page 3:
This paper looks at the comparison of original or raw monthly unemployment rates in 69 Labour Force regions, across Australia, and uses simple modelling to benchmark these percentage figures against our Elaborate database.
But there is no description of the Elaborate database, so we can’t really tell. It’s the opposite of rigorous. This survey tells us nothing meaningful about employment in the Australian cultural industries.
Conclusion? Australian Database Strategies might enjoy a high media profile thanks to former Senator John Black, but that doesn’t mean we should take factoids like this too seriously.