Some disconnected thoughts on journalism in an age of data abundance

August 29, 2009 — 1 Comment

Slate has a great article up about whether quality journalism is the best way to sustain democracy that is in line with some thoughts I’ve been having recently. The following is a collection of thoughts regarding journalism in an age of data abundance. (I’m reworking this into a longer article, feedback, thoughts and corrections are appreciated.)

Journalism and Data Abundance

With the long-needed changes of freedom of information laws in Australia starting to come online, it’s worth thinking about how these changes might affect the way we practice journalism and politics.


Journalism’s base narrative has long been built around Watergate. It’s a highly romantic story, and one that changed politics in America: two plucky young reporters working with a secret informant to reveal dirty tricks by the US President, culminating in the only resignation of a US President in that nation’s history. Its strength still resonates in Western journalism, with Australian journalists regularly referring to minor political scandals as (scandal)-gate.

However, Watergate must be viewed as an artifact of its time. It happened in a period when freedom of information laws and governmental disclosure were far less formed. It happened in a time before widespread computerisation and data analysis, and it happened in a time before the rise of hyper-empowered politics . It also happened in a time when politicians felt beholden to the moral force of public opprobrium. By comparison, look at the way George Bush Jr reacted to worldwide condemnation of the Iraq War when his deceptions in its service were revealed.

The major difference between the 70s and today in journalistic practice – besides ratcheting up institutional pressure to produce large amounts of copy for less cost – is that information has gone from scarce to plentiful. The changes to FOI law in Australia and the widespread adoption of social media platforms promise to turn up the volume even further. How then do we deal with that?


The first is to rethink the narrativisation we as journalists employ. Narrative is normal and natural, and it is hard to read journalism without it. But narrative must arise from facts, not the other way around. Fitting facts to narrative is the easiest way to fall into fallacious thinking.

The worst kind of narrative is the he-said, she-said form of journalism. While this is an easy target – the case against fake balance in journalism has been made more eloquently, many, many times – it’s still an issue. Up to 80% of journalistic content is PR driven – in large part because PR people understand how to tell stories.

To get beyond this point, journalists need to accept their much-resisted reliance on PR has as much to do with their desire for narrative as it is their need for data.

At Radio National’s Future of Investigative Journalism panel a PR person introduced herself as a PR agent, and stated that she ‘tells stories for a living’, leading to laughter from the journalists in the audience. The look of confusion on her face was indicative: if a room full of journalists can’t understand the basic function of PR, what hope is there for public discourse?

I regularly see award-winning journalists using Twitter to ask

‘I need (age)(ethnicity)(gender) who has (experience) for a story about (issue of the moment)’

This is worrying on a number of levels. They’ve already set up the narrative of the story – (x) people from (y) who’ve experienced (z) compare perspectives. They’re not allowing the facts to emerge from research, but have essentially boiled their journalism down to online vox-poppery. While there’s a possibility such an approach could draw some interesting information if run on a large scale, limiting it to such a small sample size makes the results essentially useless.

Gatewatching and Data Analysis

More disconcertingly, they are missing out on some of the most interesting aspects of using Twitter for journalism. Andrew Sullivan’s curation of information coming out of Iran (or gatewatching, as my colleague Axel Bruns called it long before the recent hype around social journalism) is an example of how journalism can be done in an age of overwhelming information.

When I wrote my honours thesis I noted that all of the information necessary to understand Enron’s situation was publicly available but no-one had done the analysis necessary to understand it.

In the years since, we’ve seen some great examples of data-driven journalism come to the fore. Possum’s Pollytics and the Poll Bludger in Australia, the New York Times VizLab and Everyblock are great examples of what can be done. We’ve also seen a growth in the understanding of data accessibility’s role in public discourse with the popularity of Open Australia, a front-end and search interface for Hansard that is more usable than the Australian Government’s ParlInfo service.

The changes to FOI in Australia are going to result in large amounts of data being released to the public, and we will need journalists, statisticians, programmers, economists and psephologists to make sense of it. It’s simultaneously disconcerting and encouraging that people traditionally outside of journalism are driving developments in this field. When The Australian went up against the psephologists during the last election, the discomfort that the controllers of political discourse in Australia felt was palpable, and it illustrated an important point – it’s easier to teach a statistician to write than to teach a writer to understand statistics.

If the major news publications in Australia end up putting their vox pops, rewritten press releases and partisan fantasies behind a paywall and their sole contribution to public discourse is a blog aggregator that publishes political position statements and press releases as journalism and vacuous hit pieces on academic analysis, it’s hard to see that the death of newspapers will mean the death of quality journalism.

One response to Some disconnected thoughts on journalism in an age of data abundance


    I’d argue that Watergate had more impact thanks to the Cold War. America’s emphasis on the lack of personal freedoms – privacy – in the Soviet Union pretty much paved the way for Woodward and Bernstein. Given the need to vilify the Soviets – the administration left themselves wide open for a scandal to have that level of impact.

    A mistake they have not repeated.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s