Is the deluge of data good for government?

This article was first published by Total Politics.

Open data is becoming big news and, potentially, big business. Policy wonks and digital democracy campaigners have for years expounded its virtues in a variety of conferences, hack events and gov camps. It’s seen as a driver both of accountability and of public service innovation, but has remained a rather niche interest.

Over the past couple of weeks, however, a series of news reports have underlined the growing importance of open data to the mainstream contemporary political conversation but have also revealed some ambiguities in our attitude towards it. This week the Wellcome Trust announced that it would require all scientists it funds to publish their work in open source publications and not in closed journals. Meanwhile The Economist reported that the prime minister was planning to publish anonymised NHS data to fuel scientific innovation. The same article reported that Britain already publishes more official data than any other country except for the US.

In local government we’ve already seen elements of this agenda, most notably with the requirement that councils publish all spending over £500.

Local government has also borne the brunt of an increase in Freedom of Information (FOI) requests. A report by the UCL Constitution Unit last year found that requests to local government had increased fourfold from 60,000 in 2005 to nearly 200,000 in 2010. In comparison, central government received just 25,000 in 2005 rising 40,000 in 2009.

Newsnight report last week revealed government plans to start charging people for FOI requests and we know from Tony Blair’s memoirs that he came to bitterly regret introducing the Freedom of Information Act.

These are telling indicators of our anxieties about data; we like the idea of it, we recognise the potential of crowd sourcing to drive new thinking, we like the idea of transparency and accountability, but we worry about the quality and quantity of it. In an information-saturated world we worry about the unwieldability of open data and its nuisance value. In short we want data but only as long as it’s purposeful and manageable.

In terms of holding government and other public bodies to account it’s not clear that open data has yet fulfilled this potential. The Constitution Unit report concluded that the vast majority of FOI requests were about issues of private not public interest and despite the best efforts of websites like OpenlyLocal however, we’ve yet to see the army of armchair auditors the government hoped would replace, at least in part, the inspection regimes of pervious adminisrations.

We’ve seen a far greater take up of data to drive new approaches to services, everything from apps for bus timetable, or Boris bike availability, to websites like Fixmystreet use official data to allow citizens to access services in more convenient, innovative ways. Sometimes these are done for profit, other times just for public benefit, but almost invariably this innovation has been driven from outside government.

What should we conclude from this? It’s tempting to think that we should start to see data as a driver of service innovation and not worry so much about its transparency function and that government, local and national should concentrate its resources on the release of data that lends itself to useful purposes. Perhaps we should not get too hung up on usefulness however. Partly because it is so difficult to prejudge what will be useful. Predominantly, because there is a broader cultural issue at stake.

True transparency is not just about the usefulness or not of day-to-day transactions but about establishing a different way of thinking and operating that is open and collaborative, that shares information widely and draws in views from all parts of the community. This is an essential precondition of a reinvigorated local democracy, but it cannot be achieved by calculating the utility of each unit of information, only by building interaction and connection into our default setting.