* THIS INTERVIEW WITH WLL FIRST APPEARED IN ‘THE DIRECTOR’ *

William Higham of future-gazing consultancy Next Big Thing believes open data’s relevance today is part of a broader shift towards a more candid world: one in which any kind of subterfuge, obfuscation or furtiveness has become deeply unfashionable. …

Open data may sound like a dry, tech-heavy concept, but it is an exceptionally powerful philosophical movement that looks set to transform your life, business and, perhaps, the world for good. The experts tell Director why.

In 1522, Luther’s translation of the Bible from Hebrew and Greek into German changed Christianity – and the world – forever. With scripture accessible to ordinary believers, rather than just ecclesiastical scholars, there were suddenly informed arguments about how to interpret it. Mass peasant rebellion broke out across Europe, and the resulting ideological division ultimately led to the modern secular, centralised state familiar to the world today.

Approaching half a millennium later, in 2007, a small team of astrophysicists from the UK and the US asked amateur astronomy enthusiasts to help them classify about a million images of galaxies, photographed by a telescope at the Apache Point Observatory in New Mexico, as being either spiral or elliptical, smooth or bulging. Having estimated it would take the public three to five years to finish the job, the team was astounded when it took just three weeks. The project spawned a citizen science project (zooniverse.org), which is now aggregating humankind’s pool of knowledge – and therefore improving its collective understanding – in areas as diverse as climatology, biology and physics.

Both of the above are examples of the awesome power of democratised knowledge. And, as we progress into the second quarter-century since the invention of the world wide web, the time has arrived, according to the entire open data movement, to apply this principle not only to the business landscape, but also to pretty much every problem the planet faces. “I just cannot think of an area of life in which an open approach wouldn’t provide benefit,” says Gavin Starks, chief executive of Britain’s Open Data Institute – a non-profit and non-partisan organisation, founded by the inventor of the web, Sir Tim Berners-Lee, and Southampton University’s Professor Nigel Shadbolt, which helps organisations to unlock the supply of knowledge to address local and global issues, and encourage its dissemination.

“How are we going to deal with a global population of seven billion-plus people? With having to transport, feed, house and provide power for that scale of population in an equitable way?” Starks continues. “How do we get an extra million people into London each decade? How can we find the next penicillin? Improve engine design? Create more efficient solar powers, better transport management systems, better energy, waste and water infrastructures, better healthcare, education? It touches on everything. Wouldn’t it be great if our policymakers were able to make decisions based on the full picture, instead of a possibly out-of-date snapshot?”

An end to secrecy

The basic principle behind the open-data movement is that aggregated information is far more insightful, relevant and powerful than the sum total of data grouped in isolated pools. The world its advocates envisage is one in which companies ditch the secrecy inherent in our currently accepted notions of competitive enterprise; in which car manufacturers share their research on carbon emissions and driverless technology, and devote their resources instead to quality, design and service; one in which energy supply and consumption data is considered in conjunction with meteorological trend information in order to improve efficiency and help Britain reach its target of reducing greenhouse gas emissions by 80 per cent by 2050.

It’s a social utopia in which census, health centre and drug prescription data are combined to tackle the projected reality whereby the number of elderly people with disabilities will increase by 32 per cent by 2022. In short, it’s a world in which all the information generated by governments, businesses and individuals is routinely published as standard procedure, offering not only solutions to existing problems but entrepreneurial possibilities we can barely imagine now. “We have big ambitions,” says Starks of his organisation’s raison d’être. “What we’re after is a transition to open information as pretty much the only way we’re going to solve some of the problems we face as a society. We currently spend more than £500m a year digging up the roads in London. We could save hundreds of millions of pounds if we did it more effectively.”

Starks also points out the snowball effect inherent in the basic principle of open data. “If [shared knowledge and insight] helps people innovate around building energy efficiency, we can improve people’s living circumstances. If people are in well-conditioned homes they’re less likely to get ill which means lower healthcare costs and a higher rate of employment. It’s about opening all the silos of data, so you can look across entire systems.”

But to call open data an emerging trend would be inaccurate: its potential has already reached beyond the worldly. “Virgin Galactic is able to launch about 30 satellites a day,” says Starks, “and it’s likely we’ll have Google Earth, but in real time, within the next 10 years. Imagine the potential for that in terms of mapping forests and glaciers, transport, shipping? That shift in global awareness will create massive shifts in business.”

Businesses cashing in

Some wily entrepreneurs have spotted open data’s commercial potential and formed businesses based on its inevitable fruition. One example is Spend Network which, in the words of founder Ian Makgill, “takes all of the published government spending data and puts it into one place, then links it to company records. Every time the government makes a transaction to another part of government or to a charity or company – whether that’s Capita or Serco or the sandwich shop down the road – they have to publish
that data.”

The information Makgill’s enterprise collates and processes is already out there and available to anyone: however, the vast majority of organisations which could benefit from the mass of data available simply don’t have the resources to digest, process and evaluate it. “Because the data’s littered across the web in thousands of comma-separated value (CSV) files, it’s not possible to understand the totality of it, so we gather it all together and make sense of it,” says Makgill. “It’s for the benefit of suppliers asking, ‘Where’s a good place for me to direct my limited sales resources?’ They can see where their face fits.”

But, for Makgill, barriers to the free flow and quality of data come from a variety of sources: from inherent secrecy, to it simply being no one’s job in most organisations to share information, to basic human error. Then there are legislative flaws. “The Freedom of Information Act is a fairly blunt stick,” he says. “We took the Ministry of Justice all the way to ICO [Information Commissioner’s Office] judgement to get their spending data, and it took 13 months. We’re prepared to use that to look after our customers, but obviously we’d like to see a more fast-tracked approach.”

Meanwhile Jeni Tennison, founder of Open Addresses UK, is on a mission to build, from the ground up, a new database of every address in Britain – to supersede the existing ones owned by Royal Mail and Ordnance Survey. “There’s currently a great, big, fuzzy grey area, legally, around what you can and can’t take from an existing database,” says Tennison, who is also technical director of the Open Data Institute. “Getting some clarity around what is and isn’t intellectual property violation in existing databases would be extremely helpful. You err on the side of risk aversion, especially when you’re a start-up and don’t have much money to fend off legal action. That means you end up not doing things which might be absolutely fine.”

Perhaps one of the most ambitious open-data start-ups in the UK, though, is opensensors.io – an open-source platform which is marrying the increasingly ubiquitous Internet of Things [IoT] with the burgeoning potential of real-time data exchange. The company has built an IoT messaging engine which processes millions of messages from various connected devices per second. The software can, via subscribers’ smartphones, send signals to the thermostats, lights, locks and other utilities within buildings. As well as smart buildings, the company also specialises in automated car parking services and environmental sensing projects, which aggregate water quality, weather and pollution data. “Sensors in parking bays tell people not only where there’s a space, but also the local authority’s useful usage data,” says the company’s founder, Yodit Stanton. “Combine it with the air-quality data and you start getting an interesting view of the world.”

Something of great benefit to all these start-ups, according to Starks, is the fact that the relevance of accumulated information in an open-data scenario doesn’t rest on it being complete. “When you open data you can also infer the negative space,” he says. “So if everyone else is opening up their information in a supply chain and somebody says, ‘I’m not’, it doesn’t matter because you can infer it from everyone else’s.”

The UK’s advantage

But perhaps the most exciting aspect of the open-data movement is the fact that the world is so primed for it. Data – like water and air – has a natural inclination to run free, and it takes human intervention to prevent it from doing so. And the impetus to make that intervention is evidently dwindling. On Barack Obama’s first official day in office, he declared “in the face of doubt, openness prevails” and in May last year he enacted legislation requiring federal agencies to publish their spending data.

The UK is a world leader in the field – a recent report by the Washington thinktank Center for Data Innovation deemed it the nation most committed to adhering to aspects of a G8 Open Data Charter, drawn up at a summit in Northern Ireland in June 2013.

Asked why, Starks replies with no hesitation: “Leadership,” he says. “We had a lot of good leadership under the Labour government, and that’s been followed upon and built on [by the coalition]. The GDS [Government Digital Service] team has done a huge amount of work in helping us to change mindsets.” Starks adds that the arrival of a generation of digital natives on the decision-making scene can only improve the situation.

Meanwhile, the announcement in April by Microsoft Azure’s chief technology officer Mark Russinovich that it is “definitely possible” the software giant could one day make Windows open source was surely a milestone in the international open-data saga, while the UK’s domestic open-data movement was given a massive boost with the appointment of a chief data officer, Mike Brackens, to the Cabinet Office, in March. “The World Bank and the UN now have data revolution programmes,” says Starks. “The former has been working with [company database organisation] OpenCorporates to track where aid money flows into Africa. Around 89 governments now have agreed, as part of the open-government partnership, to follow the G8 Open Data Charter, which committed countries to open up information about extractors, land use, crime data and so on.”

It helps, according to Starks, that open data is an apolitical issue. “It’s useful to absolutely anyone interested in reform, whether they’re left-wing, right-wing, centre, Tea Party, Ukip,” he says. “Everyone has an interest in transparency to meet a particular reform agenda. The idea of public-sector data is a bit of a misnomer, it’s just public data. The public paid for it. It should be open.”

Starks admits that there have to be exceptions when it comes to what data is available to all – “We’re not asking for personal and confidential company data to be opened up, and obviously national security is important” – but believes that the question of what is classified as “confidential” needs re-examining.

William Higham of future-gazing consultancy Next Big Thing believes open data’s relevance today is part of a broader shift towards a more candid world: one in which any kind of subterfuge, obfuscation or furtiveness has become deeply unfashionable. “We’re in the part of a repeating cycle that is moving away from privacy, towards openness now,” he says. “Of course there are some negatives and dangers, but the younger generation grew up with access to information and also with reality TV, scandal, camera phones – it’s very hard to keep anything secret now. So they’re not valuing the concept of privacy so much. Brands need to see this.”

As far as Starks is concerned, leaders of companies of all sizes need to start thinking about implementing an open-data policy of some kind. “If not,” he warns, “someone else will use it to try to eat your lunch, because that’s the nature of competition. You can mitigate that by engaging with the open ecosystem. We run workshops with directors who want to know how to start thinking about it. This is not some technocratic utopian vision of the future – it’s a case of having much better tools to do the jobs we need to do. It’s still humans making decisions. This is like the mid-1990s with the web all over again – it’s going to be really, really important, including for reasons I can’t tell you quite why yet…”

Reason enough, surely, to bring an end to what might accurately be described as collective voluntary myopia.

Article by Nick Scott, The Director (incl. quotes from William Higham)
View original article: http://www.director.co.uk/open-data-technology-22-may-2015/