Open data cutting edge? Like top hats, Christmas trees and giant factories the Victorians got there first.
They may not have built a chimney sweep Google death map. But their approach was similar. Collect the data. Publish it. Draw conclusions. Argue for change.
Don’t believe me?
Look at Florence Nightingale in her funny lace bonnet. Historian Dr Stephen Holliday in BBC History Magazine August 2010 writes about how she used statistics to
revolutionise the care of soldiers in the Crimean War.
By using statistics – data – she painted a picture to show a revolution in care was needed.
“When she reached Scutari the base for casualties from the Crimea,” Halliday writes, “Florence calculated that deaths from disease were seven times those arising in battle and used the campaign to campaign for better food, hygeine and clothing for the troops.”
Battered by the force of Florence’s figures and cutting edge reporting that forged the reputation of The Times the British government was forced into changes.
After the war Nightingale used her Royal connections coupled with arguments based on charts and tables to press for better standards for soldiers who even in peacetime had death rates double that of civilians.
The result? Death rates fell by 75 per cent.
Florence herself said that statistics were “the cipher by way we may read the hand of God.”
We may have lost that religious zeal but it’s an argument Tim Berners-Lee would recognise as a modern-day Florence Nightingale with a passion for data.
Did she get it right all the time?
No. Here’s the warning from history.
By misreading available data Florence Nightingale later helped kill thousands of people.
How?
She used statistics to wrongly argue cholera was an airborne disease. It wasn’t.
It took London GP Dr John Snow to collect his own data on death rates in his patch to argue they were caused by a contaminated water supplies.
So what’s the message to today’s open data pioneers?
That first data visualisation you have in front of you may not be the whole picture.
There may be more to it.
Remember the phrase ‘lies, damn lies and official statistics?’
Statistics were once hailed as the magic cure-all that revealed a hidden truth.
It’s been said that all data in some form or other is political. Let’s not see open data similarly tainted.
LINKS
Florence Nightingale –http://www.florence-nightingale.co.uk/cms/
BBC History Magazine August2010 http://www.bbchistorymagazine.com/issue/august-2010
Creative Commons:
Crimean War data visualisation: Wikipedia.
Cholera map: Wikipedia
Tim Berners-Lee: Paul Clarke via Wikipedia
Good piece, but isn’t the lesson about data per se, rather than open data? In the example you cite, the GP eventually collected his own statistics and proved Nightingale wrong. If her data, not just conclusions, had been open perhaps through “wisdom of the crowd”, or at least the wisdom of one member of it, this would have come to light sooner.
I agree with the general point about the danger of statistics, and you’re right to pose the question of how we can get around this issue.
Hi Dan
Good and interesting post. You do raise some important issues. Of course as John says all data can be misinterpreted.
I’m sure we all know of cases where local authorities talked up their positive performance indicators and talked down poor performance when a more objective observer might have represented the position differently.
And we all know of cases where journalists honed in on a single piece of data because it looked more interesting than the whole dataset.
It does make me think that an additional skill for the 2011 press officer is a good grounding in statistics and analysis. And indeed for everyone…
Very interesting post, which touches on concerns I have with Open Data in the most general sense. (I’m an advocate of Open Data, by the way.)
The issue is that statistics is not an easy subject to master, and I’m not sure that developers working from their living rooms building mobile phone applications necessarily understand it. In fact, I’d go as far as to say that hardly any of them understand it. (I did Maths and Further Maths ‘A’ levels and Specials in my youth, and statistics was the one area that I never really mastered.)
Building pretty maps from all this geo-tagged data is fine, but those people who are seeking out trends, correlating open data from multiple sources, or other non-trivial exercises need to make known their credentials if we are to take their output seriously.