Wired: The End of Theory: The Data Deluge Makes the Scientific Method Obsolete I'd recommend everyone read the whole article. Here's how it starts: Sixty years ago, digital computers made information readable. Twenty years ago, the Internet made it reachable. Ten years ago, the first search engine crawlers made it a single database. Now Google and like-minded companies are sifting through the most measured age in history, treating this massive corpus as a laboratory of the human condition. They are the children of the Petabyte Age.
The Petabyte Age? Dang. But then I thought about it, and it's fair enough -- I probably "read" 200 news stories a day. I go through all the major dailies in Texas, the major national papers, and then about thirty blogs on the state and national level. The article then goes on to argue that Google -- and their new philosophy -- is largely responsible for the phenomenon: For instance, Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day. And Google was right. Google's founding philosophy is that we don't know why this page is better than that one: If the statistics of incoming links say it is, that's good enough.
And here's the crux of the argument: that because we can now amass such large statistical samples, we can use probability and algorithms to mathematically predict outcomes -- instead of creating theories to scientifically guess what the outcomes will be. Therefore, all we need are strong and large enough computer databases and programs to crunch the data. The difference is subtle, but critically important. Learning to use a "computer" of this scale may be challenging. But the opportunity is great: The new availability of huge amounts of data, along with the statistical tools to crunch these numbers, offers a whole new way of understanding the world. Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.
Pretty interesting stuff. What do you think? |
