The Known Universe

Amazon just went nuclear on the public API front, making over a terrabyte worth of information available to developers through a searchable API rivaling Google. Everything from US census information and labour statistics to the entire English section of Wikipedia in machine readable format.

What kind of programs could you write having access to publicly available genomes and the entire chemistry library for the National Center for Biotechnology Information? With thousands of computers sitting around just waiting to crunch numbers for you through Amazon’s EC2 service, we’ll have self aware robots with Stephen Hawking’s genome and the entire encyclopedic knowledge of Wikipedia knocking at our doors in no time.

The only problem I can see from this set up so far is that these are not dynamic data sets but merely snapshots (albeit huge snapshots) from a point in time and Amazon is still unclear as to how often they plan to update them. Labor statistics are all fine and dandy if they’re up to date, otherwise you’re really just doing a history project.

The amount of data on the web these days is enormous and having it all organized through one standardized interface is a powerful tool. It’s only a matter of time before we start seeing more sites like this pop up.