Twitter made the 140 character status update the norm in internet social networking and has since catalogued over half a trillion of these babies. Holy big data, Batman! With all that conversation happening in the little birdy’s network, it was only a matter of time before it became necessary to index all of that chatter in a searchable historical database. And now it’s been done.
For the majority of the platform’s existence, Twitter’s engineers have focused on organizing real-time output, which is generally about a week’s worth of tweets. That was the whole point of the platform anyway. In 2012 they supplemented this cache with around 2 billion “top tweets” from historical data. They expanded on this in 2013 and people made great use of historical tweets like President Obama’s “Four more years” post after his election.
Now, after a great deal of time in the underbelly of the Twitter processing machine, their engineers (which we imagine to be little mechanical dwarves tinkering away amongst cogs and gears in the big blue machine) have finally succeeded in incorporating the full archive of historical tweetage. It’s more than 100 times larger than their live index and gets several billion new entries each week.
In the age of big data and massive repositories of public information, this may seem like a drop in the bucket, but as a catalogue of global conversation, a searchable database for the world’s tweets may prove to be very useful in decades to come. You can already go back and read the scattered conversation and commentary on hashtags like #Election2012, #Ferguson, and #OccupyWallStreet.
Imagine years to come when future anthropologists are studying the peoples of the early 21st Century. They’ll have a wealth of information at their fingertips as to what we thought about #twerking, #jerseyshore, #girlproblems, #justinbieber, and #thuglife.
Just be careful when you tag your posts. When Blackberry’s corporate face Research In Motion (RIM) decided to use Twitter for recruiting purposes, they grabbed the under-utilized hashtag #rimjobs. Needless to say, it didn’t have the effect they were hoping for. Such is #twitterlife.
Erving
Incredible Marketing
