Research Twitter, Predict the Future

Twitter, for many critics, is seen as the platform on which a lot ‘irrelevant’ information, whether personal or not, is shared. Arguments of the presupposed irrelevance of Twitter are paralleled by critics who argue the possible predictive potential of Web 2.0 and social media in particular. My motivation of writing this post is that social media which is the product of Web 2.0 already is and will increasingly be the tool of choice for disseminating information and the sharing of interpersonal communication. [1] It is therefore important to search for the relevance of this presupposed irrelevant chatter.

Social media platforms are adopted rapidly by a diverse group of people. From politicians to businessmen, in another context friends and family, and ranging from a broad bandwidth of ages these people are disseminating cartloads of opinions, feelings, emotions and their current whereabouts.[2] Studies have shown that Twitter’s call ‘what am I doing?’ in a rhetorical way is bypassed by far more diverse ‘personal’ information. By this insight Cheong et al. draw the conclusion that Twitter is ‘a hybridization between conventional blogging and an online social network’.[3] By the openness of the Twitter platform this means this rich information is free but valuable noticeable by the ‘Trending Topics’ option which gives you insight in the top 10 mentioned terms on Twitter. This service gives you an insight of the so called ‘zeitgeist’ of the Twitter community in reflecting on the daily news and experienced lives. This ‘zeitgeist’ makes the Twitter relevance all the more apparent but still is ‘just’ a mirror of the present sentiments of the offline world aggregated online. Not to downgrade this intelligence but by virtue of the rhetoric question ‘what am I doing?’ maybe I can surprise you and write about how your ‘Tweeting’ is used for predicting real world outcomes aka the future.

Already Gruhl et al. discovered the ‘predictive power of online chatter’ by researching the amount of mentions of book titles in the blogosphere and the correlating amount of sold books in real-world Amazon.com.[4] Asur et al. did a comparable research with the chatter of Twitter.com in forecasting the box offices revenues for movies. Their research outcomes seemed to be so promising that they claim ‘that a simple model built from the rate at which tweets are created about particular topics can outperform market-based predictors.’ [5] The methods used by the Asur can be applied for more forecasting than theater movies alone ’Moreover, gathering information on how people converse regarding particular products can be helpful when designing marketing and advertising campaigns’.[6] 

The first thing that came up my mind was that the box-office revenues are related to the amount of publicity generated for the particular movie. And indeed Asur et al. first researched all the effort that’s taken to create attention and buzz about the movies on Twitter. The next step in future prediction is studying the sentiments by using text classifiers to distinguish positively oriented tweets form negative related to the particular movies. Not surprisingly the more positive Tweets the more revenues the movies will generate, at least that’s the hypothesis.

By using the Twitter Search API, Asur et al. extracted 2.89 million tweets related 24 movies over a period of three months. Before the release of the movies media companies generate promotional information in many different forms, trailer videos, news, blogs and photo’s. [7] This gives the possibility to extract tweets related to the movies by following particular URL’s that lead to the different online content. Also retweets, the forwarded tweets, are of importance while such tweets are important because of the dissemination to the complete friend-list. The prediction of the first weekend Box-office revenues was in fact more accurate than the Hollywood Stock Exchange considered the gold standard. Asur assumes that after the release of the movies the value for the sentiments rise increasingly and by measuring the ratio of positive tweets to negative a box-office revenue can be predicted. This research have shown that by using social media research accurate forecasting’s can be done. In fact the accuracy of the Twitter research method showed an astonishing 97.3 percent.  The reason why movie revenues where subjected to research was for the simple reason that a comparison can be done with other methods. This means there are many ways other topics this methods can be used on for instance designing marketing and advertising campaigns. Therefore the researchers claim’At a deeper level, this work shows how social media expresses a collective wisdom which, when properly tapped, can yield an extremely powerful and accurate indicator of future outcomes.’ [8]

So what’s the reaction of others? Is this exciting? Brian Solis, who is principal of Futureworks, an award winning New Media agency in Silicon Valley in a reaction on the research described above, writes ‘Twitter’s trends is a cultural mirror that reflects the state of attention and intention. And as such, Tweets then offer an MRI that visualizes the minds of consumers and more importantly, serve as a crystal ball that reveals the future of products and services before and soon after they’re released.’[9] Further, Solis notices that many businesses are by now trying to engage in the conversation with the consumer in stead of ‘capturing and analyzing that inherently inspires empathy and ultimately relevance.’ [10] This will change immediately partly due to the research of Asur. Definitely, if businesses are about to change their process there are changes coming for the consumer as well. A smart business will try to harness as much information from social networks such as Twitter and use it in their methodologies of product design and services. In this way it’s the consumer’s turn because the engagement between consumer and business will be form the outside in and from the bottom up as Brain Solaris states.

References:

[1],[2],[3] Cheong, M,. Vincent Lee, ‘Integrating Web-based Intelligence Retrieval and Decision-making from the Twitter Trends Knowledge Base’ ACM New York, USA, 2009.

[4] D. Gruhl, R. Guha, R. Kumar, J. Novak, and A. Tomkins, “The Predictive Power of Online Chatter,” in 11th ACM SIGKDD Chicago, IL: ACM, 2005, pp. 78 – 87

[5], [6], [7], [8]S. Asur, Bernardo A. Huberman, “Predicting the future With Social Media” National Science Foundation, Computing Research Association CIFellows Project, 2010.

[9] Solis, B. ‘A Prediciton: Twitter to Predict the Future’ http://www.briansolis.com/2010/04/a-prediction-twitter-to-predict-the-future/, consulted on 09-10-2010.