Is Twitter better at predicting elections than opinion polls?

Whether it’s Brexit or the US presidential election, in recent years traditional opinion polls have struggled to predict the final outcome accurately. A new study from European think tank Bruegel is now exploring whether social media "buzz" could be a better indicator of voting intentions.

Jointly with the Dortmund Center for data-based Media Analysis (DoCMA), Bruegel’s research team analysed nearly 900,000 tweets related to the EU referendum in the United Kingdom, with some intriguing findings.

Gauging the hearts and minds of UK voters

From the May 2015 parliamentary election, which put the Conservatives back in office on a platform that drew heavily on the promise of an EU referendum, the number of tweets on the topic rose from 1,000 a week to 100,000 by the end of May 2016, roughly one month before the vote.

The research team performed a sentiment analysis on these tweets. Sentiment analysis is a technique which combines natural language processing, text analysis and computational linguistics to establish the opinion or attitude conveyed by a speaker or writer - in this case, by the Twittersphere. While the researchers advise caution due to having used a new approach to sentiment analysis, their findings are noteworthy.

They show that for the first half of 2015, there was a slight bias for the "remain" position (negative values in Figure 3). Then, around the middle of the year, the bias swung dramatically toward leaving the EU (positive values).

This coincided with the refugee crisis in the summer of 2015. The researchers are confident that their analysis supports the widely held view that concerns about immigration stoked the Brexit debate from this point on. Once the mood on Twitter had moved towards the "leave" side, it stayed there, by and large, until the day of the referendum.

None of these trends were reflected to the same extent in traditional opinion polls (Figure 5).

Based on the analysis, the researchers propose that the pro-Brexit stance may have been strengthened by the dynamics of social media, especially the ability of high-profile Brexiteers to attract growing levels of interest on Twitter. They conclude that whoever gets significant attention on Twitter can count on their messages travelling much further, not least via traditional media channels. Quick to pick up on social media trends in their endless quest for news, the latter help to amplify the reach of social media figureheads and their messages, swaying public opinion in the process.

Appealing to the electorate’s emotions, negativity and breaking taboos - all these "post-truth" communication techniques serve only to fuel the dynamic. Both the Brexit movement and Donald Trump have reaped the rewards of it, especially in terms of winning over undecided voters, according to the Bruegel research team.

The study concludes that it may not be far-fetched to use Twitter sentiment analysis as an early indicator for public opinion, in particular in polarized debates such as the EU referendum. The method may be less effective in typical parliamentary elections, however, where the electorate needs to consider a wider range of positions.