Over the last two weeks, we’ve been mapping the social media conversation around the major speeches at the Republican and Democratic conventions. There’s been a lot of buzz lately about using tweets as a form of real-time polling (just look at the Twindex, and the ever-present tweets-per-minute stat). We thought the conventions would be a perfect time to either prove or disprove that Twitter could serve as a real-time barometer of public opinion, by looking at how the Twitter reaction to a given night at each of the conventions could forecast the size of the convention bounce tied to that night’s speakers.
We were pleasantly surprised by what we found. Social media reaction and post-speech polling agreed far more than they disagreed, making real-time Twitter analysis a harbinger of the shifts in public opinion polling that would become evident only a few days later.
Breaking Down the Polls
We had already compiled a rich data set of the most popular speakers and speech lines for both the RNC and the DNC, and now we needed to correlate it to polling. This can be tricky, since tracking polls do not break out results by day.
We started by looking at the daily results released by the three major national tracking polls active during both conventions — Gallup, Rasmussen, and Reuters-Ipsos — and looked at the change day-to-day. We then calculated the movement in each daily release and added them up to form what we called a Bounce Index. We also set the Bounce Index to 0 for the first day all three tracking polls were going at once, August 27th, the day before the Republican convention started in earnest. This serves as the baseline for the bounce from both conventions.
Though we can’t definitively associate the change in the numbers to activity on the last day of the sample, the polling surges typical of conventions render it more likely that the last day of the sample was responsible for the shift. Thus, we can get an interesting approximation of which days of a convention were more likely to be responsible for polling shifts. The full results are summarized in the spreadsheet below:
In summary, we found that Republicans made their biggest gains when the full polling days following Tuesday and Wednesday nights were added to the polling samples, and Democrats made their biggest gains following Wednesday and Thursday nights in Charlotte.
Tying this back to the speakers, the polling would indicate the strongest GOP speakers in primetime were Ann Romney, Chris Christie and Paul Ryan. For the Democrats, the speakers who moved numbers the most were Bill Clinton and Barack Obama — and Bill Clinton is in a category of his own here, coinciding with a shift in the Bounce Index of 7 points.
Furthermore, Gallup did some interesting polling of the speeches of Romney, Clinton, and Obama, finding Clinton far ahead of the pack, Obama eliciting a lukewarm but still positive reaction, and Romney eliciting a slightly negative reaction. Each in their own way, these results were predicted by real-time Twitter analysis the night of the speeches themselves.
Speech Mapping the RNC and DNC
In analyzing tweets, we looked at a variety of things across both conventions — overall interest in the speakers measured by mentions, sentiment, quotability, and the use of superlatives like “awesome,” “wow,” and “amazing” associated with the speakers.
I want to caution that there is no slam dunk formula or single measure for correlating speech reaction to polling. Each speaker was strong in different areas, and some speakers who had successful speeches looking at overall interest and even sentiment seemed not to generate any movement in the polls. This is where a layer of common sense political analysis must be applied. Probably our most head-scratching conclusion is that despite the wildly positive reaction to Michelle Obama, no movement in the polls was evident the day after her speech.
We think this is likely because Barack Obama, the person, is a known quantity in a way that Mitt Romney is not. The character testimony provided by Ann Romney may have provided voters with new information in a way that Michelle Obama could not. Furthermore, we found that it was harder for moving personal stories about the candidates to break through on Twitter — with the exception of inspirational quotes that could serve as a life guide to those in the audience. So, Julian Castro’s “My mother fought hard for civil rights so instead of a mop, I could hold this microphone” got three times as many mentions as Michelle Obama’s “We were so young, so in love, and so in debt.”) Though imagemakers love these personal stories, moving numbers with them can be a tricky proposition. Combined with the fact that voters already know Barack Obama, this could help explain why Michelle Obama’s night coincided with no polling movement for the Democrats.
The Republicans
We’ve just discussed the potential impact of Ann Romney’s speech on Tuesday night in Tampa. Though the speech was found less quotable than others, it did meet with positive reactions (69% of the positive or negative sentiments recorded, according to Topsy). Chris Christie’s speech was filled with zingers and was found more quotable than most. Sentiment for the New Jersey governor was a bit lower, but still positive, which is actually an achievement for many of the more “political” speakers — both major party nominees routinely attract more negative than positive tweets, and this was true of both acceptance speeches. All of these factors combined to make Tuesday night in Tampa a success for the GOP.
On Wednesday, the convention’s main speaker was VP nominee Paul Ryan. Here, what stood out was the use of terms like “awesome” in tweets about his speech. Like Obama’s speech the next week, it had the effect of rallying the party base. This reaction is borne out in continued positive movement for the GOP ticket in post-Wednesday night polling.
Now, we get to Thursday night. The two questions worth tackling are the fallout from Clint Eastwood’s remarks, and the effectiveness of Romney’s acceptance speech — particularly important as the GOP seemed only to rise slightly after the last night of the convention.
Earlier, we pointed out that despite his often rambling presentation in a speech that veered numerous times from complete disaster to smashing success, Eastwood delivered some of the most memorable lines of the convention, including “We own this country” and “When someone does not do the job, you gotta let ‘em go.” These arguably channeled the zeitgeist of the disillusioned independent voter more effectively than any canned political speech ever could.
People called Eastwood and his empty chair routine a distraction, but it’s more likely it didn’t contribute meaningfully to a bounce because it wasn’t seen as relevant to the campaign.
It was the Twitter reaction to Romney’s speech itself that pointed the way to a more muted reaction to the last night of the GOP convention. Viewers were far less likely to tweet that his speech was “awesome” than any other major speaker, and this seems to have been a precursor to that Gallup poll of Romney’s speech. If you’re looking for points of agreement between instant Twitter reaction and later polling, notch another one here.
The Democrats
We’ve already noted how Democrats did not move forward in the first full night of polling after Michelle Obama’s speech, and some potential reasons why. Though sentiment was favorable for Julian Castro’s keynote address, viewers also didn’t find it terribly interesting — volume was very low for a speech in the 10pm hour, and was dwarfed by Michelle Obama’s remarks.
While we are very tempted to attribute the lack of bounce due to Michelle Obama to a quirk in the polling — and that still may be the case — there is probably some kernel of truth to the idea that softball speeches that don’t teach the voter anything new don’t move polls, no matter how well received.
Bill Clinton’s speech is a different story. We found the movement in the polls attributable to his convention night that exceeded any other moment of the conventions. The Democrats started the night in their worst position of the election to date — and nearly erased the Republicans’ entire bounce from their convention. It’s not unreasonable to believe that Clinton was able to unlock previously Obama-leaning independents who had switched to Romney during the Republican convention. For a highly partisan speaker, it is notable that sentiment analysis on Clinton’s speech was in positive territory — 60% to 40% among those tweets that could be coded either way.
(We’ll discuss another reason why the Clinton-related bump was so big, and why the gains from the Democratic convention seemed more dramatic, in a section addressing the bounce.)
The picture of Obama’s speech is muddled when you consider the negative media reaction, but the Twitter reaction points to it having been a better speech than the media initially assumed, and the polling bears this out as well. Obama faced sky-high expectations; after all, his initial candidacy and election can be traced back to his soaring 2004 keynote speech. But his actual speech seemed to be a modest success — rallying the Democratic base and defending his administration’s record, as evidenced by his post-convention approval rating bounce.
At a topline level, Obama’s speech generated a great deal of interest on Twitter and was the clear leader in volume, generating over 52,000 tweets per minute at its peak. Negative sentiment outweighed positive sentiment 57% to 43%, but this negative tilt appears to be the norm for most politicians and for Obama on most days. Usage of “awesome” to describe the President’s speech was at middling levels. Gallup polling showed that public reaction to the President’s speech was below other nominees historically but ahead of Romney. In aggregate, the Twitter reaction agreed with the polling reaction: it wasn’t a great speech, but it did the job.
A Note on Bounces
// <