Analyzing the Twitterati's take on the Super Bowl Commercials

Like it?

comments

Posted February 5, 2008

Keywords: advertising, super bowl, twitter metrics

by Josh Bernoff

My colleague Jeremiah Owyang created twitter.com/superbowlads so we could all rate the Super Bowl ads live. Twitter.com/superbowlads received over 2500 tweets during the game. As Jeremiah pointed out in his blog post, it was incredible to be a part of this, but a big unwieldy to figure out what happened.

Since there is an empty spot in my soul in the wake of the Patriots' historic choke, I thought I would try to fill it by analyzing all those tweets. Wrangling 2500 tweets into categories and making rankings out of them took some ingenuity and I still don't claim to have a definitive analysis, but by making some reasonable assumptions (see end of post for details) I not only could make sense of what actually happened, but gain some insight into people's reactions to the ads.

Disclaimers: 1. This is a summary of tweets from about 70 people who are not representative of ANYTHING. They're more interested in media and advertising than the average person, but that's about all I can say. 2. My classification system is the best I could do, but classifying random tweets is not perfect, especially when people who tweeted aren't always clear about what ads they were referring to. Did the best I could. 3. I didn't include any ads with less than 9 tweets, but I DID include the FOX promos since a bunch of people rated them.

Here's a table showing the results of my analysis. What's interesting is that many, many ads generated both positive AND negative sentiment. This table is sorted by what I call "Net Positive" which is # who rated 4 or higher, minus percent who rated 2 or lower -- sort of like a net promoter score. (The instructions called for a 5-point scale -- I accepted fractional scores and zeros, too.) More insights below the table.

Superbowl Ads Twitter Ratings compiled by Josh Bernoff, Forrester	Count	Mean	Median	4 or more	2 or less	Net Positive
Coke balloon	48	4.32	4.50	83%	2%	81%
FedEx	59	4.09	4.00	83%	7%	76%
Coke politics	52	4.14	4.00	75%	6%	69%
NFL Pitts	34	4.01	4.00	76%	9%	68%
WallE	28	4.02	4.25	75%	11%	64%
Pepsi Timberlake	52	3.86	4.00	69%	15%	54%
Tide	52	3.90	4.00	65%	13%	52%
Bud Horse	64	3.68	4.00	63%	14%	48%
eTrade clown	31	3.59	4.00	68%	26%	42%
Doritos mouse	42	3.63	4.00	60%	19%	40%
Bridgestone R. Simmons	25	3.46	4.00	52%	12%	40%
eTrade spitup	50	3.51	4.00	54%	20%	34%
IronMan	15	3.51	3.50	47%	13%	33%
Terminator	12	3.42	4.00	67%	33%	33%
VitaminWater	30	3.50	4.00	53%	20%	33%
Bud Light firebreather	30	3.15	3.25	47%	20%	27%
Adam Sandler Zohan	14	3.07	3.50	43%	21%	21%
Wanted	20	3.43	3.50	50%	30%	20%
Victoria's Secret	26	3.27	3.50	50%	31%	19%
Bud Light Will Ferrell	32	3.11	3.75	50%	34%	16%
Garmin	45	3.20	3.20	40%	27%	13%
T Mobile	33	3.17	3.00	42%	30%	12%
Planters	52	2.95	3.00	44%	33%	12%
Daytona	10	3.10	3.50	50%	40%	10%
Bridgestone animals	44	3.34	3.50	34%	25%	9%
Audi	54	3.17	3.00	43%	35%	7%
Diet Pepsi Max	63	3.06	3.00	40%	33%	6%
Bud Light cheese	32	3.03	3.00	41%	38%	3%
Narnia	12	3.21	3.25	33%	33%	0%
LifeWater	55	2.90	3.00	35%	38%	-4%
Toyota badger	38	2.87	3.00	34%	42%	-8%
Doritos itunes	25	2.68	3.00	36%	44%	-8%
Antidrug	11	2.71	3.00	18%	27%	-9%
Leatherheads	10	2.80	3.00	30%	40%	-10%
Bud Light caveman	18	2.67	2.75	28%	39%	-11%
American Idol	17	2.44	2.50	35%	47%	-12%
Toyota big wheel	17	3.00	3.00	29%	41%	-12%
Bud Light flying	23	2.48	3.00	30%	43%	-13%
CareerBuilder heart	51	2.58	2.50	29%	49%	-20%
cars.com circle	20	2.60	3.00	25%	45%	-20%
Dell red	19	2.53	2.00	32%	53%	-21%
GoDaddy	33	2.14	2.00	30%	52%	-21%
cars.com headshrinker	23	2.52	2.50	26%	48%	-22%
Chase secret agent	9	2.64	2.25	22%	44%	-22%
Amp	40	2.33	2.00	28%	53%	-25%
Hyundai 2	11	2.18	2.50	18%	45%	-27%
GMC Yukon	42	2.21	2.25	21%	50%	-29%
Gatorade	24	2.48	2.00	29%	58%	-29%
icebreakers	13	2.50	2.00	23%	54%	-31%
CareerBuilder spider	13	2.42	2.00	23%	54%	-31%
Taco Bell	21	2.45	2.00	19%	52%	-33%
Gatorade G2	20	2.48	2.00	20%	55%	-35%
Hyundai	17	2.24	2.00	24%	59%	-35%
UnderArmour	34	2.40	2.00	15%	53%	-38%
Zantac	13	2.08	2.00	15%	54%	-38%
Dell sera	9	2.17	2.00	11%	56%	-44%
Bud Light pickup	31	1.81	1.00	19%	71%	-52%
Sunsilk	11	1.59	1.50	0%	73%	-73%
Salesgenie ramesh	27	1.24	1.00	0%	85%	-85%
Ford truck	9	1.83	2.00	0%	89%	-89%
Claritin	9	1.22	1.00	0%	100%	-100%
Salesgenie Panda	25	0.78	1.00	0%	100%	-100%

Some insights from reviewing these tweets:

Coke scored two of the top three spots, FedEx got the other. The coke ads also scored high on USA Today's Ad Meter, but not as high as here. The twitterati loved the positive messages in the Coke ads. FedEx's silly pigeon got the other top spot. Both will "build the brand" but there's no real call to action.
Amazingly, the NFL's own ad about the oboe playing Chester Pitts got a huge score. Apparently you like warm, uplifting stories. None of the other NFL ads even got enough tweets to score them.
Salesgenie scored two of the bottom four spots. Many twitterati found these animated spots racist. I wonder if salesgenie will actually benefit from desperate salespeople who want the leads they are promising, or if the racist backlash will hurt them. That's a nearly $5 million gamble. Amazingly, one Salesgenie ad scored an average of 0.78 on a scale of 1 to 5 -- indicating the scoring range didn't go low enough to account for viewers' disdain. The lowest-scoring Bud Light ad, about immigrants picking up women, was also cited as racist by several.
Claritin, Ford, and Sunsilk left people unmoved. These three ads, like Salesgenie's scored not a single 4 or 5 rating, but only moved 9 or 11 people to rank them at all. In contrast to Salesgenie, the reaction was boredom, not a negative reaction. Why spend so much on an an avail and then create a lackluster spot?
The commentary was interesting, revealed more than numbers. Jeremiah is onto something here. I participated in Nielsen's "Hey Nielsen" polling but couldn't write free-form text. These twitter commenters, by contrast, told us that they would have been happier if Richard Simmons got run over (Bridgestone), that they forgot the name of the product in the dancing lizards commercial (it's Ice Breakers), and that they enjoy seeing Justin Timberlake get hit in the crotch with a mailbox (Pepsi).

Obviously, a superbowl ad is like a Hail-Mary pass -- it's great if you connect, but costly if you don't. And what's the value of an ad like the Toyota Corolla badger ad, which got many positives but just as many negatives? I know all of you marketers have tested these ads to an infinite degree before spending all this money -- but then why do so many of them leave people cold?

Do you think any were designed to provoke?

Finally, ask yourself this. Imagine that it is August 1, 6 months from now. Which of these ads will have made a positive impact on their company's sales? How will you quantify that? And could you have made that impact more cheaply? How?

I look forward to your comments.

Final notes:

Here are links to Google Spreadsheets with all the comments summarized and sorted by commercial. It's fascinating to see the range of comments on some of these commercials. (Some spaces, commas, and other special characters have been deleted due to the way I did the analysis.)

Ratings summary (table above)
Comments on brands A-C
Comments on brands D-N
Comments on brands P-Z

How I did the analysis:

Sorted comments alphabetically to identify brands.
Where comments on multiple brands, created duplicates
Where comments on multiple commercials, identified which one based on time stamp, comments
Assigned a numerical value based on comment. Scores above 5 scored as 5. Negative scores scored as zero
Put comments for each ad on a separate page
Removed duplicate comments from same individual (except promos, which aired multiple times)
Sorted from highest to lowest score
Computed statistics for each ad
Ads with less than 9 comments were put into miscellaneous category. This includes some promos, lots of local ads, and a few national ads.