Marketing’s Red Sock Problem

March 28, 2012 8:31 pm

“With a decrease in the number of pirates, there has been an increase in global warming over the same period. Therefore, global warming is caused by a lack of pirates.”
~ The Church of The Flying Spaghetti Monster

Since this post is about data, it seems only right to begin with three data points.

1. Five exabytes of data are created every two days.
2. There’s a sucker born every minute.
3. Some of these suckers will end up in marketing and advertising.

Speed, Scale And Sensors

There have always been people who draw the wrong conclusions from data and accidentally shoot themselves in the foot. But in the past they shot with a cap pistol. Today, they have a rocket launcher.

In a presentation at PARC, Google’s Marissa Mayer said that there have been three big changes to Internet data: Speed (real-time data); Scale (“unprecedented processing power”); and Sensors (“new kinds of data”).

For marketers desperate to crack the code and unlock the secrets to what makes people buy, the flood of data feels like a miracle.

Innovation at Google: the physics of data [PARC Forum]

Puny Consumers, Behold My Mighty Data!

For people who believe more data guarantees more insights (pro tip: it never has and never will), this makes them feel invincible. “Puny consumers, behold my mighty data! You can resist my sales pitches no more! Tremble at my God-like omniscience!”

Even if you approach it with a healthy amount of skepticism, it’s hard not to get enthusiastic about it. More data and more lights on the dashboard sound pretty cool. We can even augment that data with millisecond- to-millisecond Sentiment Analysis monitored from our secret lair in our Social Media Command Centers.

How can more data, faster data and newer data ever be a bad thing?

The Red Sock Problem

Imagine you’re the freshly-minted CMO of RMS Titanic Airways. On her analyst call with Wall Street, your CEO proclaimed, “RMS Titanic’s stock will soar on the twin engines of real-time data and real-time insight”.

With these as your marching orders, you dive headfirst into the data. Some of the data is hugely, obviously, instantly actionable. For example, everybody hates the processed cheese on the domestic flights and would love pretzels. Instant win: bye-bye cheese, hello happy customers.

Other data is just weird: you suddenly know how many passengers on each flight wear socks and what the color, fabric and approximate age of each sock is.

Red SocksHere begins the road to perdition.

Once the company has data about how many people on a given airplane are wearing red socks, some people will assume that this data by itself MUST be consequential.

Decision Quicksand

A recent working paper titled “Decision Quicksand: When Trivial Choices Suck Us In,” by Aner Sela and Jonah Berger theorizes that this sort of problem is a metacognitive mistake. We confuse an array of options and excess of information with importance.

Our central premise is that people use subjective experiences of difficulty while making a decision as a cue to how much further time and effort to spend. People generally associate important decisions with difficulty. Consequently, if a decision feels unexpectedly difficult, due to even incidental reasons, people may draw the reverse inference that it is also important, and consequently increase the amount of time and effort they expend. Ironically, this process is particularly likely for decisions that initially seemed unimportant because people expect them to be easier.”

Of Correlations, Causality and Charlatans

If the team tortures the Red Sock numbers enough, they will find correlations that they can mistake for causality. Some will do this because they are naïve; others because they are charlatans.

Once the Red Sock numbers are in a column next to factual data, they appear to be factual too. Once the Red Sock numbers turn into pie charts in PowerPoint, they will appear to predict the future. And once everyone’s bonus starts depending on how many red sock wearers convert to becoming Facebook “Likes” for RMS Titantic Airways, you’ll eventually have to shrug and chase the mirage. I suspect this is happening right now at a more than a few companies. We’re just not used to having this much data.

What Can We Do?

“I notice increasing reluctance on the part of marketing executives to use judgment; they are coming to rely too much on research, and they use it as a drunkard uses a lamp post — for support, rather than for illumination.” ~ David Ogilvy

The first thing we can do is to recognize that the torrent of available data means we are at constant risk of becoming savant idiots who know so much about irrelevancies that we know nothing at all. We have to be on guard against the seductions. Find some people you trust at your company, and ask them to monitor you for signs of Red Sock delirium. The best CEOs and CMOs I have ever met have been masters at asking simple questions that cut to the heart of the matter. We would do well to emulate them.

Why does this number matter? What assumptions have we made? Are we combining pieces of unrelated data? Are we confirming our own biases? How do we know this is relevant?

It’s easier than ever to make business complicated.

It’s harder than ever to strip things down to what is important and actionable.

But that’s the job today. Time to get to it.



  • A fun, entertaining and sobering post Tom. Liked it a lot.

    There are some companies/enterprises however that are getting really good at finding the signal in the noise, and then making money on those correlations that none of us see, or when we do it’s too late. James Simons’ Renaissance Technlogies is one example of trading on spurious correlations such as correlating daily price of mustard seed and chili peppers in the New Dehli market place with Bahamian dollars (just making this up of course) . I don’t quite know how they do it, but they keep making money year after year. So maybe there’s something to this red sock theory after all?

    • Tom Cunniff

      Thanks Paul. In truth, I don’t think we entirely know what’s happening. Even in the red sock example I offered, it’s possible that the airline would pursue the whole red sock thing energetically… and sales would go up dramatically. Because this is the expected result of the activity, the company’s leaders would say this validates the theory that red socks are critically important. Yet — despite the great volumes of data the activity would produce — it proves nothing at all about causation.

      I suspect the same thing often happens on Wall Street. When the algorithm produces the desired result, it is “proof” that the secret has been unlocked. When it does not, we tweak the algorithm until success happens again. On the plus side, it may be good at improving our odds a little bit here and there. On the con side, much of our activity may be an advanced form of superstition: if we do a rain dance for long enough, eventually the rain will fall and we can claim one caused the other.

Leave a reply