You're reading...
Six Sigma, Social BPM

If you can’t get “small data” right, don’t even think about “big data”

I’ve been reading an article by a social media marketing person who presented a case study on how to reinvigorate a website.  The case study claimed to have been very successful and presented some Google Analytics charts to show the impact.  Here’s a couple of them:

Small_data_1 Small_data_2

Neither of these suggest to me that there’s been any impact of the relaunch. What was the level of performance prior to the re-launch on 17th November? Show me the 30 days before and 30 days after the launch and I might be persuaded.  The data over 8 days could simply be normal variation in the system. Where’s the evidence of a “special cause” i.e. relaunch, having an impact?

The article even says “I received one direct lead this week. It’s not clear if this was a result of my blog relaunch, or past marketing exposure/efforts, but it’s worth highlighting the real value in social lead generation“.

All this reminds me of the classic marketing quote about half the money spent on advertising is wasted, you just don’t know which half.

This case study and its lack of understanding of how to present evidence of performance improvement reminded me of  one organisation where senior managers thought the workload in a team had been increasing.  I was asked to look at what was going on.  When the data are presented as a Control Chart, it’s clear there is no increase and the process is completely in control with a predictable level of “normal variation”:


No doubt we’ll be seeing more of this sort of thing over Christmas as the government tries to persuade us that the annual “Don’t drink and drive” campaign has made a difference.  Expect to hear “the rate of arrests has dropped from 9.5% to 9.3%”; with an underlying assumption that things have improved.  That might simply be normal variation, but without trend data and a calculation of the upper and lower control limits, it’s just a fantasy claim.

I recently ran a series of workshops on “small data” and the highlight learning points for many of the attendees were simple data handling tools such as Pareto Analysis and Moving Average Charts. You can view the slides here:

Read more about Statistical Thinking and managing by numbers (pdf).

I don’t mean to pick on Marketing people, so if you are one, you probably won’t want to read “We’re in Marketing and we don’t do projects“.









Comments are closed.

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 594 other followers

Connect with Ian Seath

Find us on Facebook Improvement Skills Consulting Ltd. on LinkedIn Follow IanJSeath on Twitter


Copyright Notice

© Improvement Skills Consulting Ltd. and Ian Seath, 2007-20. Unauthorised use and/or duplication of this material without express and written permission from this site’s author and/or owner is strictly prohibited. Excerpts and links may be used, provided that full and clear credit is given to Improvement Skills Consulting Ltd. and Ian Seath with appropriate and specific direction to the original content.

%d bloggers like this: