97% of Marketers Will Lie With Bad Statistics

I used to be one of those people who would read articles in major news outlets and be shocked at certain statistics. “50% of people in Oklahoma City don’t have high school diplomas!” “Eating strawberries reduces your chance of developing eye cancer by 67%!” “80% of married men cheat!” I’d scurry off to my friends and repeat what I’d read, devising how I’d need to change my lifestyle to ensure I didn’t end up with a bad case of psoriasis from eating cole slaw.

Little did I know what a mistake I was making. It’s fascinating how easily statistics and research data can be manipulated to present a very biased picture of what’s actually going on. As marketers, we need to champion the use of good market research as the base upon which all marketing programs are built, and be certain we’re building our research methods as effectively as possible.

Here are some tips to help you avoid falling into these all-too-common research data traps:

1. Be a skeptic. 

People tend to use statistics carelessly, and the actual research study that was performed is rarely linked to the article. A writer might not disclose whether a corporate sponsor paid for the research and is skewing the results. Before you run off implementing some new social networking strategy after reading an article claiming 67% of American households use Facebook and/or Twitter, do YOUR research. Dig out the research study and read it. With the Internet, we have the power to find these things. I hate to say, “Don’t believe anything you read,” but it’s often true.

2. Understand what could constitute bad research. 

Experiments are set up differently for many reasons. For example, you can’t force someone to start smoking, or drive without a seatbelt. This means that inferences have to be drawn based on an observational study of smokers, and—yes, you got it—that pool of subjects is already biased. The results from that group would not yield results applicable to a non-biased population. A randomized study usually provides less biased data. Observational studies should always have a control group, and sometimes they don’t. Worse, some studies completely falsify their data. Be aware!

3. Pay attention to sample size.

This is a very common trap. Studies often exaggerate the feedback from a very small data set as being representative of an entire population. If 10 people are giving you positive feedback about your new HR policies, be sure to be thorough in finding out if the other 500 employees feel the same. You often find wild swings in variation in small sets of data. Leave your ego out of it—collect enough data to be sure you’re your sample represents the entire population within a reasonable margin of error.

4. Everything regresses toward the mean.

Disappointing, I know. If you changed nothing, those outstanding click-through rates you got two months in a row after months of steady results are probably just chance. In time, all swings even out.

As marketers, we know how easy it is to design campaigns to manipulate people’s minds and habits. However, we have a responsibility to the consumer and to ourselves to use data correctly. Back up your claims. Use credible research. Collect enough data to make sure your conclusions are correct. Data doesn’t lie, and neither should we.