Market Research

I used to be one of those people who would read articles in major news outlets and be shocked at certain statistics. “50% of people in Oklahoma City don’t have high school diplomas!” “Eating strawberries reduces your chance of developing eye cancer by 67%!” “80% of married men cheat!” I’d scurry off to my friends and repeat what I’d read, devising how I’d need to change my lifestyle to ensure I didn’t end up with a bad case of psoriasis from eating cole slaw.

Little did I know what a mistake I was making. It’s fascinating how easily statistics and research data can be manipulated to present a very biased picture of what’s actually going on. As marketers, we need to champion the use of good market research as the base upon which all marketing programs are built, and be certain we’re building our research methods as effectively as possible.

Here are some tips to help you avoid falling into these all-too-common research data traps:

1. Be a skeptic. 

People tend to use statistics carelessly, and the actual research study that was performed is rarely linked to the article. A writer might not disclose whether a corporate sponsor paid for the research and is skewing the results. Before you run off implementing some new social networking strategy after reading an article claiming 67% of American households use Facebook and/or Twitter, do YOUR research. Dig out the research study and read it. With the Internet, we have the power to find these things. I hate to say, “Don’t believe anything you read,” but it’s often true.

2. Understand what could constitute bad research. 

Experiments are set up differently for many reasons. For example, you can’t force someone to start smoking, or drive without a seatbelt. This means that inferences have to be drawn based on an observational study of smokers, and—yes, you got it—that pool of subjects is already biased. The results from that group would not yield results applicable to a non-biased population. A randomized study usually provides less biased data. Observational studies should always have a control group, and sometimes they don’t. Worse, some studies completely falsify their data. Be aware!

3. Pay attention to sample size.

This is a very common trap. Studies often exaggerate the feedback from a very small data set as being representative of an entire population. If 10 people are giving you positive feedback about your new HR policies, be sure to be thorough in finding out if the other 500 employees feel the same. You often find wild swings in variation in small sets of data. Leave your ego out of it—collect enough data to be sure you’re your sample represents the entire population within a reasonable margin of error.

4. Everything regresses toward the mean.

Disappointing, I know. If you changed nothing, those outstanding click-through rates you got two months in a row after months of steady results are probably just chance. In time, all swings even out.

As marketers, we know how easy it is to design campaigns to manipulate people’s minds and habits. However, we have a responsibility to the consumer and to ourselves to use data correctly. Back up your claims. Use credible research. Collect enough data to make sure your conclusions are correct. Data doesn’t lie, and neither should we.

Another Oscar show is in the books and the ratings did tick up a bit from last year’s record low.  While the show format is tired, it is not the real story of why it no longer holds America’s interest as it once did.  The original purpose of the awards was to provide a sales boost to the winning movies and to get some PR love for the industry’s image.  That is still the purpose today and it is reflected in the nominated movies and the tone of the show.  This is the industry’s opportunity to show that they make “Big important movies about big important ideas”. 

The nominated movies are often loaded with political messages and diversity.  That is not a bad thing, but a complete disconnect with what the movie industry actually produces in large quantity today.  The majority of releases are marketing concoctions aimed at the at the lucrative demo of young men and women and kids.  Tad Friend wrote a fascinating article in the New Yorker about the movie marketing business.  The story takes you into the sausage factory and it is eye opening.  The artistic process has given way to focus groups and market testing to make sure the product resonates with the target audience.  Between 25-35% of a movie’s budget is taken up by marketing efforts.  Much of that money is dumped into a barrage of unmeasurable and unaccountable 15 and 30 second TV spots.  Like their cousins in the auto industry, movie marketers show no signs of breaking their addiction to print and broadcast advertising that is high in cost and low in return. When it comes to “renting” media channels, the movie industry spends money with a fire hose.

The whole situation becomes a self fulfilling prophecy.  The added marketing costs make it harder for movies to make a profit, so they market test them and alter the film to resonate with the ticket buying young men and women.  This is apparent when reviewing the top movies of 2008.  The top grossing movies are mostly action and comedy films aimed at 15-28 yr old men and women.  There is also a number of animated movies for the under 12 crowd.  The first “adult” movie checks in at numbers 11 and 12 with “Sex in the City” and “Mama Mia”.  After that it is slim pickings for the over 35 set (I could also say the over 110 IQ set, but that would be mean). 

The fact that Hollywood is primarily producing products for the youth market is not a crime.  They are in business to make money and those are the people who buy movie tickets.  Ironically, the target audiences for their movies are the people least likely to settle in on a Sunday evening to watch an Oscar telecast that moves at the pace of a 1970’s variety show.  Unlike the Grammy awards that live in the moment, the Oscars are often about the past.  And, giving technical and behind the scenes awards (sound, costumes, editing) during prime time is not appealing to anyone except the families of the nominees.

There is also another factor at play in why America is tuning out the Oscars and that is political persuasion of the movie and entertainment industry.  They are not shy in their support of Obama and liberal causes.  This manifests itself in both financial and vocal public support.  Sean Penn came through last night talking about how America elected an “elegant” man.  Who knew Obama was like Fred Astaire.

This can be a turn off to the roughly 48% of the country that did not vote for President Obama – elegance and all.  It is one thing to tolerate a difference in opinion from elected officials such as Barney Frank and Nancy Pelosi, but quite another to be lectured by rich, spoiled celebrities like Matt Damon.  Don’t underestimate some of the hard feelings from moderate/conservative voters who got fed up with the media and entertainment industry during the past election.  Nobody likes to be told how we are not doing enough about global warming and healthcare from surgically enhanced people who earn millions to stand in front of a camera and recite someone else’s words.  The individuals in the movie business have every right to speak their minds, but it can be a risky proposition when you are selling a product to a broad consumer audience. 

This explains some of the reasons why the Oscars are losing relevance, but what about the future of the movie industry in the age of new media.  We’ll explore that next week….

Looking back at our election preview it seems that the bulk of our contributors did foresee Barack Obama’s win.  Junta42’s Joe Pulizzi, Brian Bucky and our own Kimba Jackson probably came in the closest to the final result.  For their efforts they win a year supply of Rice-a-Roni and brand spanking new President.  A look at the efforts of professional pollsters is a good lesson in market research technique and how to interpret data.

The subject of surveying was a big one this year as pollsters had the very tough task of trying to estimate the proportion of voters by party identification.  That is the key variable for weighting the results.  It is usually based on the last national election – Presidential or mid term congressional.  In 2008 pollsters tried to grapple with the unknown factor of how many new voters Obama would be bringing into the process.  It was tempting to jack up the percentage of democratic voters based on the response to Obama’s appearance and fund raising.  The results are in and Fordham University did a study comparing the results of all of the polls.  The winners were Rasmussen and Pew which nailed the final margin.  See the full list here.

All during the campaign I was reading and relying on Rasmussen because they were the most conservative in their estimates, and always had it as a pretty tight race.  I had a gut feel that Gallup was over weighting to Democratic Party identification and as a result they came in near the bottom for accuracy.  A couple of lessons learned:

The best poll methodology is the tracking polls like Rasmussen because they survey every day and keep reporting a three day rolling average.  They also screen on likely voters rather than registered voters which tend to favor democratic candidates over republican.

Because of accelerated news cycles and proliferation of news and information sources opinions move and change quickly.  That makes the tracking polls superior to the polls that are just a snapshot.  The “snap shot” polls only capture a moment in time vs. a rolling average.

Put your trust in pure research companies over polls sponsored or done by media companies.  The bottom half of the list is dominated by media outlets such as WSJ, NBC, ABC and FOX.  And the two overall worst were the New York Times/CBS poll and Newsweek’s.  All of these snapshot polls consistently over rated Obama’s support and put him much further ahead.  They gave the appearance of a much bigger margin between Obama and McCain.  Media companies do these polls for marketing and PR purposes as much as to provide content to their viewer and readers.  Going forward, they should be somewhat discounted after this very poor performance.  Newsweek’s poll was particularly galling – they usually had Obama up by 12-15 points.  And, since they promoted the heck out of their survey it made news because it was showing such a wide spread.

I’d like to give them the benefit of the doubt that their poll was just a poorly done marketing gimmick.  I would be shocked, shocked to learn that objective, unbiased news organizations such as Newsweek and the NY Times would try to manipulate the election by showing Obama ahead by huge margins that no research company was finding. 

Anyway, it’s time to move on and wish President-elect Obama good luck.  And one final message to the media.  Please give Mr. Obama some space and let him pick his cabinet without the need to vet (a.k.a. try to find information to ruin their lives) the people he may be considering.  We will be a better country if you show some restraint and class.

I had the opportunity to contribute another article to Chief Marketer.com.  I wrote about how you can use market research as part of a private custom media program.  For best practices, you should survey your customers and prospects to find out how they like to be communicated with in terms of content and format.  Click here to read how it can help you drive up your marketing ROI.

I recently came across some interesting brand loyalty research.  This article from Chief Marketer  refers to a study done by Brand Keys that looks at a “Customer Loyalty Engagement Index”. In a nutshell the findings suggest customer loyalty greatly increases when products and services are customized to the consumer.  This makes perfect sense, as customization is becoming more critical to marketers as product differentiation is becoming hard to come by these days.  It is a testament to world wide engineering and manufacturing skills that most products sold in the US today are made very well.  Since most competitive products do the same thing reasonable well, the ability to customize for a specific buyer has become an attractive product attribute.

The two factors driving the age of customization are the same ones that are driving the move to private media channels and content marketing.  An advance in technology and web adoption has made ordering customized products or consuming custom content real and easy to do.  However, the biggest factor is that consumers have the power and are in control of their choices.  The web has empowered consumers to shop and/or gather information across the globe.  It also plays into the consumer’s mindset – they would much rather have something created just for them instead of the same product purchased by the guy down the street.

You can see how this can apply to your marketing choices – if you send your customers content that has been customized for them, you will increase your chances of building a relationship of affinity, trust and loyalty.  Many companies are missing a golden opportunity to build a stronger relationship with their customers by not communicating with them in a customized private custom media venue.  How many companies are still relying on mass market bulk mailings to talk to their customers if they talk to them at all?  The technology exists where you can mine your database to know a customer’s interests and tastes; why not send them content and marketing messages that are customized for them instead of a generic catalog or email?  Or even worse, do think you are hitting your current customers with a mass media ad that is focused at prospects?  Marketers are required to evaluate their customer communications in light of changes in the way people desire and consume information. 

We are in the age of customization, and savvy marketers don’t want to be sending yesterday’s news to tomorrow’s customers.

Missing old friends? Want invites to parties? How about offers to join dozens of new online social networks (all of which are stories in themselves)? Start blogging. Can’t say that we’ve been offered a custom media gig for The Who yet, but given the volume of contacts surfacing in the last 45 days, I’m holding-out hope.

At Think Tank, our staff and some interesting opinioned-types have been offering their thoughts on media and its many current offerings and applications. We’re in the business, and we live to challenge the status quo of media you can rent versus owning your own media channel. We call that Private Media, but what’s in a name? What Private Media does for marketers says it all, and the next 125 words is for those people who’ve checked in and want to hear about what’s new at the shop in Salem.

• Our events group just wrapped the Appreciative Inquiry conference where 500 attendees spent 3 ½ days in keynotes and workshops designed to spur positive change in the workplace. Fascinating subject, and powerful speakers. Click here to read what my fellow blogger Gordon Plutsky has to say about the event.

• Webcasts have evolved beyond straight-forward audio and video platforms; the duration is shortening, content is punchier, and other platforms are deployed that include interactive PDFs, virtual tradeshows and even more engaging uses of video. Cool and smart stuff.

• We’re currently conducting a phone and interactive survey for 100 customers of a King Fish client for a comprehensive market research project. This formidable media company has outsourced this significant custom media program to us to determine who makes the cut for an important performance-based award ceremony next quarter. Brilliant use of Private Media to maximize employee retention.

Recent King Fish acknowledgements by Entrepreneur and Inc. magazines confirm that our approach is working, and that our clients are benefiting from this fresh method of assessing business challenges. There’s the update – and onto a request: for any old contacts stopping by Think Tank, please feel free to invite me to any of the following social networks from my past:

• The guys who get 69 GTO convertibles
• Small bars/clubs where the best music still lives
• My sailing friends from PLP (1976-1979)
• A place I can lose myself laughing again with Gardner, Rich, Jamie, Jon, Woozie,  Spike and JB
• An online community for all of us tortured by my 1st grade swimming teacher.