The Secret to Finding True Customer Insight

“I guess we just made a big mistake,” Jonah said.

He looked directly at me for a moment and then downward at his wrinkled hands collecting his thoughts as the sun was setting, casting long shadows across the kitchen table.

His wife reached out and laid her hands protectively on his while Jonah looked me directly in the eye.  They both had something to say. The retired couple were visibly upset but were doing their best to keep their composure.

“We are of modest means,” he started.

As he began his tale I could see tears swelling in his partner’s eyes. Jonah was sitting broomstick straight stoic, his bubbling rage barely constrained behind his dark blue eyes. She sat silent, interjecting on occasion. They both had a message to send. I was to be their messenger. I was there to be their confessor and offer, perhaps, some small degree of absolution.

In this in-home interview, we weren’t talking about a loved one lost. We weren’t talking about a swindled retirement savings or some form of nefarious flim-flam that convinced Jonah and his wife to buy brackish real estate in Florida. We were discussing an automotive purchase, about which they now had very serious misgivings.

“We just believed in them…we feel so betrayed….” she said.

This was very real for them, and in their presence I felt their worry. I experienced their remorse. I felt their pain. I walked away from the interview moved, a bit sad and much wiser. What survey could have captured this?

An undergraduate professor I once had related how he couldn’t conduct surveys because he thought them impersonal and even insulting to participants. He judged them, on the whole, disrespectful and indignant. At the time I dismissed him as being an overly sensitive neo-hippy lacking monetary drive who should just go back to his yurt and carve a new bong out of reclaimed barn wood.

I have recently started to rethink my judgment of the good professor’s position.

Much of the research industry has long viewed the respondent as a commoditized, and largely free, raw ingredient in which to make insight sausages. The ideas being we could just zap enough people in a panel to fill our quotas and off we go. A filled quota is a filled quota after all.

As much as the thought of the mechanization of queuing people to answer your inquiries is appealing, I would encourage you to think about who exactly are taking these long boring surveys.

I have become deeply suspicious of some panels that prod their herds to complete some really poorly constructed “self service” surveys through incentives. Flat-liners, speeders, repeaters, and plain old cheaters are prevalent. There are even software programs that will cheat for you that are available for purchase. A.I. is let lose upon the untended yearning fields of uncompleted questionnaires to do what humans are loathe to do.

As a Professor at the University of Arkansas, I asked my students to describe what taking survey was like using only one word.

“Annoying” “boring” and “pointless” led the list.

In designing surveys, I encourage those same students to ask themselves “would I fill this out”? If not, you should think about a redesign…or different approach completely.

Surveys have their place. I don’t think they are going anywhere soon, but we should become much better at choosing our audiences and creating better and engaging designs.

Good data comes at a price. That price is investing the time to really care about what people have to say and listening in ways that they want to communicate. To not view them as the raw grist for the insight mill to process.

Every day millions of posts are made on Facebook, LinkedIn, and Twitter. Much of this communication is the expression opinions about people, product, places, and things. Why do people share these things? Because they are provided a forum in which they feel heard. A forum, in which some small way, their voices matter. This empowers people.

So let’s not dump surveys altogether, but become more attuned to the people you are talking to. They are real people with real problems, fears, and dreams. Be with them as a partner and confident, not as some indifferent corporate scientist.

The key to gaining great insight can be found in respectful and careful listening. It is about engaging in deliberate empathy. This is where real insight is uncovered; the ability to see the world from the viewpoint of the people you are interested in understanding.

You will get so much more from this approach even in smaller sample sizes than some sterile panel survey that goes out to god knows who or what. Keep it real. Keep it human.

The fundamentals of consumer research are sound. Companies are still very interested in what their customers have to say. Customers are very interested in sharing their opinions and experiences.

The trick is to listen to your customers in a way they want to be heard. Let them know you care. Be their messenger. Respect them. And if possible, be their advocate.

Free Data: A Scrappy Guide to Secondary Data Sources

Who are my target customers?  Who are my competitors?  Is my industry growing or shrinking? What sources of information are consumers in my industry using to make purchasing decisions?  If you have any of these questions, or others that involve traditional “insights” or consumer research, then I have good news.

I have some FREE answers for you.

Well, almost free, you need to do a little work too.

In my previous article, I discussed the importance of getting your business question right. After all, if you don’t know what you are looking to answer, it’s pretty hard to answer it. Now that you have that nailed, its time to identify good data sources.

Good research doesn’t have to be expensive.  There are numerous resources out there to help with product development, market sizing, targeting, channel strategy, and numerous other business questions.  In consumer research, we look at two types of data: primary data and secondary sources.

Primary data is any data collected for a specific purpose.  It is usually a survey, but can include mystery shops, social media harvesting, focus groups, ethnographies, and a litany of other primary research techniques.

Secondary research, sometimes referred to as “desk research,” are data sources gathered for numerous specific and general purposes. For this article, I will focus solely secondary sources.

There’s loads of great secondary research out there, and in many cases you already paid for it!  If you are a U.S. taxpayer, Uncle Sam has been busily gathering all kinds of useful information for you.  In addition, quasi- and non-governmental sources abound.  Last but not least, even for-profit firms are not entirely stingy with their data.  Let’s look at some of the more useful sources.

Government Sources

In 2009 the Obama administration swung open the doors to  This is essentially an “overlay” site for multiple government data sources.  It is chock full of data that can help the small- and medium-sized business owner answer all kinds of marketing strategy and product development questions. Here are a few waiting to be explored.

Brought to you by the good folks at the Bureau of Labor Statistics, the Consumer Expenditure Survey  looks at U.S. buying habits, income, and household characteristics.  Another great resource is more or less an addendum to the decennial U.S. Census called the American Community Survey.  This data set tracks population down to a very fine level of geographic detail with a sample size of over 163,663 interviews (and estimates for over 130,000 others).  The survey inquires about age, sex, race, income, health insurance, education, and a variety of other tidbits that could help you identify where and whom to focus your business on.

From a B2B and competitive intelligence perspective, the Economic Census asks 4 million businesses to provide feedback about their organizations every 5 years.  Thinking about opening a pet food supply store?  You can determine the number of players in that space and the annual revenue for that industry.  Questions such as, is the industry growing or shrinking, who do they sell to, and what types of products they sell can help you identify the size and attractiveness.  There is also a plethora of information from the Census Bureau on Retail Trade.

The USDA provides heaps of information on what is produced and consumed by the American public.  Wondering about how often people eat out?  Look no further than the ERS Food Expenditure Series.  Wondering about Pork consumption; you can find all about it here.

That’s not the end.  There are gigabytes of free data at your disposal for many questions.  Unfortunately, the government doesn’t always make it easy to access, but they do offer tools like American FactFinder and DataFerrett to help. Many have useful tables and reports that require no analytical software.  However, you may need to breakout out Excel or even SPSS (IBM’s predictive analytics software) or SAS (business analytics software) for the larger datasets.

Association, Trade Data, and Non Profit Sources

Industry and trade associations are commonly established to support a particular industry and give it a common voice.  There is literally an association for almost any industry.  From the American Pet Products Association, which offers some data free and the rest for a reasonable price (special member price of $495!), to the National Barbecue Association.  There’s even The Association for Dressings and Sauces.  I am sure there is an industry (and data) for your specific industry needs.  One of the more generic industry associations that appears to be the especially well heeled is the National Retail Federation

The NRF has all kinds of helpful information within its Insight Center.  Wondering where consumers shopped for toys last month? It’s right here and free.  How important is free shipping to consumers when online?  Look right here.  Wondering what people are planning to do for St. Patrick’s Day?  Right here.

For more general trends and facts, like feelings about personal finance or parental time use, the Pew Research Center is a nonpartisan “fact tank” that can be accessed for free.  Similarly, the University of Michigan Survey Research Center has data concerning a number of topics of potential interest to small and medium business owners such as the Health and Retirement Study, the foreboding-sounding Panel Study of Income Dynamics, and a collection of useful data about Economic Behavior.   NAICS is not only the national organizer of businesses (replacing SIC codes in 1997) but is also a good source of Firmographics, and a great place to figure out who is doing what with a comprehensive list of almost every business in the United States.  Spokeis also a handy website for competitive intelligence gathering.

You can burn many, many hours looking at all of these data sources, so remember to stick to your original question and try not to get distracted.

For Profit (mostly) Freebies

There are many research agencies that provide tables and reports publicly for public relations and promotional purposes.  Lucky for us, they are free.  Check out this insightful report on consumers who are less or more likely to be harsh on your company.  Gallup has some excellent free information about social trends.  Find out about daily retail spending, economic confidence,  or where people exercise the most.  The Roper Center also does quite a bit of social polling, some of which is free. ASCI tracks how hundreds of brands in 48 industries perform on customer experience.  Find out which supermarkets hotels, orairlines are best in their industries.  JD Power and Associates has similar benchmarking data for selected industries.

Wonder where to start your business?  ZoomProspector is a cool tool to identify potential best locations.  City-Data is also a remarkable collection of geo-demographic data to plan your next retail outlet.   Ad Age’s data center is free and has a ton of free information on advertising, media spend, and more. provides a treasure trove of competitive intelligence from B2B applications.  A good chunk of the site is free, with a fee for more details.  Finally, a massive archive of current secondary research can be found at, but you must crack the wallet open just a bit to take advantage of most the content.

Check Your Sources

All data is not the same.  Data is the building block of any good business case or strategy, so it’s important to make sure it’s solid. Most of these sources I have personally used before and found them to be based on sound scientific rigor.  No one is going to doubt statistics from the Bureau of Labor Statistics, while they might hesitate to trust that Facebook poll you conducted with 200 of your friends.   As you see, you can get a long way without spending a nickel on data collection.  However, there will come a time when you need to do some primary data collection.  For example, getting direct feedback on service, features, or products for a specific target group.  There are also cost effective ways to gather that data.  It’s not free, but much easier to do than in the past.

Insights: A Small Investment for Big Returns

It’s no secret that the majority of new product introductions fail.  According to the Harvard Business Review less than 25% of consumer packaged goods and retail products fail to earn $7.5m in their first year.  The annals of business literature are littered with failed products from New Coke, Crystal Pepsi, Frito Lay’s Wow Chips, Microsoft Zune, McDonald’s Arch Deluxe, the Pontiac Aztec, and more recently the Samsung Galaxy Gear.  Why did they fail?  Well, it usually comes down to the fundamental problem of not understanding the customer.  This ailment comes in three flavors: not conducting any market research, doing it wrong, or ignoring the results and voice of the customer.

Due to the presence of Olestra, some consumers experienced intestinal discomfort

No Consumer Research

In this failed product scenario there is no consumer feedback collected whatsoever.  This happens often in small business start-ups.   Entrepreneurs tend to love their own babies, even when they may be ugly.  Many times these founders are so passionate they convince their friends and family that their idea is fantastic.  The very passion of entrepreneurs  have can be blinding, which makes the need for consumer insight even more important.   “Research is too expensive” is the statement I hear, to which I retort, “what is the cost of failure?”

Henry Ford and Steve Jobs are both lauded, rightfully so, in the business press as inspirational and visionary thinkers. They both shared a disdain for market research.  While their product legacy lives on to today in Ford and Apple they were both instrumental in bringing to market two huge product failures: the Edsel and the NeXT computer respectively.  Both of these products had some very advanced features and in many ways were well ahead of their time. Unfortunately, in both instances there was a fundamental misunderstanding of the market and customer expectations.

Good intent may result in disastrous consequences

Truly great visionaries may get by for a while without any customer feedback, but without any kind of check on the market they run a very high risk of making some very big mistakes. Launching a new product or service without any consumer feedback is like trying to land an aircraft on a runway at night with no instrumentation. You might make it, but it could be catastrophic.  Why risk it?  It doesn’t have to be complicated.

Bad Customer Research

In other cases, the research conducted is just bad. Coca Cola conducted a massive amount of product testing only to find out that people actually preferred the taste of Pepsi. The logical thing to do?  Make it taste better.  That’s actually what they did with the launch of New Coke.  What they didn’t do is realize the love affair the world had with the brand.  To customers, Coca Cola was a national treasure and they did not want their national treasure altered.  The result was pulling of New Coke of the shelf and the accidental re-ignition of the love affair the world had with Coke.

This is an all too familiar instance of not framing the question properly.  It wasn’t all about the product attributes; it was about the brand and emotional connection people had with it. If Coca Cola conducted their research more broadly they would have found that out.  I have seen many instances where consumer research is too narrowly focused on features without understanding the higher order emotional and psychological desires of people.

Of course there’s about a bazillion technical ways to conduct research poorly; bad sampling, bad design, inaccurate data collection techniques, faulty and misleading analyses, but that’s a topic for another article. The basic point here is; if you going to do research make sure you do it with an open mind and get someone not as intimate with your idea to help think about the right questions to ask.


It always amazes me the amount of money that big companies spend on consumer research only to ignore it. As one of my more sardonic colleagues is fond of saying…

“Research, when it confirms what we thought it’s a waste of money, when it doesn’t it must be wrong.”

Ignoring customer feedback is the sign of a company out of touch and a tell tale sign of bad things to come.  Arguably one of the ugliest vehicles ever made was the Pontiac Aztec.  Former GM Executive Bob Lutz summarized it best in Road and Track
2001 Pontiac Aztec

Early on, the Aztek obviously failed the market research. But in those days, GM went ahead with quite a few vehicles that failed product clinics. The Aztek didn’t just fail—it scored dead last. Rock bottom. Respondents said, “Can they possibly be serious with this thing? I wouldn’t take it as a gift.” And the GM machine was in such denial that it rejected the research and just said, “What do those a**holes know?”

Hubris and a feeling of product invincibility brought down many a goliath organization.  In Jim Collins’ book “How The Mighty Fall” he outlines the stages where great companies fall.  At the apex of a declining organization is Stage 3 entitled “Denial of Risk and Peril”.  One the key markers for this stage is a tendency for the organization to explain away or discount negative data.  Become aware of that is starting to happen at your organization.

Another mistake is ignoring current ongoing customer feedback about products and services.  In a world where word mouth travels to millions in milliseconds, getting on top of this through listening posts and responding is absolutely critical.   Unfortunately, many companies are not monitoring channels effectively.  Many companies have active feedback coming into their call centers or through sales representatives but it is not consolidated and sent back to those who can do something about it. More alarmingly are those companies that actively “hide” from customers once they sell their product.  You can tell these companies as they offer only a text box email to contact them usually with no phone number or address listed on their website.  A company that hides from their customers is doomed to fail.

The Good News

The good news is you don’t have to be “that company”.  Consumer research is not the end all, be all.  Creativity and innovation is not usually distilled from sitting behind a one-way mirror watching a focus group while eating M&Ms. In fact, I think that method of customers has experienced a rightful death and should be buried.

However, not listening and observing your customers as an input into your product development, implementation, and delivery efforts is a guarantee for failure.  So, does gathering this feedback need to be time consuming and expensive?  Nope.  There are many techniques and services that are no cost or low cost that can help you avoid some fatal market place blunders.  In future articles I will be reviewing some of those techniques, services, and data sources that can help you along the way to become a successful and customer centric business.