Gathering real-world feedback from customers is a core concept of Customer Development as well as the Lean Startup. But what information to collect?
Only 57 Questions
Yesterday I got an email from an ex-student lamenting that only 2% of their selected early testers responded to their on-line survey. The survey said in part:
The survey has 57 questions, the last three of which are open ended, and should take about 20 minutes to complete. Please note that you must complete the entire survey once you begin. You cannot stop along the way and have your responses to that point saved.
If it wasn’t so sad it would be funny. I called the founder and noted that there are SAT tests that are shorter than the survey. When I asked him if he actually had personally left the building and talked to these potential customers, or even had gotten them on the phone, he sounded confused. “We’re a web startup, all our customers are on the web. Why can’t I just get them to give me the answers I need this way?”
Continual Data Flow
Customer Development suggests that founders have continual and timely customer, channel and market information.
Founders need three views of information to truly understand what is going on:
- First-hand knowledge
- A “birds-eye” view
- The view from the eyes of customers and competitors
First-hand knowledge is “getting outside the building” and talking to potential or actual customers. Customer Development proposes that the best way to get customer data is through personal observation and experience—getting out from behind your desk and getting up close and personal with customers, competitors, and the market.
Web startups are at real disadvantage here as founders may confuse web metrics, A/B testing and on-line surveys as the entirety of first-hand knowledge – for most web business models they are not. In fact, this mistake can be a “going out of business” strategy. Metrics tell you that something is happening. A/B testing can tell you that one something is better than another. But neither can tell you why. And getting answers back from customers only with on-line surveys when you can’t watch their pupils dilate or hear the intonation of their voice is not something I’d build a business on.
Of course you need to collect metrics, do A/B testing and run online surveys. It’s just that without having founders “get outside the building”, you are missing a key point of Customer Development – the numeric data you collect may be blinding you to the fact that you’re more than likely working to optimize the wrong business model. Customer needs are non-deterministic.
The second picture founders need is a synthesized “birds-eye view” of the customer, market and competitive environment. You assemble this view by gathering information from a variety of sources: web sites, social media (Facebook, Twitter, blogs, et al) sales data, win/loss information, market research data (i.e. compete.com, Alexa, etc.,) competitive analyses, and so on. From this big-picture view, founders try to make sense of the shape of the market and the overall patterns in the unfolding competitive and customer situation. At the same time, they can gauge how well industry data and the actual sales match the company’s revenue and market-share expectations.
(Just remember that most market research firms are excellent at predicting the past. If they could predict the future, they’d be entrepreneurs.)
My test for how well you understand this “order of battle” is to hand the founder a marker, have them go up to the whiteboard and diagram the players in the market and where they fit. (Try it.)
See through the eyes of customers and competitors
The third view is of the action as seen through the eyes of customers and competitors. Put yourself in your customers’ and competitors’ shoes in order to deduce possible competitors’ moves and anticipate customer needs. In an existing market this is where you ask yourself, “If I were my own competitor and had its resources, what would my next move be?” In looking through the eyes of a customer, the question might be, “Why should I buy from this company versus the incumbent.” In a new or resegmented market, the questions might be “Why would more than a few early adopters use this app, web site or buy this product? How would I get my 90-year grandmother to understand and buy this product?”
Think of this technique as playing chess. You need to be looking at all the likely moves from both sides of the chessboard. What would we do if we were our competitors? How would we react? What would we be planning? After a while this type of role playing will become an integral part of everyone’s thinking and planning.
Putting it All Together
First-hand knowledge is clearly the most detailed and essential, but offers a narrow field of view. Founders who focus only on this information risk losing sight of the big picture.
The “birds-eye view” provides a view of the market but lacks the critical detail. Founders who focus only on this image risk missing the “ground truth.”
Seeing through the eyes of customers and competitors is a theoretical exercise limited by the fact that you can never be sure what your customers and competitors are up to.
The combination of all three views helps founders form an accurate picture of what is going on in their business and help them hone in on product/market fit.
Even with information from all three views, founders need to remember there will never be enough information to make a perfect decision.
Building an Information Culture
The most important element of data gathering is what to do with the information once you collect it. Customer information dissemination is a cornerstone of Lean and agile companies. This information, whether good or bad, must not be guarded like some precious commodity. Large company cultures reward executives who hoard knowledge or suppress bad news. In any of my companies, that is a firing offense.
All news, but especially bad news, needs to be shared, dissected, understood, and acted upon.
This means that understanding poor click-through rates, retention numbers and sales losses are more important than understanding sales wins; understanding why a competitor’s products are better is more important than rationalizing ways in which yours is still superior. Winning startups build a startup culture that reward not punish messengers of bad news.
- Three views of information: First-hand knowledge, “birds-eye” view, view from the eyes of customers and competitors
- Web startups can fall into the trap of confusing metrics, testing and surveys with Customer Interaction.
- Goal is to build an information culture to help you get to product/market fit.