Gathering real-world feedback from customers is a core concept of Customer Development as well as the Lean Startup. But what information to collect?
Only 57 Questions
Yesterday I got an email from an ex-student lamenting that only 2% of their selected early testers responded to their on-line survey. The survey said in part:
The survey has 57 questions, the last three of which are open ended, and should take about 20 minutes to complete. Please note that you must complete the entire survey once you begin. You cannot stop along the way and have your responses to that point saved.
If it wasn’t so sad it would be funny. I called the founder and noted that there are SAT tests that are shorter than the survey. When I asked him if he actually had personally left the building and talked to these potential customers, or even had gotten them on the phone, he sounded confused. “We’re a web startup, all our customers are on the web. Why can’t I just get them to give me the answers I need this way?”
Continual Data Flow
Customer Development suggests that founders have continual and timely customer, channel and market information.
Founders need three views of information to truly understand what is going on:
- First-hand knowledge
- A “birds-eye” view
- The view from the eyes of customers and competitors
First-hand knowledge
First-hand knowledge is “getting outside the building” and talking to potential or actual customers. Customer Development proposes that the best way to get customer data is through personal observation and experience—getting out from behind your desk and getting up close and personal with customers, competitors, and the market.
Web startups are at real disadvantage here as founders may confuse web metrics, A/B testing and on-line surveys as the entirety of first-hand knowledge – for most web business models they are not. In fact, this mistake can be a “going out of business” strategy. Metrics tell you that something is happening. A/B testing can tell you that one something is better than another. But neither can tell you why. And getting answers back from customers only with on-line surveys when you can’t watch their pupils dilate or hear the intonation of their voice is not something I’d build a business on.
Of course you need to collect metrics, do A/B testing and run online surveys. It’s just that without having founders “get outside the building”, you are missing a key point of Customer Development – the numeric data you collect may be blinding you to the fact that you’re more than likely working to optimize the wrong business model. Customer needs are non-deterministic.
“Birds-eye view”
The second picture founders need is a synthesized “birds-eye view” of the customer, market and competitive environment. You assemble this view by gathering information from a variety of sources: web sites, social media (Facebook, Twitter, blogs, et al) sales data, win/loss information, market research data (i.e. compete.com, Alexa, etc.,) competitive analyses, and so on. From this big-picture view, founders try to make sense of the shape of the market and the overall patterns in the unfolding competitive and customer situation. At the same time, they can gauge how well industry data and the actual sales match the company’s revenue and market-share expectations.
(Just remember that most market research firms are excellent at predicting the past. If they could predict the future, they’d be entrepreneurs.)
My test for how well you understand this “order of battle” is to hand the founder a marker, have them go up to the whiteboard and diagram the players in the market and where they fit. (Try it.)
See through the eyes of customers and competitors
The third view is of the action as seen through the eyes of customers and competitors. Put yourself in your customers’ and competitors’ shoes in order to deduce possible competitors’ moves and anticipate customer needs. In an existing market this is where you ask yourself, “If I were my own competitor and had its resources, what would my next move be?” In looking through the eyes of a customer, the question might be, “Why should I buy from this company versus the incumbent.” In a new or resegmented market, the questions might be “Why would more than a few early adopters use this app, web site or buy this product? How would I get my 90-year grandmother to understand and buy this product?”
Think of this technique as playing chess. You need to be looking at all the likely moves from both sides of the chessboard. What would we do if we were our competitors? How would we react? What would we be planning? After a while this type of role playing will become an integral part of everyone’s thinking and planning.
Putting it All Together
First-hand knowledge is clearly the most detailed and essential, but offers a narrow field of view. Founders who focus only on this information risk losing sight of the big picture.
The “birds-eye view” provides a view of the market but lacks the critical detail. Founders who focus only on this image risk missing the “ground truth.”
Seeing through the eyes of customers and competitors is a theoretical exercise limited by the fact that you can never be sure what your customers and competitors are up to.
The combination of all three views helps founders form an accurate picture of what is going on in their business and help them hone in on product/market fit.
Even with information from all three views, founders need to remember there will never be enough information to make a perfect decision.
Building an Information Culture
The most important element of data gathering is what to do with the information once you collect it. Customer information dissemination is a cornerstone of Lean and agile companies. This information, whether good or bad, must not be guarded like some precious commodity. Large company cultures reward executives who hoard knowledge or suppress bad news. In any of my companies, that is a firing offense.
All news, but especially bad news, needs to be shared, dissected, understood, and acted upon.
This means that understanding poor click-through rates, retention numbers and sales losses are more important than understanding sales wins; understanding why a competitor’s products are better is more important than rationalizing ways in which yours is still superior. Winning startups build a startup culture that reward not punish messengers of bad news.
Lessons Learned
- Three views of information: First-hand knowledge, “birds-eye” view, view from the eyes of customers and competitors
- Web startups can fall into the trap of confusing metrics, testing and surveys with Customer Interaction.
- Goal is to build an information culture to help you get to product/market fit.
Share this:
Filed under: Customer Development Manifesto, Marketing |
Talking to your customers directly is awesome. But, what is even better is to get a group of your customers to talk to you AND eachother.
We (at Ask My BrainTrust) believe business owners should always have THREE BrainTrusts:
1) Key team members in your org or your whole team if you are small
2) Key/Representative set of customers
3) Other founders/business owners.
Constantly collaborating with these three groups about the challenges you face is definitely going to increase the likelihood of your startup’s success.
“But, what is even better is to get a group of your customers to talk to you AND each other. ”
In the early market this is can cause problems when interviewing prospects: focus on one conversation at a time. Don’t let one prospect’s perspective who speaks first on a topic inadvertently anchor the group somewhere. Instead ask open ended questions and listen, prepared to be surprised.
Consider an appreciative inquiry approach to understand the customer’s operating reality.
Customer groups are effective for feature planning but more problematic in determining product/market fit in the early market.
By the way, I visited your website and have no idea who is behind it or why I should trust your firm with a repository of confidential conversations. You really should put up an about page and explain who the team is and why they are trustworthy.
[…] just read the latest from Lean Startup guru Steve Blank in which he talks about building an information culture. Lean manufacturing relies on transparency of the process so that any worker on the line can see […]
At what point in the process of our startup do we want to start getting interactive feedback from our target market? How much focus should we give to gathering customer preference while we are still in the inception phase of our idea?
As soon as you can clearly articulate your hypotheses about the customer’s problem you should get out of the building and start having serious conversations. Customer Development proceeds in parallel with product development and informs it.
One piece of paper with a prospect’s name and a few questions can communicate that you care about their perspective and have given some thought to making it a productive 10-20 minute conversation (if they want to talk longer you should let them, but you should be able to finish a short conversation in ten minutes or so).
Can say how much I really like this post. The last line about an information culture is so true.
We live in an information society, work with information technology, sometimes call ourselves information workers, yet never really pay heed to understanding and truly using information across our organizations. Too often “systems” or “applications” are implement to “share” information, but without an information culture, all those tools will be of little use.
I’ve blogged about this concept myself. I call it an Information Supply Chain.
http://onproductmanagement.net/2008/01/11/product-manager-vs-product-management-part-4/
[…] via Building a Company with Customer Data – Why Metrics Are Not Enough « Steve Blank. […]
[…] Building a Company with Customer Data – Why Metrics Are Not Enough « Steve Blank (tags: customer-development startup analytics) […]
[…] Blank had a great post today “Building a Company with Customer Data, Why Metrics Are Not Enough” that highlights the need–even for Web Startups–to get out of the BatCave and […]
As for interactive feedback from your target market, start early and never stop. Comparing customer (or potential customer) opinions year over year is the best way to move your business forward.
At bigpocket we are going to fallow Steve Blank model for our startup…
Far too many people spend thousands of pounds developing surveys that are produced online. Many of these are sent to people like ValueOffers, who pay the people to do them.
I will always choose to go out and interact.
[…] quote from a comment I left on Steve Blank’s blog As soon as you can clearly articulate your hypotheses about the customer’s problem you should get […]
[…] http://steveblank.com/2009/12/17/buildin… […]
[…] problem with metrics is that they don’t tell you why, only what (and, aside, possibly lead to neglecting brand value – if you can afford to care about that). You […]
[…] not coincide with your own. Steve Blank reminds us that Customer Development is not just collecting web metrics and it’s not about focus groups. I’ve written before that Customer’s own the […]
[…] soon for us to decide if a hypothesis is valid or not, but when to know that enough is enough. ( Building a Company with Customer Data by @sgblank) However, this interviewing process has helped us understand our customers better. Our […]
[…] Steve Blank says, “get outside the building” and test your assumptions for success, but do not focus exclusively on product and forget […]
[…] Building a Company with Customer Data – Why Metrics Are Not Enough […]
[…] first if you’re not a natural born sales person. ‘I don’t have time’ or ‘I would rather assess behavior through metrics‘ reasons will only push the answer out further. A simple 15 minute conversation to talk about […]
[…] Building a Company with Customer Data – Why Metrics Are Not Enough écrit par Steve Blank (@SgBlank). Il est très connu pour être l’auteur de « The Four Steps to the Epiphany » très bon bouquin qui prône le « Customer Development » (le livre est très dense à lire, Venture Hacks a fait un résumé assez complet) Dans l’article, Steve Blank rappelle que les metrics ne font pas tout et qu’il faut également parler à ses utilisateurs, les comprendre pour développer son produit de manière efficace. Toujours bon d’avoir un article qui équilibre un peu enthousiasme général pour les metrics… […]
[…] blog amplifies the ideas in his book, e.g., why you need to get out of the building to interview users, attributes of a scalable startup, the definition of a business model, and why startups should […]
Unfortunately, I’m inundated with requests to ask customers – since the founder doesn’t want to do it – trivial things. Would you prefer this sort order or that one? Don’t you want to sort products by price? Why don’t you? When they don’t hear what they want: dejection.
When I point out that such trivialities can, indeed, be found in rather reliable survey data in the industry, that those customer preferences have been known for years, and they’ve been stable, dejection. They want to go out and ask customers about every trivial thing for which they happen to lack knowledge – but it is never good enough to stand on the shoulders of the real scientists who came before them. Rather, we would like to do a survey with limited data, with a skewed sample. Oh sure, there’s a national survey with 1000 respondents, decent confidence level. But, because they haven’t “gotten out of the building”and asked customers directly, that knowledge is meaningless to them.
. Do customers like the box tilted this way, or that way? Trivial! But they want to ask it anyway, regardless as to how many times you explain how it is that, when you put garbage in, you get garbage back!
As with all decent management ideas, this one in the hands of actual people .. is turned into banalities.
Is there a good way to compile the surveys without creating e.g. 100 internal Word/Google documents? That’s currently what I’m having difficulty with – what to use for this especially if we do go back and update the information for them during the customer discovery stage.