I am always surprised when critics complain that the Lean Startup’s Build, Measure, Learn approach is nothing more than “throwing incomplete products out of the building to see if they work.”
Unfortunately the Build, Measure, Learn diagram is the cause of that confusion. At first glance it seems like a fire-ready-aim process.
It’s time to update Build, Measure, Learn to what we now know is the best way to build Lean startups.
Here’s how.
Build, Measure, Learn sounds pretty simple. Build a product, get it into the real world, measure customers’ reactions and behaviors, learn from this, and use what you’ve learned to build something better. Repeat, learning whether to iterate, pivot or restart until you have something that customers love.
Waterfall Development
While it sounds simple, the Build Measure Learn approach to product development is a radical improvement over the traditional Waterfall model used throughout the 20th century to build and ship products. Back then, an entrepreneur used a serial product development process that proceeded step-by-step with little if any customer feedback. Founders assumed they understood customer problems/needs, wrote engineering requirements documents, designed the product, implemented/built the hardware/software, verified that it worked by testing it, and then introduced the product to customers in a formal coming out called first customer ship.
Waterfall Development was all about execution of the requirements document. While early versions of the product were shared with customers in Alpha and Beta Testing, the goal of early customer access to the product was to uncover bugs not to provide feedback on features or usability. 
Only after shipping and attempting to sell the product would a startup hear any substantive feedback from customers. And too often, after months or even years of development, entrepreneurs learned the hard way that customers were not buying their product because they did not need or want most of its features.
It often took companies three tries to get products right. Version 1 was built without customer feedback, and before version 1 was complete work had already started on version 2 so it took till version 3 before the customer was really heard (e.g. Microsoft Windows 3.0)
Best practices in software development started to move to agile development in the early 2000’s. This methodology improved on waterfall by building software iteratively and involving the customer. But it lacked a framework for testing all commercialization hypotheses outside of the building. With Agile you could end up satisfying every feature a customer asked for and still go out of business.
Then came the Build-Measure-learn focus of the Lean Startup.
Build-Measure-Learn
The goal of Build-Measure-Learn is not to build a final product to ship or even to build a prototype of a product, but to maximize learning through incremental and iterative engineering. (Learning could be about product features, customer needs, the right pricing and distribution channel, etc.) The “build” step refers to building a minimal viable product (an MVP.) It’s critical to understand that an MVP is not the product with fewer features. Rather it is the simplest thing that you can show to customers to get the most learning at that point in time. Early on in a startup, an MVP could simply be a PowerPoint slide, wireframe, clay model, sample data set, etc. Each time you build an MVP you also define what you are trying to test/measure. Later, as more is learned, the MVP’s go from low-fidelity to higher fidelity, but the goal continues to be to maximize learning not to build a beta/fully featured prototype of the product.
A major improvement over Waterfall development, Build Measure Learn lets startups be fast, agile and efficient.
The three-circle diagram of Build Measure Learn is good approximation of the process. Unfortunately, using the word “build” first often confuses people. The diagram does seem to imply build stuff and throw it out of the building. A more detailed version of the Build Measure Learn diagram helps to clarify the meaning by adding three more elements: Ideas-Build-Code-Measure-Data-Learn.
The five-part version of the Build Measure Learn diagram helps us see that the real intent of building is to test “ideas” – not just to build blindly without an objective. The circle labeled “code” could easily be labeled “build hardware” or “build artificial genome.” The circle labeled “data” indicates that after we measure our experiments we’ll use the data to further refine our learning. And the new learning will influence our next ideas. So we can see that the goal of Build-Measure-Learn isn’t just to build things, the goal is to build things to validate or invalidate the initial idea.
The focus on testing specific ideas counters the concern that build-measure-learn is just throwing things against the wall and see if they work.
But it’s still not good enough. We can now do better.
Start With Hypotheses
What Build-Measure-Learn misses is that new ventures (both startups and new ideas in existing companies) don’t start with “ideas”, they start with hypotheses (a fancy word for guesses.) It’s important to understand that the words “idea ” and “hypotheses” mean two very different things. For most innovators the word “idea” conjures up an insight that immediately requires a plan to bring it to fruition. In contrast, a hypothesis means we have an educated guess that requires experimentation and data to validate or invalidate.
These hypotheses span the gamut from who’s the customer(s), to what’s the value proposition (product/service features), pricing, distribution channel, and demand creation (customer acquisition, activation, retention, etc.)
That the Lean Startup begins with acknowledging that your idea is simply a series of untested hypotheses is a big idea. It’s a really big idea because what you build needs to match the hypothesis you want to test.
The minimum viable product you’ll need to build to find the right customers is different from the minimum viable product you need for testing pricing, which is different from an MVP you would build to test specific product features. And all of these hypotheses (and minimal viable products) change over time as you learn more. So instead of Build-Measure-Learn, the diagram for building minimal viable products in a Lean Startup looks like Hypotheses – Experiments – Tests – Insights.
Generating Hypotheses
Using this new Hypotheses – Experiments – Tests – Insights diagram the question then becomes, “What hypotheses should I test?” Luckily Alexander Osterwalder’s business model canvas presents a visual overview of the nine components of a business on one page. They are:
- value proposition, product/service the company offers (along with its benefits to customers)
- customer segments, such as users and payers or moms or teens
- distribution channels to reach customers and offer them the value proposition
- customer relationships to create demand
- revenue streams generated by the value proposition(s)
- activities necessary to implement the business model
- resources needed to make the activities possible
- partners 3rd parties needed to make the activities possible
- cost structure resulting from the business model
And it brings us to the definition of a startup: A startup is a temporary organization designed to search for a repeatable and scalable business model.
Testing Hypotheses
And once these hypotheses fill the Business Model Canvas, how does an entrepreneur go about testing them? If you’re a scientist the answer is easy: you run experiments. The same is true in a Lean Startup. (The National Science Foundation described the Lean LaunchPad class as the scientific method for entrepreneurship.)
The Customer Development process is a simple methodology for taking new venture hypotheses and getting out of the building to test them. Customer discovery captures the founders’ vision and turns it into a series of business model hypotheses. Then it develops a series of experiments to test customer reactions to those hypotheses and turn them into facts. The experiments can be a series of questions you ask customers but most often a minimal viable product to help potential customers understand your solution accompanies the questions.
So another big idea here is startups are not building minimal viable products to build a prototype. They are building minimal viable products to learn the most they can.
Finally, the goal of designing these experiments and minimal viable products is not to get data. The data is not the endpoint. Anyone can collect data. Focus groups collect data. This is not a focus group. The goal is to get insight. The entire point of getting out of the building is to inform the founder’s vision. The insight may come from analyzing customer responses, but it also may come from ignoring the data or realizing that what you are describing is a new, disruptive market that doesn’t exist, and that you need to change your experiments from measuring specifics to inventing the future.
Lessons Learned
- Build, Measure, Learn is a great improvement over Waterfall product development and provided the framework to truly join the customer to agile development
- However, emphasizing “Build” or “Ideas” as the first step misses the key insight about a Lean Startup – you are starting with hypotheses to be tested and are searching for repeatable and scalable business model
- Hypotheses, Experiments, Test, Insights better represents the Lean startup process:
- Use the Business Model Canvas to frame hypotheses, Customer Development to get out of the building to test hypotheses, and Agile Engineering to build the product iteratively and incrementally
Share this:
Filed under: Business Model versus Business Plan, Corporate/Gov't Innovation, Customer Development, Lean LaunchPad, Teaching |
worth reading !
Great insights Steve. It’s like peeling the union. Every time you dig deeper in your model, it provides more insight on its workings. The magic of the model remains the circular motion (instead of the waterfall). To me Build-Measure-Learn still is the starting point, but I will definitely add the others as cards to the Lean Startup stack. Thanks for sharing!
Thanks Steve, this has been such a problem that I have seen over and over again in the startup community.
Many founders just build something and measure something and learn (of course, they do) and think it is a lean approach. Even worse, one group I know just changed beta to MVP in “alpha, beta, v1.0” and considered it to be a Lean Startup approach.
When I present “Lean Startup with Customer Development” I have been using a Hypothesis-Build-Measure loop, where founders formulate an hypothesis, build something to experiment with, and measure to collect data to validate (or not) the hypothesis.
Unfortunately, it seems to me that neither Hypothesis-Build-Measure nor Hypotheses-Experiments-Test-Insights rolls off the tongue as nicely as Build-Measure-Learn, so I think it may be a bit of a battle to win mindshare 🙂
Steve, this is the most complete and succinct explanation I have ever heard you give. Thank you.
Steve – this is an excellent post/update! I would love to have you come back to speak to our veteran entrepreneurs – we actually have grown quite a bit, and with this being Military AppreciationMonth – we have alot going on around the country. On May 26th – not sure if in your travels youcan make it but we have events titled “Education for Veteran Entrepreneurs” – in SF, Boston,and NYC. Another possibility will be at our hackathon at Facebook in August. Will you pleasecome speak to a group of VetsinTech veteran entrepreneurs? Let me know what might workthis month, August, and November are “Entrepreneur” quarterly months at VetsinTech. Best,Katherine
Reblogged this on Chillik Media.
I applaud Steve Blank for introducing the “HETI (Hypothesize-Experiment-Test-Insight) Loop” as an improvement of Eric Ries’s BML (Build-Measure-Learn) Loop. However, there are still some missing parts to be added to the HETI Loop. I’m assuming that the HETI Loop as well as BML Loop are summary descriptions of the process of Scientific Problem Solving or “Scientific Method.” Based on that assumption, scientists don’t just conjure or postulate hypotheses. Based on my experience, scientists first “Observe” and then “Think (Explore)” before “Hypothesizing”, “Experimenting”, and “Reflecting (Insights; Lessons Learned).” Using the first letter of each activity, this process of scientific problem solving can be summarized using the acronym, OTHER: Observe-Think-Hypothesize-Experiment-Reflect. Scientists without exception – from Newton through Einstein and Feynman and beyond – inexpensively and rapidly use the OTHER Loop to come with knowledge tools including scientific laws and theories. Like I said before, Steve’s HETI Loop misses some parts, that is, the “O (Observe)” and “T (Think).” The complete loop for scientific problem solving can be summarized using the acronym, OTHETI; this acronym sounds like a mouthful. Admittedly, I prefer the OTHER Loop which not only is ear-friendly but also has the connotation of doing things creatively by: Observe different; Think different; Hypothesize different; Experiment different; Reflect different. Although learning are problem solving are two sides of the same coin, focusing on a paradigm of “Emergent and Deliberate Problem Solving” rather than “Organizational Learning” can resolve most of the apparent conflict in Lean Startup theory, processes, and tools. I’m looking forward to a simpler visual process of describing scientific problem solving in the Lean Startup community.
Great explanation on going deeper into the Lean StartUp process. However, the challenge is, too many people just want a checklist of things to do in order to be successful – “If I follow steps 1-10, then I’ll be successful”. To really understand the underlying concepts of the Lean StartUp, one must first understand the foundational principles of Lean Thinking – Value, Value Stream, Flow, Pull, and Perfection.
It’s all about continuously trying to improve the product, service or process you are working on. It’s as much a mindset as it is a formal process.
Great points Glenn, but just to be clear for people who are reading, it’s not the development of product/service/process that one is selling that they should be trying to be lean about (value, value stream, flow, pull etc.), although that is ultimately important too. During startup stage, it is the search for the repeatable, scalable, and viable business model that needs to be lean, i.e. don’t waste time and resources on anything that doesn’t help with validate learning. As is often said, “your product isn’t the product, the product is the business model.” All IMHO, of course.
This sounds exactly like the Deming Cycle, minus the first phase of planning.
https://en.wikipedia.org/wiki/PDCA
In truth Deming did not endorse PDCA but PDSA (S for Study) because PDCA puts too much focus on control (post control) and indeed it was Shewhart cycle who created it from scientific process P for Plan is actually the Hypotheses stage not about doing fancy planning 🙂
Thanks, Steve for continuing to support the original Business Model Canvas as a framework for startup experiments. I try to steer Startups using other Canvas variations into the BMC as a ” lingua franca” for extracting the needed information. Much of the related experimental terminology uses overloaded words with multiple meanings. Some of the replies already posted here have interesting approaches to help resolve some of those issues. Mitch Spiegel
Have you considered modeling something based on the full OODA loop?
Thanks for touching on the topic. Of late I have been sharing Design related topics to the entrepreneurs on my meetup group. I am starting to feel there is some value of directly using the principles of Co Design, Service Design, Eco Deisng etc at the business level. ie how we organise the entire business – instead of organising a business around functions.
The lean startup, BMC approach are a step in this direction, but donot seem to cover the strategy / operation interplay needed to accurately realise the vision (Clausewitz). eg operational timelines, risks, complexities, stages of the uncertainty funnel, etc. seem to not be covered.
Be great to have some of your thoughts on this.
Reblogged this on Alvaro's blog and commented:
Latest on Lean Start Up
Really insightful! And I also find much easier to live with a ‘failure’, if I see it as the result of a wrong hypothesis instead that the consequence of a bad idea.
Excellent articulation of MVP .. i.e., It is not about building ‘stuff’ (With so called bare minimum features) and throwing it on the other side of wall to capture the market , rather it is a ‘simplest’ thing that we can show to potential customers to get the most learning at that point in time.
In a nutshell, Early learning is the key to success irrespective of the size of the business. (Very much applicable wherever the cost of investment is more along with higher risk/uncertainty)
Thanks a lot Steve for your thoughts on the Lean principles.
Great update, thanks Steve. Since this is a discussion about better terminology to reflect customer development ideas, one question I’ve had about the terminology you use – why call these people who are the target of a founder’s experiments “customers”? They have usually neither bought nor received anything during early stages, and they have usually not committed to buying anything. At best they are “prospects” or “target audience”.
I see many founders who swear by the Lean Startup methodology forget the difference between a real “customer” and a “prospect”, which can be quite fatal IMHO. Prospects may at best have expressed interest in the problem/solution but can take ages to convert to customer/user, and have low conversion rate in a majority of cases.
Just a thought…
Thanks.
Ashmi,
I completely agree. It’s sloppy terminology on my part.
In fact, I remind startups I advise of exactly this point – they’re prospects or target audience until they buy or activate.
Thanks for the reminder.
steve
Hey Steve, thanks for the article. Again, great content.
Question: You wrote:
“The five-part version of the Build Measure Learn diagram helps us see that the real intent of building is to test “ideas” – not just to build blindly without an objective.”
I’m seeing six parts on the diagram. Was this a typo, or am I missing something? If it was intentional, would you mind explaining, which five parts?
Thanks,
sanjayananda
Really insightful, thanks for this Steve.
Thanks, Steve for continuing to support the initial Business Model Canvas as a structure for start-up experiments. I attempt to guide Startups utilizing other Canvas variations into the BMC as a” lingua franca” for drawing out the required info.
What happens when the core is managing their own new business with their usual slow way, because of powerful feud thinking middle age managers. You need, according to the strategic importance of the startup, dedicated teams, at least 50% of the time (2 projects) or even 100% of time. Core businesses can´t do effective startups and fluid business models building (Time thinking)