New Lessons Learned from Berkeley & Stanford Lean LaunchPad Classes

Our Stanford and Berkeley Lean LaunchPad classes are over for this year, and as usual we learned as much from teaching the teams as the teams did from us.

Here are a few of the Lessons Learned from these two classes.

Have each team talk to 10 customers before the class starts
Each year we learn how to move more of the Lean LaunchPad class logistics outside our classroom so teams have more time for in-class learning.

A few years ago, we moved the formation of teams’ to before the class started and in doing so, saved a week of what normally be an in-class time activity. To make this happen, we hold three “information sessions” two weeks apart before the class starts. In these “info sessions” we describe the purpose of the class, and then let students mix, meet and form teams. During this pre-class time we share a Google doc where students who have ideas can find other team members, and students without an idea can find a team that matches their skills and interests. Application and admission to the class is by interview with a fully formed team.

Info session announcement

Info session announcement

The next thing we learned is to make applying to the class an integral part of the learning process. Teams apply by filling out both a business model canvas and a “competitive petal slide.” Having the teams do this accomplishes three things.  First it forces the students to read and understand “what’s a business model canvasbefore they even come to class.

Freewire application

Team Application: Business Model Canvas

Second, the competitive slide enforces a modicum of due diligence on the product and market. (We got tired of knowing more about each team’s market by doing a Google search as they presented. Now it’s their job.)

Farmsense competive slide

Team Application: Competitive “Petal” Slide

Finally, having teams spend time on the canvas and competition as part of the application process saves weeks of what would normally be an in-class activity (and as a bonus gives the team a heads-up about the difficulty of the class and shows whether they’re serious about the class or just shopping.)

This year we learned to raise the bar once again.  Could we get the teams to come into class having already talked to 10 customers? Instead of using the first class to have teams just present their business model canvas, this time the team’s first presentation would be about what they learned outside the building about their value proposition. (We pointed them to our tutorials on customer discovery and how to conduct customer interviews but didn’t expect them to be experts on week 1.)

SignUP week 1

1st week team title slide – 11 interviews before class started

We did an A/B test by requiring our teams in one school do this while not requiring it for the teams in the other school. The result?  Teams that had to talk to customers before the class hit the ground running. There was a substantive difference in team trajectory and velocity that continued throughout the quarter. The amount of learning between the two felt quite different. While there may have been other factors (team selection bias, team make up, etc.), we’ll now make this an integral part of all the classes.

Have each team put the number of Mentor interactions on their weekly title slide
The second innovation this year involved mentors. Each team is assigned a mentor as a coach. We’ve been trying to figure out how to make mentor engagements with their teams a regular rather an adhoc activity. While we have required the teams to add a summary of any mentor interaction to their LaunchPad Central narrative, we felt we didn’t have sufficient high-level visibility for these essential interactions.

GiveModo Class 8

But this year, a seemingly minor change to the teams’ weekly cover slide had an important impact. As teams present each week, their cover slides show the number of customers interviewed for that week (>10) along with the cumulative customers interviewed. This year we added one more metric for their cover slides– the number of mentor interactions for that week (>1) along with the cumulative number of mentor interactions.

This enhanced the visibility of the teams interaction (or lack of) with their mentors and allowed us to proactively intervene early if there wasn’t sufficient interaction.

Here are a few of the Final Presentations (see here for all of them)

If you can’t see the presentation above, click here

If you can’t see the video above, click here

If you can’t see the presentation above, click here

If you can’t see the video above, click here

If you can’t see the presentation above, click here

If you can’t see the video above, click here

If you can’t see the presentation above, click here

If you can’t see the video above, click here

If you can’t see the presentation above, click here

If you can’t see the video above, click here

If you can’t see the presentation above, click here

Listen to the blog post here


Download the podcast here

Get the Heck Out of the Building in Founder’s School: Part 2

With a ~$2 billion endowment the Kauffman Foundation is the largest non-profit focused on entrepreneurship in the world. Giving away $80 million to every year (~$25 million to entrepreneurial causes) makes Kauffman the dominant player in the entrepreneurship space.

Kauffman launched Founders School - a new education series to help entrepreneurs develop their businesses during the startup stage by highlighting how startups are different from big companies.

In January 2014 Part 1 of the “Startups” section of Founders School went online.

Now you can watch Part 2 “The Lean Approach“.

Founders School

This group of six videos provides an overview of how to successfully do Customer Discovery. You’ll learn how to:

  • get to know your customers
  • devise ways to test your hypotheses
  • glean insights from what you learn outside the building
  • get, keep and grow customers

As in the first part of this series, I’m in good company – I’m joined in Founders School by Noam Wasserman of Harvard teaching Founder’s Dilemmas, Craig Wortmann University of Chicago covering Entrepreneurial Selling, Peter McDermott helping understand Intellectual Property, and Nathan Gold offering how to give Powerful Presentations.

These videos are not only great tutorials for founders but also provide educators with another source of well produced and curated resources.

These “Startup and The Lean Approach” videos are a great general purpose companion to my “How to Build a Startup” lectures on Udacity.

And you get a tour of my living room and office…

Introduction, for Part 2 is here

Module 1, The Lean Method

  • 0:50: There are No Facts Inside Your Building — Get Outside
  • 1:28: Using the Business Model Canvas
  • 1:49: Use Customer Development to Test Your Hypotheses
  • 2:44: What is a Pivot?
  • 4:24: No Business Plan Survives First Contact with Customers

Module 2, Getting Out of the Building: Customer Development

  • 0:24: What is Customer Development?
  • 1:09: How Do You Start the Customer Development Process?
  • 1:36: Customer Discovery is a Series of Conversations
  • 2:05: The Founder and Customer Development
  • 3:16: Real World Example of Customer Development

Module 3, Customer Development Data

  • 0:31: Designing Experiments to Test Hypotheses
  • 0:48: Doing Customer Discovery Without Collecting Data is a Sin
  • 1:06: Insight is Key
  • 1:49: Why Accountants Don’t Run Startups

Module 4, Minimum Viable Products

  • 0:18: What is a Minimum Viable Product?
  • 0:38: What to Test, Why to Test and How to Test
  • 2:05: You’re Not Building a Product … You’re Getting Customer Feedback
  • 2:53: Use MVPs to Run Experiments
  • 4:15: Real World Example of an MVP

Module 5, Customer Acquisition and Archetypes

  • 0:47: Get, Keep and Grow Customers
  • 1:00: Create Customer Demand
  • 1:46: Customer Archetypes: Getting to Know Your Customers
  • 3:35: Matching Archetypes to Acquisition
  • 5:28: Growing Customers: The Lifetime Value
  • 7:35: The Biggest Mistake in Customer Acquisition

Listen to the blog post here


Download the podcast here

Beyond the Lemonade Stand: How to Teach High School Students Lean Startups

While the Lean LaunchPad class has been adopted by Universities and the National Science Foundation, the question we get is, “Can students in K-12 handle an experiential entrepreneurship class?” Hawken School has now given us an answer. Their seniors just completed the school’s first-ever 3-credit semester program in evidence-based entrepreneurship. Students are fully immersed in real-world learning during the 12-week Entrepreneurial Studies course.

Here’s what Doris Korda Associate Head of School and Tim Desmond, Assistant Director of Entrepreneurial Studies did, and how they did it.

Teaching students to think like entrepreneurs not accountants
We realized that past K-12 Entrepreneurial classes taught students “the lemonade stand” version of how to start a company: 1) come up with an idea, 2) execute the idea, 3) do the accounting (revenue, costs, etc.).

We wanted to teach our students how to think like entrepreneurs not accountants. Therefore we needed them to think and learn about two parts of a startup; 1) ideation - how to create new ideas and 2) customer development – how do they test the validity of their idea (is it the right product, customer, channel, pricing, etc.)

Our first insight was that if broke the class in half, and separated ideation from customer development, our students would understand 1) that an idea is not the company, 2) almost all initial ideas are wrong. 

So rather than starting with their own business ideas, we decided to first give our students experience doing customer discovery on someone else’s idea.  Then in the second half of our semester let our students come up with their own ideas and then run the customer discovery process on their own product.

Customer Discovery in the Real World
Our students first worked with two local startups who agreed to be their clients, on real-problems. These two startups had problems they could not solve on their own due to lack of resources—time, people, money. The startups and the teaching team crafted a challenge for the kids to tackle using the Customer Development methodology, Lean Launchpad tools and the business model canvas.

For their first startup, we chose a 3-year old funded company that was working to refine its customer segment and channel for its physical product.  For the second startup, we chose a year-old web/mobile startup whose market is college bound teens, with a founder who had skipped the initial customer validation process. These two startups served as the students’ introduction to customer development methodology. Each student team conducted over 100 detailed interviews in an effort to develop results for each client.

Hawken students doing Customer Discovery in a mall

Hawken students practicing Customer Discovery in a mall

Because these were high school kids with, for the first time, a real business relying on them, this portion of the class shook them so badly they couldn’t move from their seats–literally. All their hard-wired school habits turned to dust as the kids realized their school tools were useless: there were no solution keys, no rubrics, no answers in the back of the book. Feeling the pressure, after 3 wasted days, one student on one team finally convinced her team they needed to get out of the building, like in Steve’s video. That’s when everything changed.

Knowing they had 3 weeks before presenting to the company co-founders, the kids felt intensity like no traditional classroom could generate. The pace and uncertainty of the class picked up and never let up from that point.

The second startup, because it was in an earlier stage and more complicated than the first, had the kids going even deeper into the 9 blocks of the business model canvas. Because the students’ customer development narratives revealed the client’s user interface was problematic, students with no programming experience began redesigning the user interface using Lean UX principles, tools and strategies.

Yet students were still afraid to rethink the client’s product: “if we tell [her] how to unclutter the interface, it will cost her a ton of money and she’ll be mad at us.”  One of their mentors, a professional UX designer, encouraged the kids, “You have the facts. You’ve developed archetype and narratives from a ton of real customer interviews. You need to propose disruptive solutions. What can you propose that will solve the customers’ problems and set this product apart in a meaningful way?” This was a huge learning moment. Their final presentations were substantive and evidence-based.

Starting Your Own Company
For the last three weeks of class, the 16 students came up with their own business ideas that they pitched to their peers; 4 of these ideas moved forward in the quest for viable business models. Interestingly the four founders of teams whose ideas “won” argued over which team would get the students with the most advanced technical skills. By the end of the half hour, students were suggesting that our school should offer more programming earlier in school and throughout this course.

We assigned one mentor to each of the four teams and used LaunchPad Central to hold all the details together, including hundreds of customer interviews, narratives, days in the life, archetypes, storyboards, user interfaces, live presentations and tons of often painful feedback.

Teams spent 3 weeks getting out of the classroom and iterated and pivoted, put to use the visual and lean tools along with all they learned from Steve Blank’s Udacity lectures, then pulled together enough data and crafted compelling stories for their final Shark-Tank style presentations. They presented to local sharks from four local accelerators—Bizdom, JumpStart, FlashStarts and LaunchHouse (which runs the country’s first kid launched/kid run accelerator for kids, called LightHouse).

Hawken students pitching the local "Sharks"

Hawken students pitching the local “Sharks”

Having practiced negotiating terms, students calculated their companies’ valuations, ranging from 50k- 300k, and wrestled with the sharks over equity. Sharks, in turn, argued with one another and even attempted to form syndication in one instance.  At the close of the presentations, two teams were invited to apply for funding through local accelerators. The semester concluded with pizza and ice cream.

Pioneers are the Ones With Arrows in Their Backs
Trying to fit an Entrepreneurial Studies course into a college prep high school outside of Silicon Valley is an interesting challenge.

Being a pioneer means that there’s nothing familiar about this for parents, students and our administration. Hawken is exactly the right school to do this, but still our high school students and parents and other teachers are steeped in more traditional classes and subjects, college placement-related pressures, graduation requirements, AP courses, grades, etc.

Creating it feels a lot like building something totally new inside an existing business. But the course was spectacular for its students in ways that no other course is, so we’re getting money and institutional support for growing it. We’re learning a lot as we go…we’re figuring it out.

Summary
The Entrepreneurial Studies course serves as a vehicle for the school to realize its mission — forward-focused preparation for the real world through development of character and intellect.  The 16 seniors who just completed the first Entrepreneurial Studies course told us that it was different from anything they have ever done in school – all the learning was active and all the work was collaborative and team-oriented. In evaluations they explained the biggest lessons they learned were often about themselves and how they handled failure, their character and their own strengths and weaknesses.

From one senior: “For the first time I am working because I care. Not just for a grade.”

Lessons Learned:

  • Students work harder, better and deeper when the stakes are real
  • Working for local startups gives them a great way to quickly gain business and life experience alongside customer development experience
  • Working for local startups creates real world intensity and urgency in the course
  • Kids freak out, get paralyzed and waste time doing so. It’s all part of the learning process
  • The learning and growth of how to work well on a team is reason enough for students to enroll in Lean Launch Pad
  • We never anticipated the amount of learning that happened here
  • Even at a very progressive school, we are breaking new ground and challenging all the traditions and biases of regular school

This June, Hawken School is holding an Educator’s Workshop for middle and high school educators who want to build or grow their own LLP-based programs.

Listen to the post here


Download the podcast here

Is This Startup Ready For Investment?

Since 2005 startup accelerators have provided cohorts of startups with mentoring, pitch practice and product focus. However, accelerator Demo Days are a combination of graduation ceremony and pitch contest, with the uncomfortable feel of a swimsuit competition. Other than “I’ll know it when I see it”, there’s no formal way for an investor attending Demo Day to assess project maturity or quantify risks. Other than measuring engineering progress, there’s no standard language to communicate progress.

Corporations running internal incubators face many of the same selection issues as startup investors, plus they must grapple with the issues of integrating new ideas into existing P&L-driven functions or business units.

What’s been missing for everyone is:

  • a common language for investors to communicate objectives to startups
  • a language corporate innovation groups can use to communicate to business units and finance
  • data that investors, accelerators and incubators can use to inform selection

While it doesn’t eliminate great investor judgment, pattern recognition skills and mentoring, we’ve developed an Investment Readiness Level tool that fills in these missing pieces.

—-

Investment Readiness Level (IRL) for Corporations and Investors
The startups in our Lean LaunchPad classes and the NSF I-Corps incubator use LaunchPad Central to collect a continuous stream of data across all the teams. Over 10 weeks each team gets out of the building talking to 100 customers to test their hypotheses across all 9 boxes in the business model canvas.

We track each team’s progress as they test their business model hypotheses. We collect the complete narrative of what they discovered talking to customers as well as aggregate interviews, hypotheses to test, invalidated hypotheses and mentor and instructor engagements. This data gives innovation managers and investors a feel for the evidence and trajectory of the cohort as a whole and a top-level view of each teams progress. The software rolls all the data into an Investment Readiness Level score.

(Take a quick read of the post on the Investment Readiness Level – it’s short. Or watch the video here.)

The Power of the Investment Readiness Level: Different Metrics for Different Industry Segments
Recently we ran a Lean LaunchPad for Life Sciences class with 26 teams of clinicians and researchers at UCSF.  The teams developed businesses in 4 different areas– therapeutics, diagnostics, medical devices and digital health.  To understand the power of this tool, look at how the VC overseeing each market segment modified the Investment Readiness Level so that it reflected metrics relevant to their particular industry.

Medical Devices
Allan May of Life Science Angels modified the standard Investment Readiness Level to include metrics that were specific for medical device startups. These included; identification of a compelling clinical need, large enough market, intellectual property, regulatory issues, and reimbursement, and whether there was a plausible exit.

In the pictures below, note that all the thermometers are visual proxies for the more detailed evaluation criteria that lie behind them.

Device IRL

Investment Readiness Level for Medical Devices

You can watch the entire presentation here

Therapeutics
Karl Handelsman of CMEA Capital modified the standard Investment Readiness Level (IRL) for teams developing therapeutics to include identifying clinical problems, and agreeing on a timeline to pre-clinical and clinical data, cost and value of data points, what quality data to deliver to a company, and building a Key Opinion Leader (KOL) network. The heart of the therapeutics IRL also required “Proof of relevance” – was there a path to revenues fully articulated, an operational plan defined. Finally, did the team understand the key therapeutic liabilities, have data proving on-target activity and evidence of a therapeutic effect.

Therapeutics IRL

You can see the entire presentation here

Digital Health
For teams developing Digital Health solutions, Abhas Gupta of MDV noted that the Investment Readiness Level was closest to the standard web/mobile/cloud model with the addition of reimbursement and technical validation.

Digital Health

Diagnostics
Todd Morrill wanted teams developing Diagnostics to have a reimbursement strategy fully documented, the necessary IP in place, regulation and technical validation (clinical trial) regime understood and described and the cost structure and financing needs well documented.

Diagnostics IRL

You can see the entire presentation here

For their final presentations, each team explained how they tested and validated their business model (value proposition, customer segment, channel, customer relationships, revenue, costs, activities, resources and partners.) But they also scored themselves using the Investment Readiness Level criteria for their  market. After the teams reported the results of their self-evaluation, the  VC’s then told them how they actually scored.  We were fascinated to see that the team scores and the VC scores were almost the same.

Lessons Learned

  • The Investment Readiness Level provides a “how are we doing” set of metrics
  • It also creates a common language and metrics that investors, corporate innovation groups and entrepreneurs can share
  • It’s flexible enough to be modified for industry-specific business models
  • It’s part of a much larger suite of tools for those who manage corporate innovation, accelerators and incubators

Listen to the blog post here


Download the podcast here

How to Be Smarter than Your Investors – Continuous Customer Discovery

Teams that build continuous customer discovery into their DNA will become smarter than their investors, and build more successful companies.

Awhile back I blogged about Ashwin, one of my ex-students wanted to raise a seed round to build Unmanned Aerial Vehicles (drones) with a Hyper-spectral camera and fly it over farm fields collecting hyper-spectral images. These images, when processed with his company’s proprietary algorithms, would be able to tell farmers how healthy their plants were, whether there were diseases or bugs, whether there was enough fertilizer, and enough water.

(When computers, GPS and measurement meet farming, the category is called “precision agriculture.” I see at least one or two startup teams a year in this space.)Optimized water and fertilizer

At the time I pointed out to Ashwin that his minimum viable product was actionable data to farmers and not the drone. I suggested that to validate their minimum viable product it would be much cheaper to rent a camera and plane or helicopter, and fly over the farmers field, hand process the data and see if that’s the information farmers would pay for. And that they could do that in a day or two, for a tenth of the money they were looking for.

Walnut orchard

(Take a quick read of the original post here)

Fast forward a few months and Ashwin and I had coffee to go over what his company Ceres Imaging had learned. I wondered if he was still in the drone business, and if not, what had become the current Minimum Viable Product.

It was one of those great meetings where all I could do was smile: 1) Ashwin and the Ceres team had learned something that was impossible to know from inside their building, 2) they got much smarter than me.

Crop Dusters
Even though the Ceres Imaging founders initially wanted to build drones, talking to potential customers convinced them that as I predicted, the farmers couldn’t care less how the company acquired the data. But the farmers told them something that they (nor I) had never even considered – crop dusters (fancy word for them are “aerial applicators”) fly over farm fields all the time (to spray pesticides.)

They found that there are ~1,400 of these aerial applicator businesses in the U.S. with ~2,800 planes covering farms in 44 states. Ashwin said their big “aha moment” was when they realized that they could use these crop dusting planes to mount their hyperspectral cameras on. This is a big idea. They didn’t need drones at all.

If you can’t see the video above click here

Local crop dusters meant they could hire existing planes and simply attach their Hyper-spectral camera to any crop dusting plane. This meant that Ceres didn’t need to build an aerial infrastructure – it already existed. All of sudden what was an additional engineering and development effort now became a small, variable cost. As a bonus it meant the 1,400 aerial applicator companies could be a potential distribution channel partner.

Local Crop Dusters

The Ceres Imaging Minimum Viable Product was now an imaging system on a cropdusting plane generating data for high value Tree Crops. Their proprietary value proposition wasn’t the plane or camera, but the specialized algorithms to accurately monitor water and fertilizer. Brilliant.

logo

I asked Ashwin how they figured all this out. His reply, “You taught us that there were no facts inside our building.  So we’ve learned to live with our customers.  We’re now piloting our application with Tree Farmers in California and working with crop specialists at U.C. Davis.  We think we have a real business.”

It was a fun coffee.

Lessons Learned

  • Build continuous customer discovery into your company DNA
  • An MVP eliminates parts of your business model that create complexity
  • Focus on what provides immediate value for Earlyvangelists
  • Add complexity (and additional value) later

Listen to the blog post here


Download the podcast here

What I Learned by Flipping the MOOC

Two of the hot topics in education in the last few years have been Massive Open Online Courses (MOOC’s) and the flipped classroom. I’ve been experimenting with both of them.

What I’ve learned (besides being able to use the word “pedagogy” in a sentence) is
1) assigning students lectures as homework doesn’t guarantee the students will watch them and 2) in a flipped classroom you can become hostage to the pedagogy.

Here’s the story of what we tried and what we learned.

MOOC’s – Massive Open Online Courses
A MOOC is a complicated name for a simple idea – an online course accessible to everyone over the web. I created my MOOC by serendipity. Learning how to optimize it in my classes has been a more deliberate and iterative process.

If you can’t see the video above click here

When my Lean LaunchPad class was adopted by the National Science Foundation, we taught our original classes to scientists scattered across the U.S.  We adopted WebEx, a web video conferencing tool, to hold our classes remotely. Just like my students at Stanford, these NSF teams got out of the building and spoke to 10-15 customers a week. Back in their weekly class, the scientists would present their results in front of their peers – in this case via Webex, as the teaching team gave them critiques and “guidance”. When their presentations were over, it was my turn. I lectured to these remote students about the next week’s objectives.

Is it Live or Is It a MOOC?
After the first NSF class held via videoconference, it dawned on me that since I wasn’t physically in front of the students, they wouldn’t know if my lecture was live or recorded.

Embracing the “too dumb to know it can’t be done,” I worked with a friend from Stanford, Sebastian Thrun and his startup Udacity, to put my Lean LaunchPad lectures online. Rather than just have me drone on as a talking head, I hired an animator to help make the lectures interesting, and the Udacity team had the insight to suggest I break up my lecture material into small, 2-4 minute segments that matched students’ attention spans.

If you can’t see the video above click here

Over a few months we developed the online lectures, then tried it as a stand-in for me on the NSF videoconferences, and found that because of the animations and graphics the students were more engaged than if I were teaching it in person. Ouch.

Now the NSF teams were learning from these online lectures instead of video conferenced lectures – but the online lectures were still being played during class time.

I wondered if we could be more efficient with our classroom time.

Flipping
Back at Stanford and Berkeley, I realized that I could use my newly created Lean LaunchPad MOOC and “flip” the classroom.  It sounded easy, I had read the theory:
1) A flipped classroom moves lectures traditionally taught in class, and assigns them as homework. Therefore my  students will all eagerly watch the videos and come to class ready to apply their knowledge, 2) this would eliminate the need for any lecture time in class.  And as a wonderful consequence, 3) I could now admit more teams to the class because we’d now have more time for teams to present.

So much for theory. I was wrong on all three counts.

Theory Versus Practice
After each class, we’d survey the students and combine it with a detailed instructor post mortem of lessons learned.  (An example from our UCSF Lean LaunchPad for Life Sciences Class is here.)

Here’s what we found when we flipped the classroom:

  • More than half the students weren’t watching the lectures at home.
  • Without an automated tool to take an attendance, I had no idea who was or wasn’t watching.
  • Without lectures, my teaching team and I felt like observers. Although we were commenting and critiquing on students presentations, the flipped classroom meant we were no longer in the front of the room.
  • No lectures meant no flexibility to cover advanced topics or real time ideas past the MOOC lecture material.

We decided we needed to fix these issues, one at a time.

  • In subsequent classes we reduced class size from ten teams to eight. This freed up time to get lecture and teaching time back in the classroom.
  • We manually took attendance of who watched our MOOC (later this year this will be an automated part of the LaunchPad Central software we use to manage the classes.)
  • To get the teaching team front and center, I required students to submit questions about material covered in the MOOC lecture they watched the previous evening. I selected the best questions and used them to open the class with a discussion. I cold-called on students to ensure they all had understood the material.
  • We developed advanced lectures which combined a summary of the MOOC material with new material such as lectures focused on domain specific perspectives. For example, in our UCSF Life Sciences class the four VC’s who taught the class with me developed advanced business model lectures for therapeutics, diagnostics, medical devices and digital health. (These advanced lectures are now on-line and available to everyone who teaches the class.)

The class, now taught as hybrid flipped classroom, looks like this: Lean LaunchPad Class Organization

There’s still more to do.

  • While we use LaunchPad Central to have the teams provide feedback to each other, knowledge sharing across the teams still needs to be deeper and more robust.
  • While we try to give students tutorials for how to do Customer Discovery we need a better way to integrate these into the short time in quarter/semester.
  • While we insist that an MVP is part of the class, we need a more rigorous process for building the MVP in parallel with Customer Discovery

Outcomes
Besides finding the right balance in a flipped classroom, a few other good things have come from these experiments. The Udacity lectures now have over 250,000 students. They are not only used in my classes but are also part of other educators’ classes, as well as being viewed by aspiring entrepreneurs as stand-alone tutorials.

My experiments in how to teach the Lean LaunchPad class have led to a 2 ½ day class for 75 educators a quarter (information here.) And we’ve found a pretty remarkable way to use the Lean LaunchPad to organize corporate innovation/incubator groups. (We opened source our teaching guide we use in the classes here.)Educator's Program cover

Lessons Learned

  • Creating engaging MOOC’s are hard
  • Confirming that students watched the MOOC’s is even harder
  • The Flipped classroom needs to be balanced with:
    • Student accountability
    • Instructor time in front of the class
    • Advanced lectures

Listen to the blog post here


Download the podcast here

Sometimes It Pays to be a Jerk

That he which hath no stomach to this fight,
Let him depart; his passport shall be made
William Shakespeare Henry V | Act 4, Scene 3

band of brothers

The concepts in my Lean LaunchPad curriculum can be taught in a variety of classes–as an introduction to entrepreneurship all the way to a graduate level “capstone class.”

I recently learned being tough when you select teams for a capstone class pays off for all involved.

Here’s why.

—-

Our Lean LaunchPad class requires student teams to get out of the building and talk to 10-15 customers a week while they’re building the product.  And they do this while they are talking a full load of other classes.  To say it’s a tough class is an understatement.  The class is designed for students who said they want  a hands-on experience in what it takes to build a startup – not just writing a business plan or listening to lectures.

The class syllabus has all kinds of “black box” warnings about how difficult the class is, the amount of time required, etc.

Yet every year about 20% of teams melt down and/or drop the class because some of the team members weren’t really committed to the class or found they’ve overcommitted.

This year that drop out rate went to zero when I ran an accidental “be a jerk” experiment.

Here Are the Rules
We set up the Lean LaunchPad class so that teams hit the ground running in the first class. Before students are admitted, they formed teams, applied as a team with a business model canvas, had homework and were expected to be presenting their business model canvas hypotheses on day one of the class. Our first class session is definitely not a “meet and greet”.  The syllabus is clear that attendance was mandatory for the first class.

This year, at one of the universities where I teach in the engineering school, our quarter was going to start right after the New Year.  Some of the teams had students from the business school, law school and education school whose start dates were a few days later.

To remind everyone that attendance at the first class was required, we sent out an email to all the teams in December. We explained why attendance at the first class was essential and reminded them they agreed to be there when they were admitted to the class. The email let them know if they missed the first class, they weren’t going to be allowed to register.  And since teams required 4 members, unless their team found a replacement by the first week, the team would not be allowed to register either. (We made broad exceptions for family emergencies, events and a few creative excuses.)

I had assumed everyone had read the syllabus and had planned to be back in time for class.

Then the excuses started rolling in.

Be A Jerk
About 25% of the teams had team members who had purposely planned to miss the first class.  Most of the excuses were, “I thought I could make it up later.”

In past years I would have said, “sure.”  This year I decided to be a jerk.

I had a hypothesis that showing up for the first class might be a good indicator of commitment when the class got tough later in the quarter.  So this time, unless I heard a valid excuse for an absence I said, “too bad, you’ve dropped the class.”

You could hear the screaming around the world (this is in a school where the grading curve goes from A to A+.)  The best was an email from a postdoc who said “all his other professors had been accommodating his “flexible” schedule his entire time at the school and he expected I would be as well.“  Others complained that they had paid for plane tickets and it would cost them money to change, etc.

I stuck to my guns – pointing out that they had signed up for the class knowing this was the deal.

Half the students who said they couldn’t make it magically found a way to show up.  The others dropped the class.

The results of the experiment?  Instead of the typical 20% drop out rate during the quarter none left – 0.

We had a team of committed and passionate students who wanted to be in the class.  Everyone else failed the “I’m committed to making this happen” test.

Lessons Learned

  • Commitment is the first step in building a startup team.
  • It washes out the others
  • Setting a high bar saves a ton of grief later

Listen to the blog post here


Download the podcast here

Time For Founders School

Having a film crew in your living room for two days is something you want to put on your bucket list.

photo 2

photo 3-1

With a ~$2 billion endowment the Kauffman Foundation is the largest non-profit focused on entrepreneurship in the world. Giving away $80 million to every year (~$25 million to entrepreneurial causes) makes Kauffman the dominant player in the entrepreneurship space.

Kauffman just launched Founders School - a new education series to help entrepreneurs develop their businesses during the startup stage by highlighting how startups are different from big companies. After weeks honing the script and days of filming, I’m honored to present the “Startups” section of Founders School.

And I’m in good company – also in the series is Noam Wasserman of Harvard teaching Founder’s Dilemmas, Craig Wortmann University of Chicago covering Entrepreneurial Selling, Peter McDermott helping understand Intellectual Property, and Nathan Gold offering how to give Powerful Presentations.

These videos are not only great tutorials for founders but also provide educators another source of well produced and curated resources.

These “Startup” videos are a great general purpose companion to my “How to Build a Startup” lectures on Udacity.

And you get a tour of my living room…

Startups” introduction is here

Module 1, What We Know About Startups

  • 0:17: A Startup is not a smaller version of a large company
  • 0:45: The definition of a startup
  • 1:53: Types of Startups
  • 2:18: Startups in an Existing Market
  • 3:10: Startups in a New Market
  • 4:31: Startups in a Resegmented Market
  • 5:28: Startups in a Clone Market

Module 2, Startups Versus Big Companies

  • 0:43: Business Plans versus Business Models
  • 1:46: The Differences: Accounting, Engineering & Sales
  • 2:21: Accounting Metrics in a Large Company vs. Metrics that Matter in a Startup
  • 3:35: Job Titles in a Large Company can Sink a Startup
  • 6:07: Engineering: Waterfall Development in a Large Company vs. Minimum Viable Product in a Startup

Module 3, The Lean Method

  • 0:50: There are No Facts Inside Your Building — Get Outside
  • 1:28: Using the Business Model Canvas
  • 1:49: Use Customer Development to Test Your Hypotheses
  • 2:44: What is a Pivot?
  • 4:24: No Business Plan Survives First Contact with Customers

Module 4, Building Your Startup

  • 0:41: Don’t outsource Customer Discovery
  • 1:33: How to build your startup
  • 2:48: How to building your team
  • 3:15: Look for overlapping skill sets and complementary temperaments

Module 5, Pivot or Proceed, How to Decide

  • 0:33: Is there Product-Market Fit?
  • 1:00: Most startups fail
  • 1:20: Adopt a mindset of learning
  • 1:27: Proceed, pivot or restart

The second half of the “Startups” series is coming in March.

Go watch Founders School now.

Listen to the blog post here


Download the podcast here

Lessons Learned in Diagnostics

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them. The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

Mira Medicine is one of the 26 teams in the class. The team members are:
  • Pierre-Antoine Gourraud – PhD, MPH UCSF neuroscientist and c0-leader of the MS BioScreen project
  • Jason Crane - PhD UCSF Manager Scientific Software Development
  • Raphaelle Loren - Managing Director – Health Practice at the Innovation Management Institute

Todd Morrill was the diagnostics cohort instructor. Matt Cooper CEO of Carmenta BioSciences was the Mira Medicine team mentor.

Multiple Sclerosis - MS
Multiple Sclerosis – MS – is an immune system disease that attacks the myelin, the fatty sheath that surrounds and protects nerve fibers of the central nervous system (brain, spinal cord, and optic nerve). T-cells, (a type of white blood cell in the immune system,) become sensitized to myelin and cross the blood-brain barrier into the central nervous system (CNS). Once in the CNS, these T-cells injure myelin, and secrete chemicals that damage nerve fibers (axons) and recruit more damaging immune cells to the site of inflammation. multiple sclerosis and therapeutic targets

There are currently ten FDA approved MS medications for use in relapsing forms of MS. None of these drugs is a cure, and no drug is approved to treat the type of MS that shows steady progression at onset. MS disease management decisions are complex and requires a patients neurologist to figure out what drugs to use.

Mira Medicine and Multiple Sclerosis
The team came to class with the thought of commercializing the UCSF Multiple Sclerosis BioScreen Project a Precision Medicine application that integrates a patients medical records with the latest population-based data from hundreds of other Multiple Sclerosis patients, (including their 3D MRI scans,) and using predictive algorithms makes it possible to chart a unique course of treatment for each patient. (Mira Medicine team member Pierre-Antoine Gourraud was the project co-leader.) MS BioScreen

Mira wanted to commercialize the UCSF Multiple Sclerosis Bioscreen project and to add additional neurological diseases which require multiple types of data  (including biomarkers, clinical, and imaging). They wanted to help medical centers and large providers assess disease progression to guide therapeutic decision-making.  Over the course of the class Mira Medicine team spoke to over 80 customers, partners and payers.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Then reality hit. First, the team found that their Multiple Sclerosis Bioscreen application (which they used as their MVP) was just a “nice-to-have”, not a “must-have”. In fact, the “must have features” were their future predictive algorithms. Next, they found that if their tool can enable a diagnosis, (even without claiming it could) then it was likely that the FDA would require a  510(k) medical device clearance. Then they found to get reimbursed they need a CPT code (and they had to decide whether to code stack - using multiple codes for “one” diagnosis, and thereby getting multiple reimbursements for one test. (The rules have changed so that code stacking is hard or impossible), Or get a new CPT code, or use miscellaneous code.) To get a new CPT and a 510(k) they would have to perform a some sort of clinical study. At a minimum a 1-year prospective study (a study to see if the neurologists using the application had patients with a better outcome then those who didn’t have access to the app). Getting approval to use an existing (aka old) CPT code means showing equivalence to an existing dx process or test, and the requirements are code-specific. Finally, to get access to data sources of other MS patients they would need to have HIPPA Business Associate Agreement.

Watch their Lessons Learned video below and find out how they pivoted and what happened.

If you can’t see the video above click here

Look at their Lesson Learned slides below If you can’t see the presentation above, click here

Lessons Learned

  • Researchers and PI’s come in believing “My science/project/data are so good that people will immediately see the value and be willing to pay for it.  It will “sell itself”.
  • A business is much more than just good science: it is about customers seeing value and being willing to pay and proper validation and reimbursement coding and
  • A successful business is the sum (and integration!) of all the parts of the business model canvas.
    • It includes reimbursement, regulation, IP, validation, channel access, etc.

Listen to the blog post here


Download the podcast here

Lessons Learned in Therapeutics

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them. The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

We are redefining how translational medicine is practiced.

Traditional view of translational medicineWe’ve learned that translational medicine is not just about the science.

More on this in future blog posts.

Lean view of translational medicine

Vitruvian Therapeutics is one of the 26 teams in the class. The team members are:
  • Dr. Hobart Harris  Chief of  General Surgery, Vice-Chair of the Department of Surgery, and a Professor of Surgery at  UCSF.
  • Dr. David Young,  Professor of Plastic Surgery at UCSF. His area of expertise includes wound healing, microsurgery, and reconstruction after burns and trauma. 
  • Cindy Chang is a Enzymologist investigating novel enzymes involved in biofuel and chemical synthesis in microbes at LS9

Karl Handelsman was the therapeutics cohort instructor. Julie Cherrington CEO of Pathway Therapeutics was the team mentor.

Vitruvian Therapeutics is trying to solve the Incisional hernia problem. An incisional hernia happens in open abdominal surgery when the area of the wound doesnt heal properly and bulges outward. This requires a second operation to fix the hernia.Ventral Herniaincisional hernia

Hobart Harris’s insight was what was needed wasn’t one more new surgical technique or device to repair the hernias, but something to prevent the hernia from occurring in the first place. Vitruvian Therapeutics first product, MyoSeal, does just that. It promotes wound repair via biocompatible microparticles plus a fibrin tissue sealant. So far in 300 rats it’s been shown to prevent incisional hernias through enhanced wound healing.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Two weeks into the class and interviews with 14 of their potential customers (surgeons) reality intruded on their vision of how the world should work. We happened to catch that moment in class in this 90 second clip.

Watch  and find out how talking to just the first 14 customers in the Lean LaunchPad class saved Hobart Harris and the Vitruvian Therapeutics team years.

If you can’t see the clip above click here.

The Vitruvian Therapeutics Lessons Learned Presentation is a real-eyeopener. Given that this product could solve the incisional hernia problem, Hobart and his team naturally assumed that insurance companies would embrace this and their fellow surgeons viewed the problem as they did and would leap at using the product. Boy were they in for a surprise. After talking to 74 surgeons, insurance companies and partners appeared that no one – insurance companies or surgeons – owned the problem. Listen to their conclusions 8-weeks after the first video.

Watch the video and find out how they pivoted and what happened.

Don’t miss Karl Handelsman comments on their Investment Readiness Level at the end. Vitruvian is a good example of a great early stage therapeutics idea with animal data missing and many key components of the business model still needed to verify.

If you can’t see the video above click here

Look at their Lesson Learned slides below

If you can’t see the presentation above, click here

Market Type
During the class the Vitruvian Therapeutics class struggled with the classic question of visionaries: are we creating a New Market (one which doesn’t exist and has no customers)? In Vitruvian’s case preventive measures to stop incisional hernias before they happen.  Or should we position our product as one that’s Resegmenting an Existing Market? i.e. reducing leakage rates.  Or is there a way to get proof that the vision of the New Market is the correct path.

When Hobart Harris of Viturvian asked, “… what if you’re a visionary, and no one but you sees the right solution to a problem” we had a great in-class dialog. Karl Handelsman‘s comments at 3:15 and 4:16 and Allan May at 4:35 were incredibly valuable. See the video below for the dialog.

If you can’t see the video above, click here

Further Reading

Lessons Learned

  • Principal Investigators, scientists and engineers can’t figure out commercialization sitting in their labs
  • You can’t outsource commercialization to a proxy (consultants, market researchers, etc.)
  • Experiential Learning is integral to commercialization
  • You may be the smartest person in your lab, but your are not smarter than the collective intelligence of your potential customers, partners, payers and regulators

Listen to the blog post here


Download the podcast here

Moneyball and the Investment Readiness Level-video

Eric Ries was kind enough to invite me to speak at his Lean Startup Conference.

In the talk I reviewed the basic components of the Lean Startup and described how we teach it. I observed that now that we’ve built software to instrument and monitor the progress of new ventures (using LaunchPad Central), that we are entering the world of evidence-based entrepreneurship and the Investment Readiness Level.

This video is a companion to the blog post here. Read it for context.

If you can’t see the video above, click here

You can follow the talk along using the slides below

If you can’t see the slides above, click here

Additional videos here

Startup Tools here

Listen the blog post here


Download the podcast here

Lessons Learned in Medical Devices

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them.  The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

We are redefining how translational medicine is practiced. It’s Lean, it’s fast, it works and it’s unlike anything else ever done.

—–

Sometimes teams win when they fail.

Knox Medical Devices was building a Spacer which contained a remote monitoring device to allow for intervention for children with Asthma . (A Spacer is a tube between a container of Asthma medicine (in an inhaler) and a patient’s mouth.The tube turns the Asthma medicine into an aerosol.)Asthma

Knox’s spacer had sensors for basic spirometry measurements (the amount of air and how fast it’s inhaled and exhaled) to see how well the lung is working. It also had a Nitrous Oxide sensor to provide data on whether the lungs airways are inflamed, an inhaler attachment and a GPS tracking device.

Knox SpacerThe Spacer hardware was paired with data analysis software for tracking multiple facets of asthma patients.

The Knox team members are:

Allan May founder of Life Science Angels was the Medical Device cohort instructor. Alex DiNello CEO at Relievant Medsystems was their mentor.

The Knox team was a great mix of hands-on device engineers and business development. They used agile engineering perfectly to continually test variants of their Minimum Viable Product (MVP’s) in front of customers often and early to get immediate feedback.

Knox was relentless about understanding whether their device was a business or whether it was technology in search of a market. In 10 weeks they had face-to-face meetings with 117 customers, tested 33 hypotheses, invalidated 19 of them and 53 instructor and mentor interactions.

Here’s Knox Medical’s 2 minute video summary

If you can’t see the video above, click here

Knox was a great example of having a technology in search of a customer. The initial hypothesis of who would pay for the device – parents of children with asthma – was wrong and resulted in Knox’s first pivot in week 4. By week 6 they had discovered that; 1) Peak Flow Meters are not as heavily prescribed as they thought, 2) Insurance company reimbursement is necessary for anything upwards of $15, 3) Nitrous Oxide testing isn’t currently used to measure asthma conditions.

After the pivot they the found that the most likely users of their device would be low income Asthma patients who are treated at Asthma clinics funded by federal, state or county dollars. These clinics reduce hospitalization but Insurers weren’t paying to cover clinic costs nor would they cover the use of the Knox device. The irony was that those who most needed the Knox device were those who could least afford it and wouldn’t be able get it.

Watch their Lesson Learned presentation below. Listen to the comments from Allan May the Device instructor at the end.

If you can’t see the video above, click here

In the end Knox, like a lot of startups in Life Science and Health Care, discovered that they had a multi-sided market.  They realized late in the class the patients (and their families) were not their payers - their payers were the insurance companies (and the patients were the users.)  If they didn’t have a compelling value proposition for the insurers (cost savings, increased revenue, etc.) it didn’t matter how great the technology was or how much the patients would benefit.

The Knox Medical Device presentation slides are below. Don’t miss the evolution of their business model canvas in the appendix. It’s a film strip of the entrepreneurial process in action.

If you can’t see the slides above, click here

Knox is a great example of how the Lean LaunchPad allows teams to continually test hypotheses and fail fast and inexpensively. They learned a ton. And saved millions.

Lessons Learned

  • In medical devices, understanding reimbursement, regulation and IP is critical
  • Sometimes teams win when they fail
Listen to the blog post here


Download the podcast here

We’ve seen the Future of Translational Medicine and it’s Disruptive

A team of 110 researchers and clinicians, in therapeutics, diagnostics, devices and digital health in 25 teams at UCSF, has just shown us the future of translational medicine.  It’s Lean, it’s fast, it works and it’s unlike anything else ever done.

It’s going to get research from the lab to the bedside cheaper and faster.

Welcome to the Lean LaunchPad for Life Sciences and Healthcare (part of the National Science Foundation I-Corps).

This post is part of our series on the Lean Startup in Life Science and Health Care.

——–

Our class talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them.  They had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

In a packed auditorium in Genentech Hall at UCSF, the teams summarized what they learned after 10 weeks of getting out of the building. This was our version of Demo Day – we call it “Lessons Learned” Day. Each team make two presentations:

  • 2 minutes YouTube Video: General story of what they learned from the class
  • 8 minute Lessons Learned Presentation: Very specific story about what they learned in 10 weeks about their business model

In the next few posts I’m going to share a few of the final “Lessons Learned” presentations and videos and then summarize lessons learned from the teaching team.

Magnamosis
Magnamosis is a medical device company that has a new way to create a magnetic compression anastomosis (a surgical connection between two tubular structures like the bowel) with improved outcomes.

anastomosis

Team Members were: Michael Harrison (the father of fetal surgery), Michael Danty, Dillon Kwiat, Elisabeth Leeflang, Matt Clark.  Jay Watkins was the team mentor. Allan May and George Taylor were the medical device cohort instructors.

Their initial idea was that making an anastomosis that’s better, faster and cheaper will have surgeons fighting to the death to get a hold of their device.  magnamosisThey quickly found out that wasn’t the case.  Leak rates turned out to a bigger issue with surgeons and a much larger market.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Watch their Lessons Learned video below and see how a team of doctors learned about product/market fit, channels and pricing.

If you can’t see the video above, click here

Their slide deck is below. Don’t miss the evolution of their business model in the Appendix.

If you can’t see the presentation above, click here

The best summary of why Scientists, Engineers and Principal Investigators need to get out of the building was summarized by Dr. Harrison below. After working on his product for a decade listen to how 10 weeks of the Lean LaunchPad class radically changed his value proposition and business model.

If you can’t see the video above, click here.

For further reading:

Listen to the blog post here


Download the podcast here

It’s Time to Play Moneyball: The Investment Readiness Level

Investors sitting through Incubator or Accelerator demo days have three metrics to judge fledgling startups – 1) great looking product demos, 2) compelling PowerPoint slides, and 3) a world-class team.

We think we can do better.

We now have the tools, technology and data to take incubators and accelerators to the next level. Teams can prove their competence and validate their ideas by showing investors evidence that there’s a repeatable and scalable business model. And we can offer investors metrics to play Moneyball – with the Investment Readiness Level.

Here’s how.

————–

We’ve spent the last 3 years building a methodology, classes, an accelerator and software tools and we’ve tested them on ~500 startups teams.

  • A Lean Startup methodology offers entrepreneurs a framework to focus on what’s important: Business Model Discovery. Teams use the Lean Startup toolkit: the Business Model Canvas + Customer Development process + Agile Engineering. These three tools allow startups to focus on the parts of an early stage venture that matter the most: the product, product/market fit, customer acquisition, revenue and cost model, channels and partners.

Lean moneyball

  • An Evidence-based Curriculum (currently taught in the Lean LaunchPad classes and NSF Innovation Corps accelerator). In it we emphasize that a) the data needed exists outside the building, b) teams use the scientific method of hypothesis testing c) teams keep a continual weekly cadence of:
    • Hypothesis – Here’s What We Thought
    • Experiments – Here’s What We Did
    • Data – Here’s What We Learned
    • Insights and Action – Here’s What We Are Going to Do Next

Evidence moneyball

  • LaunchPad Central software is used to track the business model canvas and customer discovery progress of each team. We can see each teams hypotheses, look at the experiments they’re running to test the hypotheses, see their customer interviews, analyze the data and watch as they iterate and pivot.

LPC

We focus on evidence and trajectory across the business model. Flashy demo days are great theater, but it’s not clear there’s a correlation between giving a great PowerPoint presentation and a two minute demo and building a successful business model. Rather than a product demo – we believe in a “Learning Demo”. We’ve found that “Lessons Learned” day showing what the teams learned along with the “metrics that matter” is a better fit than a Demo Day.

“Lessons Learned” day allows us to directly assess the ability of the team to learn, pivot and move forward. Based on the “lessons learned” we generate an Investment Readiness Level metric that we can use as part of our “go” or “no-go” decision for funding.

Some background.

NASA and the Technology Readiness Level (TRL)
In the 1970’s/80’s NASA needed a common way to describe the maturity and state of flight readiness of their technology projects.  They invented a 9-step description of how ready a technology project was.  They then mapped those 9-levels to a thermometer.NASA TRL

What’s important to note is that the TRL is imperfect. It’s subjective. It’s incomplete.  But it’s a major leap over what was being used before.  Before there was no common language to compare projects.

The TRL solved a huge problem – it was a simple and visual way to share a common understanding of technology status.  The U.S. Air Force, then the Army and then the entire U.S. Department of Defense along with the European Space Agency (ESA) all have adopted the TRL to manage their complex projects. As simple as it is, the TRL is used to manage funding and go/no decisions for complex programs worldwide.

We propose we can do the same for new ventures – provide a simple and visual way to share a common understanding of startup readiness status. We call this the Investment Readiness Level . 

The Investment Readiness Level (IRL)
The collective wisdom of venture investors (including angel investors, and venture capitalists) over the past decades has been mostly subjective. Investment decisions made on the basis of “awesome presentation”, “the demo blew us away”, or “great team” is used to measure startups. These are 20th century relics of the lack of data available from each team and the lack of comparative data across a cohort and portfolio.

Those days are over.

Hypotheses testing and data collection
We’ve instrumented our startups in our Lean LaunchPad classes and the NSF I-Corps incubator using LaunchPad Central to collect a continuous stream of data across all the teams.  Over 10 weeks each team gets out and talks to 100 customers. And they are testing hypotheses across all 9 boxes in the business model canvas.

We collect this data into a Leaderboard (shown in the figure below) giving the incubator/accelerator manager a single dashboard to see the collective progress of the cohort. Metrics visible at a glance are number of customer interviews in the current week as well as aggregate interviews, hypotheses to test, invalidated hypotheses, mentor and instructor engagements. This data gives a feel for the evidence and trajectory of the cohort as a whole and a top-level of view of each teams progress.

leaderboard moneyball

Next, we have each team update their Business Model Canvas weekly based on the 10+ customer interviews they’ve completed.

canvas updates moneyball

The canvas updates are driven by the 10+ customer interviews a week each team is doing. Teams document each and every customer interaction in a Discovery Narrative. These interactions provide feedback and validate or invalidate each hypothesis.

disovery 10 moneyball

Underlying the canvas is an Activity Map which shows the hypotheses tested and which have been validated or invalidated.

activty updates moneyball

All this data is rolled into a Scorecard, essentially a Kanban board which allows the teams to visualize the work to do, the work in progress and the work done for all nine business model canvas components.

scorecard update moneyball

Finally the software rolls all the data into an Investment Readiness Level score.

IRL

MoneyBall
At first glance this process seems ludicrous. Startup success is all about the team. Or the founder, or the product, or the market – no metrics can measure those intangibles.

Baseball used to believe that as well. Until 2002 – when the Oakland A’s’ baseball team took advantage of analytical metrics of player performance to field a team that competed successfully against much richer competitors.

Statistical analysis demonstrated that on-base percentage and slugging percentage were better indicators of offensive success, and the A’s became convinced that these qualities were cheaper to obtain on the open market than more historically valued qualities such as speed and contact. These observations often flew in the face of conventional baseball wisdom and the beliefs of many baseball scouts and executives.

By re-evaluating the strategies that produce wins on the field, the 2002 Oakland A’s spent $41 million in salary, and were competitive with the New York Yankees, who spent $125 million.

Our contention is that the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to have a robust and consistent data set across teams. While it doesn’t eliminate great investor judgement, pattern recognitions skills and mentoring – it does provide them the option to play Moneyball.

if you can’t see the video above click here

Last September Andy Sack, Jerry Engel and I taught our first stealth class for incubator/accelerator managers who wanted to learn how to play Moneyball.

We’re offering one again this January here.

Lessons Learned

  • It’s not clear there’s a correlation between a great PowerPoint presentation and two minute demo and building a successful business
  • We now have the tools and technology to take incubators and accelerators to the next step
  • We focus on evidence and trajectory across the business model
  • The data gathered can generate an Investment Readiness Level score for each team
  • the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to play Moneyball

Listen to the podcast here


Download the podcast here

Lean LaunchPad for Life Sciences – Revenue Streams

We’re teaching a Lean LaunchPad class for Life Sciences and Health Care (therapeutics, diagnostics, devices and digital health) at UCSF with a team of veteran venture capitalists. The class has talked to 2,056 customers to date.

This post is an update of what we learned about life science revenue models.

Life Science/Health Care revenue streams differ by Category
For commercialization, the business model (Customers, Channel, Revenue Model, etc.) for therapeutics, diagnostics, devices, bioinformatics and digital health have very little in common.

This weeks topic was revenue streams – how much cash the company can generate from each customer segment. Revenue streams have two parts: the revenue strategy and the pricing tactics.

Figuring out revenue strategy starts by gaining a deep understanding of the target customer(s). Setting a revenue strategy starts with understanding the basics about the customer segments:

  • who’s the user, the recommender, buyer, and payer
  • How the target customer currently purchases goods and services and how much they currently pay for equivalent products
  • Their willingness to pay for value versus lowest cost?
  • How much budget they have for your type of product?

Revenue strategy asks questions like, “Should we offer cost-based or value-based pricing.  How about demand-based pricing? Freemium? Do we price based on hardware sales or do we offer hardware plus consumables (parts that need to be disposed or replaced regularly)? Do we sell a single software package or a subscription?  These strategy hypotheses are tested against the target customer segment(s).

Once you’ve established a revenue strategy the pricing tactics follow. Pricing is simply “how much can I charge for the product using the selected revenue strategy?”  Pricing may be as simple as setting a dollar value for hardware or software, or as complicated as setting a high price and skimming the market or setting a low price as a loss leader.

You can get a feel for how each of the cohorts address the Revenue Streams by looking at the Revenue lectures below – covering the therapeutics, diagnostics, devices and digital health cohorts.

At the end of the lectures you can see a “compare and contrast” video and a summary of the differences in distribution channels.

Diagnostics

Week 5 Todd Morrill Instructor 

If you can’t see the presentation above click here

Digital Health

Week 5 Abhas Gupta Instructor 

If you can’t see the presentation above click here

Devices

Week 5 Allan May Instructor 

If you can’t see the presentation above click here

Therapeutics

Week 5 Karl Handelsman Instructor 

If you can’t see the presentation above click here

Life Science and Health Care Differences in Revenue Streams
”
This weeks lecture and panel was on Revenue; how much cash the company can generate from each customer segment – and the strategy and tactics to do so. Therapeutics, diagnostics, devices and digital health use different Revenue Strategies and Pricing Tactics, in the video and the summary that follows the instructors compare and contrast how they differ.

If you can’t see the video above click here

Therapeutics (Starting at 0:30)

  • Therapeutics revenue is from drug companies not end users
  • 18 months to first revenue from a deal
  • Predicated on delivering quality data to a company
  • Deal can be front-end or back-end loaded
  • Quality of the data has to be extremely high for a deal

Diagnostics (Starting at 4:10)

  • Diagnostic revenue is from end users: a hospital or clinical lab
  • You need to figure out value of your product but…
  • Pricing is capped by your reimbursement (CPT) code limits
  • Reimbursement strategy is paramount, design to good codes avoid bad ones
  • Find a reimbursement code consultant
  • Don’t do cost-based pricing… go for value-based pricing

Medical Devices (Starting at 8:23)

  • There really is no such thing as a perfect First Generation Medical Device
    • So Medical Device companies often start with a Volkswagen product and then build to the Ferrari product
  • Revenue models are typically direct product sales
  • Don’t do cost-based pricing… go for value-based pricing, especially where your device lowers the treatment costs of the patient
  • In most cases, pricing is capped by your reimbursement (CPT) code limits
    • Or pricing can be capped by what competitors offer, unless you can demonstrate superior cost savings
    • In a new market there is no reimbursement code but if you show high cost-savings you can get a high reimbursement rate
  • A risk in device hardware is getting trapped in low-volume manufacturing with low margins and run out of cash

Digital Health (Starting at 10:35)

  • Digital Health revenue models are often subscription models to a company per month across a large number of users
    • Intermediation fees – where you broker a transaction – are another source of revenue (i.e. HealthTap)
    • Advertising is another digital health revenue model, but requires at least 10 million users to have a meaningful model, but can be lower if you have higher value uses like specialist physicians because  you can charge dollars not cents
  • Don’t do cost-based pricing… go for value-based pricing
    • Value-based pricing is based on the needs you’ve learned from the customer segment and the strength of your product/market fit
      • the sum of customer needs + product/market fit = the pricing you can achieve

Lessons Learned

  • Each of these Life Science domains has a unique revenue strategy and pricing tactic
  • In therapeutics revenue comes in lump milestone payments from drug companies based on quality data
  • Diagnostics revenue comes value pricing to hospital or clinical lab
    • capped by reimbursement (CPT) code limits
  • Device pricing starts by offering an initial value-priced base product and then following up with a fully featured product
    • capped by reimbursement (CPT) code limits
  • Digital health products use subscription value pricing. Alternatively may use advertising revenue model

Listen to the podcast here


Download the podcast here

Follow

Get every new post delivered to your Inbox.

Join 137,502 other followers