Sometimes It Pays to be a Jerk

That he which hath no stomach to this fight,
Let him depart; his passport shall be made
William Shakespeare Henry V | Act 4, Scene 3

band of brothers

The concepts in my Lean LaunchPad curriculum can be taught in a variety of classes–as an introduction to entrepreneurship all the way to a graduate level “capstone class.”

I recently learned being tough when you select teams for a capstone class pays off for all involved.

Here’s why.

—-

Our Lean LaunchPad class requires student teams to get out of the building and talk to 10-15 customers a week while they’re building the product.  And they do this while they are talking a full load of other classes.  To say it’s a tough class is an understatement.  The class is designed for students who said they want  a hands-on experience in what it takes to build a startup – not just writing a business plan or listening to lectures.

The class syllabus has all kinds of “black box” warnings about how difficult the class is, the amount of time required, etc.

Yet every year about 20% of teams melt down and/or drop the class because some of the team members weren’t really committed to the class or found they’ve overcommitted.

This year that drop out rate went to zero when I ran an accidental “be a jerk” experiment.

Here Are the Rules
We set up the Lean LaunchPad class so that teams hit the ground running in the first class. Before students are admitted, they formed teams, applied as a team with a business model canvas, had homework and were expected to be presenting their business model canvas hypotheses on day one of the class. Our first class session is definitely not a “meet and greet”.  The syllabus is clear that attendance was mandatory for the first class.

This year, at one of the universities where I teach in the engineering school, our quarter was going to start right after the New Year.  Some of the teams had students from the business school, law school and education school whose start dates were a few days later.

To remind everyone that attendance at the first class was required, we sent out an email to all the teams in December. We explained why attendance at the first class was essential and reminded them they agreed to be there when they were admitted to the class. The email let them know if they missed the first class, they weren’t going to be allowed to register.  And since teams required 4 members, unless their team found a replacement by the first week, the team would not be allowed to register either. (We made broad exceptions for family emergencies, events and a few creative excuses.)

I had assumed everyone had read the syllabus and had planned to be back in time for class.

Then the excuses started rolling in.

Be A Jerk
About 25% of the teams had team members who had purposely planned to miss the first class.  Most of the excuses were, “I thought I could make it up later.”

In past years I would have said, “sure.”  This year I decided to be a jerk.

I had a hypothesis that showing up for the first class might be a good indicator of commitment when the class got tough later in the quarter.  So this time, unless I heard a valid excuse for an absence I said, “too bad, you’ve dropped the class.”

You could hear the screaming around the world (this is in a school where the grading curve goes from A to A+.)  The best was an email from a postdoc who said “all his other professors had been accommodating his “flexible” schedule his entire time at the school and he expected I would be as well.“  Others complained that they had paid for plane tickets and it would cost them money to change, etc.

I stuck to my guns – pointing out that they had signed up for the class knowing this was the deal.

Half the students who said they couldn’t make it magically found a way to show up.  The others dropped the class.

The results of the experiment?  Instead of the typical 20% drop out rate during the quarter none left – 0.

We had a team of committed and passionate students who wanted to be in the class.  Everyone else failed the “I’m committed to making this happen” test.

Lessons Learned

  • Commitment is the first step in building a startup team.
  • It washes out the others
  • Setting a high bar saves a ton of grief later

Listen to the blog post here

Download the podcast here

Time For Founders School

Having a film crew in your living room for two days is something you want to put on your bucket list.

photo 2

photo 3-1

With a ~$2 billion endowment the Kauffman Foundation is the largest non-profit focused on entrepreneurship in the world. Giving away $80 million to every year (~$25 million to entrepreneurial causes) makes Kauffman the dominant player in the entrepreneurship space.

Kauffman just launched Founders School – a new education series to help entrepreneurs develop their businesses during the startup stage by highlighting how startups are different from big companies. After weeks honing the script and days of filming, I’m honored to present the “Startups” section of Founders School.

And I’m in good company – also in the series is Noam Wasserman of Harvard teaching Founder’s Dilemmas, Craig Wortmann University of Chicago covering Entrepreneurial Selling, Peter McDermott helping understand Intellectual Property, and Nathan Gold offering how to give Powerful Presentations.

These videos are not only great tutorials for founders but also provide educators another source of well produced and curated resources.

These “Startup” videos are a great general purpose companion to my “How to Build a Startup” lectures on Udacity.

And you get a tour of my living room…

Startups” introduction is here

Module 1, What We Know About Startups

  • 0:17: A Startup is not a smaller version of a large company
  • 0:45: The definition of a startup
  • 1:53: Types of Startups
  • 2:18: Startups in an Existing Market
  • 3:10: Startups in a New Market
  • 4:31: Startups in a Resegmented Market
  • 5:28: Startups in a Clone Market

Module 2, Startups Versus Big Companies

  • 0:43: Business Plans versus Business Models
  • 1:46: The Differences: Accounting, Engineering & Sales
  • 2:21: Accounting Metrics in a Large Company vs. Metrics that Matter in a Startup
  • 3:35: Job Titles in a Large Company can Sink a Startup
  • 6:07: Engineering: Waterfall Development in a Large Company vs. Minimum Viable Product in a Startup

Module 3, The Lean Method

  • 0:50: There are No Facts Inside Your Building — Get Outside
  • 1:28: Using the Business Model Canvas
  • 1:49: Use Customer Development to Test Your Hypotheses
  • 2:44: What is a Pivot?
  • 4:24: No Business Plan Survives First Contact with Customers

Module 4, Building Your Startup

  • 0:41: Don’t outsource Customer Discovery
  • 1:33: How to build your startup
  • 2:48: How to building your team
  • 3:15: Look for overlapping skill sets and complementary temperaments

Module 5, Pivot or Proceed, How to Decide

  • 0:33: Is there Product-Market Fit?
  • 1:00: Most startups fail
  • 1:20: Adopt a mindset of learning
  • 1:27: Proceed, pivot or restart

The second half of the “Startups” series is coming in March.

Go watch Founders School now.

Listen to the blog post here

Download the podcast here

Lessons Learned in Diagnostics

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them. The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

Mira Medicine is one of the 26 teams in the class. The team members are:
  • Pierre-Antoine Gourraud – PhD, MPH UCSF neuroscientist and c0-leader of the MS BioScreen project
  • Jason Crane – PhD UCSF Manager Scientific Software Development
  • Raphaelle Loren – Managing Director – Health Practice at the Innovation Management Institute

Todd Morrill was the diagnostics cohort instructor. Matt Cooper CEO of Carmenta BioSciences was the Mira Medicine team mentor.

Multiple Sclerosis – MS
Multiple Sclerosis – MS – is an immune system disease that attacks the myelin, the fatty sheath that surrounds and protects nerve fibers of the central nervous system (brain, spinal cord, and optic nerve). T-cells, (a type of white blood cell in the immune system,) become sensitized to myelin and cross the blood-brain barrier into the central nervous system (CNS). Once in the CNS, these T-cells injure myelin, and secrete chemicals that damage nerve fibers (axons) and recruit more damaging immune cells to the site of inflammation. multiple sclerosis and therapeutic targets

There are currently ten FDA approved MS medications for use in relapsing forms of MS. None of these drugs is a cure, and no drug is approved to treat the type of MS that shows steady progression at onset. MS disease management decisions are complex and requires a patients neurologist to figure out what drugs to use.

Mira Medicine and Multiple Sclerosis
The team came to class with the thought of commercializing the UCSF Multiple Sclerosis BioScreen Project a Precision Medicine application that integrates a patients medical records with the latest population-based data from hundreds of other Multiple Sclerosis patients, (including their 3D MRI scans,) and using predictive algorithms makes it possible to chart a unique course of treatment for each patient. (Mira Medicine team member Pierre-Antoine Gourraud was the project co-leader.) MS BioScreen

Mira wanted to commercialize the UCSF Multiple Sclerosis Bioscreen project and to add additional neurological diseases which require multiple types of data  (including biomarkers, clinical, and imaging). They wanted to help medical centers and large providers assess disease progression to guide therapeutic decision-making.  Over the course of the class Mira Medicine team spoke to over 80 customers, partners and payers.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Then reality hit. First, the team found that their Multiple Sclerosis Bioscreen application (which they used as their MVP) was just a “nice-to-have”, not a “must-have”. In fact, the “must have features” were their future predictive algorithms. Next, they found that if their tool can enable a diagnosis, (even without claiming it could) then it was likely that the FDA would require a  510(k) medical device clearance. Then they found to get reimbursed they need a CPT code (and they had to decide whether to code stack – using multiple codes for “one” diagnosis, and thereby getting multiple reimbursements for one test. (The rules have changed so that code stacking is hard or impossible), Or get a new CPT code, or use miscellaneous code.) To get a new CPT and a 510(k) they would have to perform a some sort of clinical study. At a minimum a 1-year prospective study (a study to see if the neurologists using the application had patients with a better outcome then those who didn’t have access to the app). Getting approval to use an existing (aka old) CPT code means showing equivalence to an existing dx process or test, and the requirements are code-specific. Finally, to get access to data sources of other MS patients they would need to have HIPPA Business Associate Agreement.

Watch their Lessons Learned video below and find out how they pivoted and what happened.

If you can’t see the video above click here

Look at their Lesson Learned slides below If you can’t see the presentation above, click here

Lessons Learned

  • Researchers and PI’s come in believing “My science/project/data are so good that people will immediately see the value and be willing to pay for it.  It will “sell itself”.
  • A business is much more than just good science: it is about customers seeing value and being willing to pay and proper validation and reimbursement coding and
  • A successful business is the sum (and integration!) of all the parts of the business model canvas.
    • It includes reimbursement, regulation, IP, validation, channel access, etc.

Listen to the blog post here

Download the podcast here

Lessons Learned in Therapeutics

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them. The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

We are redefining how translational medicine is practiced.

Traditional view of translational medicineWe’ve learned that translational medicine is not just about the science.

More on this in future blog posts.

Lean view of translational medicine

Vitruvian Therapeutics is one of the 26 teams in the class. The team members are:
  • Dr. Hobart Harris  Chief of  General Surgery, Vice-Chair of the Department of Surgery, and a Professor of Surgery at  UCSF.
  • Dr. David Young,  Professor of Plastic Surgery at UCSF. His area of expertise includes wound healing, microsurgery, and reconstruction after burns and trauma. 
  • Cindy Chang is a Enzymologist investigating novel enzymes involved in biofuel and chemical synthesis in microbes at LS9

Karl Handelsman was the therapeutics cohort instructor. Julie Cherrington CEO of Pathway Therapeutics was the team mentor.

Vitruvian Therapeutics is trying to solve the Incisional hernia problem. An incisional hernia happens in open abdominal surgery when the area of the wound doesnt heal properly and bulges outward. This requires a second operation to fix the hernia.Ventral Herniaincisional hernia

Hobart Harris’s insight was what was needed wasn’t one more new surgical technique or device to repair the hernias, but something to prevent the hernia from occurring in the first place. Vitruvian Therapeutics first product, MyoSeal, does just that. It promotes wound repair via biocompatible microparticles plus a fibrin tissue sealant. So far in 300 rats it’s been shown to prevent incisional hernias through enhanced wound healing.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Two weeks into the class and interviews with 14 of their potential customers (surgeons) reality intruded on their vision of how the world should work. We happened to catch that moment in class in this 90 second clip.

Watch  and find out how talking to just the first 14 customers in the Lean LaunchPad class saved Hobart Harris and the Vitruvian Therapeutics team years.

If you can’t see the clip above click here.

The Vitruvian Therapeutics Lessons Learned Presentation is a real-eyeopener. Given that this product could solve the incisional hernia problem, Hobart and his team naturally assumed that insurance companies would embrace this and their fellow surgeons viewed the problem as they did and would leap at using the product. Boy were they in for a surprise. After talking to 74 surgeons, insurance companies and partners appeared that no one – insurance companies or surgeons – owned the problem. Listen to their conclusions 8-weeks after the first video.

Watch the video and find out how they pivoted and what happened.

Don’t miss Karl Handelsman comments on their Investment Readiness Level at the end. Vitruvian is a good example of a great early stage therapeutics idea with animal data missing and many key components of the business model still needed to verify.

If you can’t see the video above click here

Look at their Lesson Learned slides below

If you can’t see the presentation above, click here

Market Type
During the class the Vitruvian Therapeutics class struggled with the classic question of visionaries: are we creating a New Market (one which doesn’t exist and has no customers)? In Vitruvian’s case preventive measures to stop incisional hernias before they happen.  Or should we position our product as one that’s Resegmenting an Existing Market? i.e. reducing leakage rates.  Or is there a way to get proof that the vision of the New Market is the correct path.

When Hobart Harris of Viturvian asked, “… what if you’re a visionary, and no one but you sees the right solution to a problem” we had a great in-class dialog. Karl Handelsman‘s comments at 3:15 and 4:16 and Allan May at 4:35 were incredibly valuable. See the video below for the dialog.

If you can’t see the video above, click here

Further Reading

Lessons Learned

  • Principal Investigators, scientists and engineers can’t figure out commercialization sitting in their labs
  • You can’t outsource commercialization to a proxy (consultants, market researchers, etc.)
  • Experiential Learning is integral to commercialization
  • You may be the smartest person in your lab, but your are not smarter than the collective intelligence of your potential customers, partners, payers and regulators

Listen to the blog post here

Download the podcast here

Moneyball and the Investment Readiness Level-video

Eric Ries was kind enough to invite me to speak at his Lean Startup Conference.

In the talk I reviewed the basic components of the Lean Startup and described how we teach it. I observed that now that we’ve built software to instrument and monitor the progress of new ventures (using LaunchPad Central), that we are entering the world of evidence-based entrepreneurship and the Investment Readiness Level.

This video is a companion to the blog post here. Read it for context.

If you can’t see the video above, click here

You can follow the talk along using the slides below

If you can’t see the slides above, click here

Additional videos here

Startup Tools here

Listen the blog post here

Download the podcast here

Lessons Learned in Medical Devices

This post is part of our series on the National Science Foundation I-Corps Lean LaunchPad class in Life Science and Health Care at UCSF. Doctors, researchers and Principal Investigators in this class got out of the lab and hospital talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them.  The class had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

We are redefining how translational medicine is practiced. It’s Lean, it’s fast, it works and it’s unlike anything else ever done.

—–

Sometimes teams win when they fail.

Knox Medical Devices was building a Spacer which contained a remote monitoring device to allow for intervention for children with Asthma . (A Spacer is a tube between a container of Asthma medicine (in an inhaler) and a patient’s mouth.The tube turns the Asthma medicine into an aerosol.)Asthma

Knox’s spacer had sensors for basic spirometry measurements (the amount of air and how fast it’s inhaled and exhaled) to see how well the lung is working. It also had a Nitrous Oxide sensor to provide data on whether the lungs airways are inflamed, an inhaler attachment and a GPS tracking device.

Knox SpacerThe Spacer hardware was paired with data analysis software for tracking multiple facets of asthma patients.

The Knox team members are:

Allan May founder of Life Science Angels was the Medical Device cohort instructor. Alex DiNello CEO at Relievant Medsystems was their mentor.

The Knox team was a great mix of hands-on device engineers and business development. They used agile engineering perfectly to continually test variants of their Minimum Viable Product (MVP’s) in front of customers often and early to get immediate feedback.

Knox was relentless about understanding whether their device was a business or whether it was technology in search of a market. In 10 weeks they had face-to-face meetings with 117 customers, tested 33 hypotheses, invalidated 19 of them and 53 instructor and mentor interactions.

Here’s Knox Medical’s 2 minute video summary

If you can’t see the video above, click here

Knox was a great example of having a technology in search of a customer. The initial hypothesis of who would pay for the device – parents of children with asthma – was wrong and resulted in Knox’s first pivot in week 4. By week 6 they had discovered that; 1) Peak Flow Meters are not as heavily prescribed as they thought, 2) Insurance company reimbursement is necessary for anything upwards of $15, 3) Nitrous Oxide testing isn’t currently used to measure asthma conditions.

After the pivot they the found that the most likely users of their device would be low income Asthma patients who are treated at Asthma clinics funded by federal, state or county dollars. These clinics reduce hospitalization but Insurers weren’t paying to cover clinic costs nor would they cover the use of the Knox device. The irony was that those who most needed the Knox device were those who could least afford it and wouldn’t be able get it.

Watch their Lesson Learned presentation below. Listen to the comments from Allan May the Device instructor at the end.

If you can’t see the video above, click here

In the end Knox, like a lot of startups in Life Science and Health Care, discovered that they had a multi-sided market.  They realized late in the class the patients (and their families) were not their payers – their payers were the insurance companies (and the patients were the users.)  If they didn’t have a compelling value proposition for the insurers (cost savings, increased revenue, etc.) it didn’t matter how great the technology was or how much the patients would benefit.

The Knox Medical Device presentation slides are below. Don’t miss the evolution of their business model canvas in the appendix. It’s a film strip of the entrepreneurial process in action.

If you can’t see the slides above, click here

Knox is a great example of how the Lean LaunchPad allows teams to continually test hypotheses and fail fast and inexpensively. They learned a ton. And saved millions.

Lessons Learned

  • In medical devices, understanding reimbursement, regulation and IP is critical
  • Sometimes teams win when they fail
Download the podcast here

We’ve seen the Future of Translational Medicine and it’s Disruptive

A team of 110 researchers and clinicians, in therapeutics, diagnostics, devices and digital health in 25 teams at UCSF, has just shown us the future of translational medicine.  It’s Lean, it’s fast, it works and it’s unlike anything else ever done.

It’s going to get research from the lab to the bedside cheaper and faster.

Welcome to the Lean LaunchPad for Life Sciences and Healthcare (part of the National Science Foundation I-Corps).

This post is part of our series on the Lean Startup in Life Science and Health Care.

——–

Our class talked to 2,355 customers, tested 947 hypotheses and invalidated 423 of them.  They had 1,145 engagements with instructors and mentors. (We kept track of all this data by instrumenting the teams with LaunchPad Central software.)

In a packed auditorium in Genentech Hall at UCSF, the teams summarized what they learned after 10 weeks of getting out of the building. This was our version of Demo Day – we call it “Lessons Learned” Day. Each team make two presentations:

  • 2 minutes YouTube Video: General story of what they learned from the class
  • 8 minute Lessons Learned Presentation: Very specific story about what they learned in 10 weeks about their business model

In the next few posts I’m going to share a few of the final “Lessons Learned” presentations and videos and then summarize lessons learned from the teaching team.

Magnamosis
Magnamosis is a medical device company that has a new way to create a magnetic compression anastomosis (a surgical connection between two tubular structures like the bowel) with improved outcomes.

anastomosis

Team Members were: Michael Harrison (the father of fetal surgery), Michael Danty, Dillon Kwiat, Elisabeth Leeflang, Matt Clark.  Jay Watkins was the team mentor. Allan May and George Taylor were the medical device cohort instructors.

Their initial idea was that making an anastomosis that’s better, faster and cheaper will have surgeons fighting to the death to get a hold of their device.  magnamosisThey quickly found out that wasn’t the case.  Leak rates turned out to a bigger issue with surgeons and a much larger market.

Here’s their 2 minute video summary

If you can’t see the video above, click here.

Watch their Lessons Learned video below and see how a team of doctors learned about product/market fit, channels and pricing.

If you can’t see the video above, click here

Their slide deck is below. Don’t miss the evolution of their business model in the Appendix.

If you can’t see the presentation above, click here

The best summary of why Scientists, Engineers and Principal Investigators need to get out of the building was summarized by Dr. Harrison below. After working on his product for a decade listen to how 10 weeks of the Lean LaunchPad class radically changed his value proposition and business model.

If you can’t see the video above, click here.

For further reading:

Listen to the blog post here

Download the podcast here

It’s Time to Play Moneyball: The Investment Readiness Level

Investors sitting through Incubator or Accelerator demo days have three metrics to judge fledgling startups – 1) great looking product demos, 2) compelling PowerPoint slides, and 3) a world-class team.

We think we can do better.

We now have the tools, technology and data to take incubators and accelerators to the next level. Teams can prove their competence and validate their ideas by showing investors evidence that there’s a repeatable and scalable business model. And we can offer investors metrics to play Moneyball – with the Investment Readiness Level.

Here’s how.

————–

We’ve spent the last 3 years building a methodology, classes, an accelerator and software tools and we’ve tested them on ~500 startups teams.

  • A Lean Startup methodology offers entrepreneurs a framework to focus on what’s important: Business Model Discovery. Teams use the Lean Startup toolkit: the Business Model Canvas + Customer Development process + Agile Engineering. These three tools allow startups to focus on the parts of an early stage venture that matter the most: the product, product/market fit, customer acquisition, revenue and cost model, channels and partners.

Lean moneyball

  • An Evidence-based Curriculum (currently taught in the Lean LaunchPad classes and NSF Innovation Corps accelerator). In it we emphasize that a) the data needed exists outside the building, b) teams use the scientific method of hypothesis testing c) teams keep a continual weekly cadence of:
    • Hypothesis – Here’s What We Thought
    • Experiments – Here’s What We Did
    • Data – Here’s What We Learned
    • Insights and Action – Here’s What We Are Going to Do Next

Evidence moneyball

  • LaunchPad Central software is used to track the business model canvas and customer discovery progress of each team. We can see each teams hypotheses, look at the experiments they’re running to test the hypotheses, see their customer interviews, analyze the data and watch as they iterate and pivot.

LPC

We focus on evidence and trajectory across the business model. Flashy demo days are great theater, but it’s not clear there’s a correlation between giving a great PowerPoint presentation and a two minute demo and building a successful business model. Rather than a product demo – we believe in a “Learning Demo”. We’ve found that “Lessons Learned” day showing what the teams learned along with the “metrics that matter” is a better fit than a Demo Day.

“Lessons Learned” day allows us to directly assess the ability of the team to learn, pivot and move forward. Based on the “lessons learned” we generate an Investment Readiness Level metric that we can use as part of our “go” or “no-go” decision for funding.

Some background.

NASA and the Technology Readiness Level (TRL)
In the 1970’s/80’s NASA needed a common way to describe the maturity and state of flight readiness of their technology projects.  They invented a 9-step description of how ready a technology project was.  They then mapped those 9-levels to a thermometer.NASA TRL

What’s important to note is that the TRL is imperfect. It’s subjective. It’s incomplete.  But it’s a major leap over what was being used before.  Before there was no common language to compare projects.

The TRL solved a huge problem – it was a simple and visual way to share a common understanding of technology status.  The U.S. Air Force, then the Army and then the entire U.S. Department of Defense along with the European Space Agency (ESA) all have adopted the TRL to manage their complex projects. As simple as it is, the TRL is used to manage funding and go/no decisions for complex programs worldwide.

We propose we can do the same for new ventures – provide a simple and visual way to share a common understanding of startup readiness status. We call this the Investment Readiness Level . 

The Investment Readiness Level (IRL)
The collective wisdom of venture investors (including angel investors, and venture capitalists) over the past decades has been mostly subjective. Investment decisions made on the basis of “awesome presentation”, “the demo blew us away”, or “great team” is used to measure startups. These are 20th century relics of the lack of data available from each team and the lack of comparative data across a cohort and portfolio.

Those days are over.

Hypotheses testing and data collection
We’ve instrumented our startups in our Lean LaunchPad classes and the NSF I-Corps incubator using LaunchPad Central to collect a continuous stream of data across all the teams.  Over 10 weeks each team gets out and talks to 100 customers. And they are testing hypotheses across all 9 boxes in the business model canvas.

We collect this data into a Leaderboard (shown in the figure below) giving the incubator/accelerator manager a single dashboard to see the collective progress of the cohort. Metrics visible at a glance are number of customer interviews in the current week as well as aggregate interviews, hypotheses to test, invalidated hypotheses, mentor and instructor engagements. This data gives a feel for the evidence and trajectory of the cohort as a whole and a top-level of view of each teams progress.

leaderboard moneyball

Next, we have each team update their Business Model Canvas weekly based on the 10+ customer interviews they’ve completed.

canvas updates moneyball

The canvas updates are driven by the 10+ customer interviews a week each team is doing. Teams document each and every customer interaction in a Discovery Narrative. These interactions provide feedback and validate or invalidate each hypothesis.

disovery 10 moneyball

Underlying the canvas is an Activity Map which shows the hypotheses tested and which have been validated or invalidated.

activty updates moneyball

All this data is rolled into a Scorecard, essentially a Kanban board which allows the teams to visualize the work to do, the work in progress and the work done for all nine business model canvas components.

scorecard update moneyball

Finally the software rolls all the data into an Investment Readiness Level score.

IRL

MoneyBall
At first glance this process seems ludicrous. Startup success is all about the team. Or the founder, or the product, or the market – no metrics can measure those intangibles.

Baseball used to believe that as well. Until 2002 – when the Oakland A’s’ baseball team took advantage of analytical metrics of player performance to field a team that competed successfully against much richer competitors.

Statistical analysis demonstrated that on-base percentage and slugging percentage were better indicators of offensive success, and the A’s became convinced that these qualities were cheaper to obtain on the open market than more historically valued qualities such as speed and contact. These observations often flew in the face of conventional baseball wisdom and the beliefs of many baseball scouts and executives.

By re-evaluating the strategies that produce wins on the field, the 2002 Oakland A’s spent $41 million in salary, and were competitive with the New York Yankees, who spent $125 million.

Our contention is that the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to have a robust and consistent data set across teams. While it doesn’t eliminate great investor judgement, pattern recognitions skills and mentoring – it does provide them the option to play Moneyball.

if you can’t see the video above click here

Last September Andy Sack, Jerry Engel and I taught our first stealth class for incubator/accelerator managers who wanted to learn how to play Moneyball.

We’re offering one again this January here.

Lessons Learned

  • It’s not clear there’s a correlation between a great PowerPoint presentation and two minute demo and building a successful business
  • We now have the tools and technology to take incubators and accelerators to the next step
  • We focus on evidence and trajectory across the business model
  • The data gathered can generate an Investment Readiness Level score for each team
  • the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to play Moneyball

Listen to the podcast here

Download the podcast here

Lean LaunchPad for Life Sciences – Revenue Streams

We’re teaching a Lean LaunchPad class for Life Sciences and Health Care (therapeutics, diagnostics, devices and digital health) at UCSF with a team of veteran venture capitalists. The class has talked to 2,056 customers to date.

This post is an update of what we learned about life science revenue models.

Life Science/Health Care revenue streams differ by Category
For commercialization, the business model (Customers, Channel, Revenue Model, etc.) for therapeutics, diagnostics, devices, bioinformatics and digital health have very little in common.

This weeks topic was revenue streams – how much cash the company can generate from each customer segment. Revenue streams have two parts: the revenue strategy and the pricing tactics.

Figuring out revenue strategy starts by gaining a deep understanding of the target customer(s). Setting a revenue strategy starts with understanding the basics about the customer segments:

  • who’s the user, the recommender, buyer, and payer
  • How the target customer currently purchases goods and services and how much they currently pay for equivalent products
  • Their willingness to pay for value versus lowest cost?
  • How much budget they have for your type of product?

Revenue strategy asks questions like, “Should we offer cost-based or value-based pricing.  How about demand-based pricing? Freemium? Do we price based on hardware sales or do we offer hardware plus consumables (parts that need to be disposed or replaced regularly)? Do we sell a single software package or a subscription?  These strategy hypotheses are tested against the target customer segment(s).

Once you’ve established a revenue strategy the pricing tactics follow. Pricing is simply “how much can I charge for the product using the selected revenue strategy?”  Pricing may be as simple as setting a dollar value for hardware or software, or as complicated as setting a high price and skimming the market or setting a low price as a loss leader.

You can get a feel for how each of the cohorts address the Revenue Streams by looking at the Revenue lectures below – covering the therapeutics, diagnostics, devices and digital health cohorts.

At the end of the lectures you can see a “compare and contrast” video and a summary of the differences in distribution channels.

Diagnostics

Week 5 Todd Morrill Instructor 

If you can’t see the presentation above click here

Digital Health

Week 5 Abhas Gupta Instructor 

If you can’t see the presentation above click here

Devices

Week 5 Allan May Instructor 

If you can’t see the presentation above click here

Therapeutics

Week 5 Karl Handelsman Instructor 

If you can’t see the presentation above click here

Life Science and Health Care Differences in Revenue Streams
”
This weeks lecture and panel was on Revenue; how much cash the company can generate from each customer segment – and the strategy and tactics to do so. Therapeutics, diagnostics, devices and digital health use different Revenue Strategies and Pricing Tactics, in the video and the summary that follows the instructors compare and contrast how they differ.

If you can’t see the video above click here

Therapeutics (Starting at 0:30)

  • Therapeutics revenue is from drug companies not end users
  • 18 months to first revenue from a deal
  • Predicated on delivering quality data to a company
  • Deal can be front-end or back-end loaded
  • Quality of the data has to be extremely high for a deal

Diagnostics (Starting at 4:10)

  • Diagnostic revenue is from end users: a hospital or clinical lab
  • You need to figure out value of your product but…
  • Pricing is capped by your reimbursement (CPT) code limits
  • Reimbursement strategy is paramount, design to good codes avoid bad ones
  • Find a reimbursement code consultant
  • Don’t do cost-based pricing… go for value-based pricing

Medical Devices (Starting at 8:23)

  • There really is no such thing as a perfect First Generation Medical Device
    • So Medical Device companies often start with a Volkswagen product and then build to the Ferrari product
  • Revenue models are typically direct product sales
  • Don’t do cost-based pricing… go for value-based pricing, especially where your device lowers the treatment costs of the patient
  • In most cases, pricing is capped by your reimbursement (CPT) code limits
    • Or pricing can be capped by what competitors offer, unless you can demonstrate superior cost savings
    • In a new market there is no reimbursement code but if you show high cost-savings you can get a high reimbursement rate
  • A risk in device hardware is getting trapped in low-volume manufacturing with low margins and run out of cash

Digital Health (Starting at 10:35)

  • Digital Health revenue models are often subscription models to a company per month across a large number of users
    • Intermediation fees – where you broker a transaction – are another source of revenue (i.e. HealthTap)
    • Advertising is another digital health revenue model, but requires at least 10 million users to have a meaningful model, but can be lower if you have higher value uses like specialist physicians because  you can charge dollars not cents
  • Don’t do cost-based pricing… go for value-based pricing
    • Value-based pricing is based on the needs you’ve learned from the customer segment and the strength of your product/market fit
      • the sum of customer needs + product/market fit = the pricing you can achieve

Lessons Learned

  • Each of these Life Science domains has a unique revenue strategy and pricing tactic
  • In therapeutics revenue comes in lump milestone payments from drug companies based on quality data
  • Diagnostics revenue comes value pricing to hospital or clinical lab
    • capped by reimbursement (CPT) code limits
  • Device pricing starts by offering an initial value-priced base product and then following up with a fully featured product
    • capped by reimbursement (CPT) code limits
  • Digital health products use subscription value pricing. Alternatively may use advertising revenue model

Listen to the podcast here

Download the podcast here

Well They “Should” be Our Customers

When scientists and engineers who’ve been working in the lab for years try to commercialize their technology they often get trapped by their own beliefs  – including who the customers are, what features are important, pricing etc.

I hear nothing

——–

One the key tenets of the Lean LaunchPad class is that every week each team gets out of the building and talks to 10+ customers/partners to validate a new part of their business model.  Back in class they present their findings to their peers and teaching team in a 10 minute Lessons Learned presentation. One of the benefits of the class is that the teams get immediate unvarnished feedback on their strategy.

For researchers and clinicians who’ve been working on a project in the lab for years, getting out of the building and talking to customers at times creates cognitive dissonance.  While they’ve been in the lab they had a target customer in mind. However when they leave the building and start talking to these  supposed customers there’s almost always a surprise when the customer is not interested in the product.

Often when they consistently hear that their expected customers aren’t interested the first reaction is “the customers just don’t get it yet.”  Rather than testing a new customer segment they keep on calling on the same group – somehow thinking that “we just need to explain it better.”

Some times it takes a nudge from the teaching team to suggest that perhaps looking at another customer segment might be in order.

They Should be Our Customers
The Mira Medicine Team is trying to accelerate the path to the right treatment for each patient in complex Central Nervous System diseases. They spent years building their first tool MS Bioscreen, which was developed for the physicians at the UCSF Dept of Neurology. So they naturally believed that their first customers would be neurologists.

This was a very smart team who ran into the same problem almost every smart researcher attempting to commercialize science faces.  Here’s what happened.

If you can’t see the video click here.

Listen for:

0:35 “Our primary customer we built this app for was neurologists…

1:00 “(but neurologists have told us) your prototype is interesting… and probably some features are nice to haves…

2:26 “What’s special about neurology?  Doesn’t cardiology and oncology have problems like this?

3:00 “Is neurology a key component of what you’re trying to do?

3:15 “I’ve worked on this for two years…”

3:24 “You’ve already done too much prototyping work. You’re hung up on the prototype.”

3:29 “You have a square peg you’er trying to jam in a round hole…”

3:43 “Don’t be afraid to think laterally”

Postscript: 70 customers later they no longer talking to neurologists.

Lessons Learned

  • Don’t get trapped by your own beliefs
  • When reality outside the building doesn’t match your hypotheses – test alternate hypotheses
  • Most of the time your vision is just a hallucination

Listen to the podcast here

Download the podcast here

Lean LaunchPad for Life Sciences – Distribution Channels

We’re teaching a Lean LaunchPad class for Life Sciences and Health Care (therapeutics, diagnostics, devices and digital health) at UCSF with a team of veteran venture capitalists. The class has talked to 1,780 customers to date.

This post is an update of what we learned about life science distribution channels.

Life Science/Health Care distribution channels differ by Category
It turns out that for commercialization, the business model (Customers, Channel, Revenue Model, etc.) for therapeutics, diagnostics, devices, bioinformatics and digital health have very little in common.

This weeks topic was distribution channels; how your product gets from your company to your potential customer segments. You can get a feel for how each of the cohorts address the channel by looking at the distribution channel lectures below – covering the therapeutics, diagnostics, devices and digital health cohorts.

At the end of the lectures you can see a “compare and contrast” video and a summary of the differences in distribution channels.

Diagnostics

Week 3 Todd Morrill Instructor 

If you can’t see the presentation above click here

Digital Health

Week 3 Abhas Gupta Instructor 

If you can’t see the presentation above click here

Devices

Week 3 Allan May Instructor 

If you can’t see the presentation above click here

Therapeutics

Week 3 Karl Handelsman Instructor 

If you can’t see the presentation above click here

Life Science and Health Care Differences in Distribution Channels
This weeks lecture and panel was on distribution channels; how your product gets from your company to your potential customer segments. Therapeutics, diagnostics, devices and digital health use different different channels, in the video and the summary that follows the instructors compare and contrast how they differ.

If you can’t see the video above click here

Medical Devices (Starting at 0:50)

  • Medical Device Distribution Channels in general are a sales team hired directly by the company.
    • A sales team typically includes a sales person and clinical applications specialists.
    •  The specialists help train and educate physician users. They assist with the sale and work with marketing to create demand.
  • Some device industries are controlled by distributors (indirect sales.)
    • Distributors tend to resell commodity products from multiple suppliers.
  • Channel Cost =  $350-400,000 per sales team. On average there’s 1 clinical applications specialist to 2 salespeople.  A lean rollout for a startup would be 4-5 sales people plus 2-3 clinical applications specialists at a cost of ~$2.5 million/year
    • Increasing the number of sales people much past 4-5 for a rollout does not proportionally increase revenue in most cases, because you are on the front end of early adopters and wrestling to overcome and reduce the sales learning curve
    • Travel and Entertainment is a big part of the sales budget since they are all flying weekly to cover accounts
  • 90-180 days for salespeople to become effective
  • Expect little or no revenue for 2- 3 quarters after they start
  • Major reason for failure = hiring sales and marketing staff too quickly
  • Generally an Educational Sale – Hire sales and clinical people first to help early adopters, such as Key Opinion Leaders (KOL’s), master the learning curve with your device so they can write and present papers to influence their peers 

Diagnostics (Starting at 5:16)

  • Diagnostic Channels = Direct sales in the US, with limited Distributor options
    • Many Distributors in Europe and in Asia
    • Sold to hospital laboratories, reference laboratories, or performed in CLIA labs
  • Channel Cost = $350,000+ per supported salesperson
  • Direct to consumer is a (rapidly) growing channel

Digital Health (Starting at 7:25)

  • Digital Health Channels = Direct Sales but you’re selling software to both end users and enterprises
  • Can use existing tech channels and new emerging channels such as Wellness platforms. (Audax Health, Humana Vitality, ShapeUp, Redbrick Health, Limeade)
  • Cloud-based Electronic Medical Records (EMR) are quickly becoming another distribution platform
  • App Stores, and Box are also channels for consumers and enterprise customers, respectively

Therapeutics (Starting at 10:17)

  • Therapeutics Channel = what you’re selling in the early stage is data and Intellectual Property to the pharmaceutical and biotech companies
  • Complicated Sales process – takes 18 months
  • Led by the CEO with a dedicated business development person and your science team
  • You need to define the data they need – this is influenced by how they view their pipeline, and how your technology can fill gaps in their pipeline
  • Pharmaceutical and biotech companies have therapeutics heads, technology scouts and business development people all searching for technology deals to fill their pipeline
  • This is a bound problem – there’s probably 80 people you need to know that make up your channel

Lessons Learned

  • Each of these Life Science domains has a unique distribution channel
  • In Devices innovative products require hiring direct sales people
    • but for commodity device products you may use a distributor
  • Diagnostics requires a direct sales force in the U.S.
    • Distributors in Europe and in Asia
  • In Digital Health direct sales is a possible channel, as are traditional software channels (App Stores, Box, etc.)
    • other DHealth channels such as Wellness Platforms, and cloud-based EMR’s are also emerging
  • In therapeutics it’s a direct sale of data and Intellectual property
    • led by the CEO with a dedicated business development person and your science team

Listen to the podcast here

Download the podcast here

A New Way to Look at Competitors

Every startup I see invariably puts up a competitive analysis slide that plots performance on a X/Y graph with their company in the top right.

Competitive XY

The slide is a holdover from when existing companies launched products into crowded markets. Most of the time this graph is inappropriate for startups or existing companies creating new markets.

Here’s what you need to do instead.

——-

The X/Y axis competitive analysis slide is a used by existing companies who plan to enter into an existing market.  In this case the basis of competition on the X/Y axes are metrics defined by the users in the existing market.

This slide typically shows some price/performance advantage.  And in the days of battles for existing markets that may have sufficed.

But today most startups are trying to ressegment existing markets or create new markets. How do you diagram that? What if the basis of competition in market creation is really the intersection of multiple existing markets?  Or what if the markets may not exist and you are creating one?

We need a different way to represent the competitive landscape when you are creating a business that never existed or taking share away from incumbents by resegmenting an existing market.

Here’s how.

The Petal Diagram
I’ve always thought of my startups as the center of the universe. So I would begin by putting my company in the center of the slide like this.

Slide1In this example the startup is creating a new category –  a lifelong learning network for entrepreneurs. To indicate where their customers for this new market would come from they drew the 5 adjacent market segments: corporate, higher education, startup ecosystem, institutions, and adult learning skills that they believed their future customers were in today. So to illustrate this they drew these adjacent markets as a cloud surrounding their company. (Unlike the traditional X/Y graph you can draw as many adjacent market segments as you’d like.)

Slide2Then they filled in the market spaces with the names of the companies that are representative players in each of the adjacent markets.companies updated

Then they annotated the private companies with the amount of private capital they had raised. This lets potential investors understand that other investors were interested in the space and thought it was important enough to invest. (And plays on the “no VC wants to miss a hot space” mindset.)

Slide4

Finally, you could show the current and projected market sizes of the adjacent markets which allows the startups to have a “how big can our new market be?” conversation with investors.  (If you wanted to get fancy, you could scale the size of the “petals” relative to market size.)

Slide5

The Petal Diagram drives your business model canvas
What the chart is saying is, “we think our customers will come from these markets.”  That’s handy if you’re using a Lean Startup methodology because the Petal Chart helps you identify your first potential customer segments on the business model canvas.add the canvasYou use this chart to articulate your first hypotheses of who are customers segments you’re targeting.  If your hypotheses about the potential customers turn out to be incorrect, and they aren’t interested in your product, then you go back to this competitive diagram and revise it.

Lessons Learned

  • X/Y competitive graphs are appropriate in an existing market
  • Mapping potential competitors in new or resegmented markets require a different view – the Petal diagram
  • The competitive diagram is how develop your first hypotheses about who your customers are

Update: I’ve heard from a few entrepreneurs who used the diagram had investors tell them “”it looks like you’re being surrounded, how can you compete in that market?”

Those investors have a bright future in banking rather than venture capital.

Seriously, I would run away fast from a potential investor who doesn’t or can’t understand that visualizing the data doesn’t increase or decrease the likelihood of success. It only provides a better way to visualize potential customer segments.
Download the podcast here

Lean LaunchPad for Life Sciences – Value Proposition and Customers

We’re deep into teaching a Lean LaunchPad class for Life Sciences and Health Care (therapeutics, diagnostics, devices and digital health) at UCSF with a team of veteran venture capitalists. (The class has talked to 1,440 customers to date.)shutterstock_81663952

One of the objectives of the class was to become a Life Science Center of Excellence for the National Science Foundation Innovation Corps. This meant capturing domain specific commercialization expertise for therapeutics, diagnostics, devices and digital health so others can teach this.

Part 1 of this post described the issues in the therapeutics drug discovery pipeline. Part 2 covered medical devices and digital health. Part 3 described what we’re going to do about it.  Part 4 gave a snapshot of what one our teams found the first week outside the building.

This is an update of our progress.

It Takes A Village
We’re teaching 110 students in 28 teams across therapeutics, diagnostics, devices and digital health. Teams are made up of clinicians, researchers, and post docs, (some of the team members include the Chief of  UCSF General Surgery, the inventor of Fetal Surgery, etc.)

Each of the four cohorts is taught by an experienced life science venture capitalist. Alan May for devices, Karl Handelsman for therapeutics Abhas Gupta for digital health and Todd Morrill for diagnostics.

Jerry Engel and Jim Hornthal, both who taught the National Science Foundation I-Corps classes, are the senior instructors. The UCSF Office of Innovation and Technology (Erik Lium and Stephanie Marrus) is the reason the program exists.

Each of the teams is assigned a mentor to match their domainv(Head of device R&D of Phillips, Genetech, Crescendo, CTO of UCSF, venture capitalists from Sofinnova, Burrill, Lightstone, M34, etc.)

Class Organization – Lots of Moving Parts
Our class meets weekly. We first meet as one group and then we break out the therapeutics, diagnostics, devices and digital health into their own cohorts. The teams present what they learned talking to 10-15 customer/week, and get comments, suggestions and critiques from their teaching team.  The instructor then presents  a cohort specific lecture explaining how the business model for their area (therapeutics, diagnostics, devices and digital health) builds on and/or differs from the canonical business model in the online Udacity lectures (which the students had to watch as homework.)

We get back together as one group and the instructors share what they learned as they compare and contrast the differences between therapeutics, diagnostics, devices and digital health.  We’ve recorded these panels for each part of the business model canvas.

The framework of the class looks like this:

Lean LaunchPad for Life Sciences

Life Science/Health Care is not a single Category
One of the reasons I teach is because of how much I learn. One of early surprises of this class for me is finding out that the broad category of “Life Sciences” fails to provide the important nuances of each category to entrepreneurs, investors, educators, policy makers, etc. It turns out that for commercialization, the business model (Customers, Channel, Revenue Model, etc.) for therapeutics, diagnostics, devices, bioinformatics and digital health have very little in common.

You can get a feel for how different by looking at the first two weeks of lectures – covering value proposition and customer segment – from each of the therapeutics, diagnostics, devices and digital health cohorts.

Then at the end of the lectures you can see a “compare and contrast” video and a summary of the differences.

Diagnostics

Week 1 Todd Morrill Instructor 

If you can’t see the presentation above click here

Week 2 Todd Morrill Instructor

If you can’t see the presentation above click here

Digital Health

Week 1 Abhas Gupta Instructor 

If you can’t see the presentation above click here

Week 2 Abhas Gupta Instructor

If you can’t see the presentation above click here

Devices

Week 1 Allan May Instructor 

If you can’t see the presentation above click here

Week 2 Allan May Instructor

If you can’t see the presentation above click here

Therapeutics

Week 1 Karl Handelsman Instructor 

If you can’t see the presentation above click here

Week 2 Karl Handelsman Instructor

If you can’t see the presentation above click here

Life Science and Health Care Differences

Once we realized that the four cohorts of therapeutics, diagnostics, devices and digital health were so different we decided to have the instructors compare and contrast how they’re different for each part of the business model. We’ll be posting these “compare and contrast” videos for every week of the class.

If you can’t see the video above click here

Therapeutics (Starting at 0:30)

  • Therapeutics customer = pharma and biotech companies
  • Therapeutics Pain & gain = be better than what these companies have in their own drug development pipeline
  • Therapeutics Validation =  18 months to a first deal with a potential customer – well before FDA trials, and even before preclinical stage

Digital Health (Starting at 2:40)

  • Digital Health Customer = typically consumer end users
  • Digital Health Pain & gain = product/market fit needs to be a need, and the value proposition must address it.  “Nice to have’s” do not equal a customer need.
  • Digital Health Validation = large scale adoption

Medical Devices (Starting at 6:00)

  • Medical Device customers = short term: physicians in private practice and hospital, long term: medical device companies
  • Medical Device Customer Goal – figure out the Minimal Viable Product.  No such thing as a perfect first generation product that targets a specific physician/customer segment.
  • Medical Device Validation= 95% of device startups are acquired by a medical device company, 5% build a large standalone company

Diagnostics (Starting at 10:45)

  • Diagnostic Customers = short term In vitro diagnostics are sold to a hospital laboratory or standalone lab, long-term you’ll be bought by Abbott, Roche, etc.

Lessons Learned

  • Each of these Life Science domains has a unique business model
  • Commercialization of therapeutics, diagnostics, devices and digital health all require the Principal Investigators / founders outside their building talking to customers, partners, regulators
  • Only the Principal Investigators / founders have the authority and insight to pivot when their hypotheses are incorrect
  • The Lean Startup process and the Lean LaunchPad class can save years in commercialization in these domains
  • This can be taught

Listen to the podcast here

Download the podcast here

300 Teams in Two Years

This is the start of the third year teaching teams of scientists (professors and their graduate students) in the National Science Foundation Innovation Corps (I-Corps). This month we’ve crossed ~300 teams in the first two years through the program.

I-Corps is the  accelerator that helps scientists bridge the commercialization gap between their research in their labs and wide-scale commercial adoption and use.

I-Corps bridges the gap between public support of basic science and private capital funding of new commercial ventures. It’s a model for a government program that’s gotten the balance between public/private partnerships just right.

While a few of the I-Corps teams are in web/mobile/cloud, most are working on advanced technology projects that don’t make TechCrunch. You’re more likely to see their papers (in material science, robotics, diagnostics, medical devices, computer hardware, etc.) in Science or Nature. The program pays scientists $50,000 to attend the program and takes no equity.

Currently there are 11 U.S. universities teaching the Lean LaunchPad curriculum organized as I-Corps “nodes” across the U.S.  The nodes are now offering their own regional versions of the Lean LaunchPad class under I-Corps.

The NSF I-Corps uses everything we know about building Lean Startups and Evidence-based Entrepreneurship to connect innovation to entrepreneurship. It’s curriculum is built on a framework of business model design, customer development and agile engineering – and its emphasis on evidence, Lessons Learned versus demos, makes it the worlds most advanced accelerator. It’s success is measured not only by the technologies that leave the labs, but how many U.S. scientists and engineers we train as entrepreneurs and how many of them pass on their knowledge to students. I-Corps is our secret weapon to integrate American innovation and entrepreneurship into every U.S. university lab.

Every time I go to Washington and spend time at the National Science Foundation or National Institute of Health I’m reminded why the U.S. leads the world in support of basic and applied science.  It’s not just the money we pour into these programs (~$125 billion/year), but the people who have dedicated themselves to make the world a better place by advancing science and technology for the common good.

I thought it was worth sharing the progress report from the Bay Area (Berkeley, Stanford, UCSF) I-Corps node so you can see what just one of the nodes was accomplishing. Multiply this by the NSF regional nodes across the U.S. and you’ll have a feeling for the scale and breadth of the program.

If you can’t see the presentation above click here

Glad to a part of it.

Lessons Learned

  • The U.S. government has built an accelerator for scientists and engineers
  • It’s scaled across the U.S.
  • The program has taught ~300 teams
  • Balance between public/private partnerships

Listen to the podcast here Download the podcast here
BTW, NCIIA is offering other accelerators and incubators a class to learn how to build their own versions of I-Corps here.

The Air Force Academy Gets Lean

I can always tell when one of my students has been in the military. They’re focused, they’re world-wise past their years, and they don’t break a sweat in the fast pace and chaotic nature of the class and entrepreneurship. Todd Branchflower took my Lean LaunchPad class having been entrepreneurial enough to convince the Air Force send him to Stanford to get his graduate engineering degree.

In class I teased Todd that while the Navy had me present my Secret History of Silicon Valley talk in front of 4,000 cadets at the Naval Post Graduate School, I had yet to hear from the Air Force Academy.  He promised that one day he would fix that.

True to his word, fast-forward three years and Todd is now Captain Todd Branchflower, teaching computer engineering at the Air Force Academy.  He extended an invitation to me to come out to the Air Force Academy to address the cadets and meet the faculty. Besides the talk I brainstormed with Todd and other faculty on how to integrate the Lean LaunchPad into the Air Force Academy Capstone engineering class (a Capstone class puts together all the pieces that a students has learned in his or her major.)

Here’s Todd’s story of how we got there and progress to date.

——-

Not That Long Ago
In 2007, I graduated United States Air Force Academy as a computer engineer and entered the Air Force’s acquisition corps, excited and confident about my ability to bring technology to bear for our airmen.

Graduation day with classmate Joseph Helton (right), killed in action in Iraq in 2009

Graduation day with classmate Joseph Helton (right), killed in action in Iraq in 2009

And I couldn’t have been put in a better place: testing the Air Force’s newest network security acquisitions. I was their technical man on the inside – making sure big defense contractors delivered on their promises. We were modernizing datacenters, buying vulnerability-scanning software, and adding intrusion detection appliances – all things typical of anyone running an enterprise-scale network..46th test sqd

I was in the thick of it – chairing telecons, tracking action items, and drafting test plans. I could recite requirements and concepts of operations from memory. I was jetsetting to team meetings and conferences across the country. I was busy.

Sure, I wasn’t working very closely with the airmen who were going to use the equipment.  But they called into the weekly telecons, right? And they were the ones who had given the program office the requirements from the outset. (Well, their bosses had.) And I’d distilled those requirements into system characteristics we could measure. Well, more measurable versions of the original requirements. And meeting the requirements was the most important thing, right?

Doing it Wrong
Here’s what I learned: I was doing it wrong. The way our process worked, customers were just a stakeholder that provided input – not drivers of the process. That meant that program offices were only accountable to a list of requirements, which were locked early. Success only consisted of passing tests against these requirements, not delighting our airmen. I began to wonder – how could we learn about user needs earlier?  How could we deliver them solutions more quickly?  More cheaply?

It was only after returning to Stanford and taking the Lean Launchpad class that I became convinced that a radically different, customer-centric approach was the solution. I returned to the Air Force Academy as an instructor in the Electrical and Computer Engineering Department, intent on spreading the gospel of Customer Development and Lean.academy ee

Our existing Capstone senior engineering design course followed the defense acquisition process; the focus of defense acquisition is to “nail down requirements” early and manage customer expectations to “avoid requirements creep”. I saw this as counter to the joint, iterative discovery process between entrepreneurs and customers I had experienced on my Lean Launchpad team.

I kept in touch with Steve as I started teaching. We discussed how the Lean Launchpad approach might find a place in our curriculum, and how it might be adapted to fit the unique Air Force Academy / military environment. We grew excited about how showing success here might prove a good model for how it could be done in the broader Air Force; how exposing future officers to the Lean philosophy might bring about change from within.

So when I invited Steve out to the Air Force Academy to speak last spring, there was more at stake than the talk.  We set up a meeting with our department head, Col Jeff Butler, and Capstone course director, LtCol Charlie Gaona, to pitch the idea.  They shared our enthusiasm about the impact it could have on our future design projects and how it might bring a change in perspective to our acquisition corps. They gave the go-ahead to send a pilot team through the program in the Fall semester, with the potential for it to be applied across the entire course if we delivered results.

I found a willing co-conspirator in Capt Ryan Silva, a star instructor who mentors a project named Neumimic, using technology to aid in the rehabilitation of patients with chronic loss of limb motion.  In the first year, they had developed a proof of concept around the Xbox Kinect – and Ryan had high hopes for the future. But he found some elements of the traditional systems engineering process cumbersome and frustrating to cadets. Ryan signed on to lead our test class.

V-Model of Systems Engineering
The current Capstone class follows the V-Model of Systems Engineering, with teams creating a detailed system design throughout the Fall semester and building their design in the Spring.

Vmodel

There are a series of formal reviews throughout the two semesters, in line with the Air Force acquisitions process.  Requirements and a concept of operations are presented at the first, the System Requirements Review.  Cadets receive instruction on the process in about a quarter of the course lessons.

What we decided to do instead was have semi-weekly informal reviews Lean Launchpad style, focusing on product hypotheses, customer interactions, learning, and validation / refinement.  We emphasize customer interaction via “getting out of the building” and rapid iteration through “cheap hacks”.  We’ve removed most of the structure and firm requirements from the original course in favor of a “whatever it takes” philosophy.  Instruction is presented in tandem with the reviews, focusing on areas we see as problematic.

Last year’s team meeting with Dr. Glen House at Penrose-St. Francis Hospital

Last year’s team meeting with Dr. Glen House at Penrose-St. Francis Hospital

Back to the Present
We’re about a quarter of the way through the fall semester. Team Neumimic consists of nine sharp cadets across multiple academic disciplines. Based on initial customer interactions, they divided themselves into two complementary but standalone teams. One will focus on design, execution, and measurement of therapy sessions – building on the original Xbox Kinect work.  The other will work on adjustable restriction of patient motion – forcing patients to use the proper muscles for each movement.

Here’s Ryan on the impact of the process change:

“Last year the team found themselves handcuffed to a process that required a 100% design solution on paper before we could even think about touching hardware…crazy right?! We spent the entire first semester nailing down requirements for a system that was supposed to meet the needs of stroke and traumatic brain injury patients as prescribed by their occupational therapists. For five months we slogged our way through the process emerged with a complete design for our system, custom-built to meet the needs of patients and doctors alike. Our design was flawless. We had nuts-and-bolts details all the way down to the schematic level. We were ready to build! The fact that we had yet to even see a patient or spend any real time with an occupational therapist had not even registered to us as a problem, until we were invited to watch a therapy session.

Our entire team walked out of the hospital ashen-faced and silent. We knew we had just wasted half the course designing a system that wouldn’t work. We were back to square one. The remainder of the course was spent in a frenzy of phone calls with doctors and therapists paired with many design reviews, but this time with our customers in the room. We were able to iterate a few solutions before we ran out of time, but the customers were thrilled with what they saw. I could only imagine what we could have accomplished if we didn’t waste the first half of the course on a solution that ultimately wasn’t what the customers wanted. I was fired up when Todd approached me with his idea to fundamentally change the way we did business.

So far the results have been incredible compared to last year. The team has learned more about the problem in a month than last year’s team learned in an entire semester. I’m not saying this year’s cadets are any more capable than last year’s; just that I believe this year’s team has been given a better chance to succeed.  They’re freed of a lot of stifling overhead and are embracing a process where requirements are derived from those who will actually use the system…imagine that! I’m excited to see what the team does with their remaining eight months.”

Current team members observing Dr. House conduct a therapy session

Current team members observing Dr. House conduct a therapy session

But we have experienced challenges in implementing this approach. Here’s what we’ve noticed so far:

In typical Lean Launchpad classes, students apply as teams with their own idea.  There’s also the potential for teams to pursue the opportunity beyond the class if they’re successful. In our Capstone, projects are predetermined and cadets are assigned based on preference and skill set.  Cadets will graduate and be commissioned as officers, doing various jobs throughout the Air Force. It’s highly unlikely they’ll be able to continue their project. These factors might make the initial motivation of our team less than that of other Lean Launchpad teams.  We found that early interactions with customers excited about their work went a long way to remedy this.

We’re offering cadets much less structure than they’re used to. Some cadets are uncomfortable with the ambiguity of the requirements (“What are you looking for?  What do I have to do to get an A?”).  I’d imagine this is typical of most high-performing students.

We’re trusting cadets with more freedom and less oversight than they’re used to.  There’s the potential for our trust to be abused.  I’m hopeful that our cadets rise to this challenge.  I think they’ll feel ownership of the project and empowerment, rather than see an opportunity to shirk responsibilities.

Since this course is a senior design experience, cadets expect to be “using their major”.  There’s the tendency for some to sit on the sideline if the pressing work isn’t directly related to their area of expertise.  It has taken some prodding for cadets to embrace the “hustler” mindset – to take any job necessary to move the team forward.

These are challenges we can overcome.  I know we’re moving in the right direction.  I know we have the right team and project to be successful.  I know our cadets will make us proud.

Up the hill!

Listen to post here
Download the post here

Follow

Get every new post delivered to your Inbox.

Join 151,382 other followers