Lean LaunchPad – For Deep Science and Technology

We just finished the 11th annual Lean LaunchPad class at Stanford — our first version focused on deep science and technology.

I’ve always thought of the class as a minimal viable product – testing new ideas and changing the class as we learn. This year was no exception as we made some major changes, all of which we are going to keep going forward.

  1. A focus on scientists and engineers. We created an additional Spring section of the class with a focus on commercializing inventions from Stanford’s scientists and engineers. The existing winter quarter of the class remains the same as we taught for the last 10 years – taking all students’ projects – e-commerce, social media, web, and mobile apps. This newly created Spring section focuses on scientists and engineers who want to learn how to commercialize deep science and technology – life sciences (medical devices, diagnostics, digital health, therapeutics,) semiconductors, health care, sensors, materials, artificial intelligence/deep learning, et al.
    This allowed us to emphasize how to differentiate a technical spec from a value proposition and expand on the parts of the business model that are unique for science and engineering startups. For example, life sciences versus commercial applications have radically different reimbursement, regulatory, clinical trials, scientific advisory boards, demand creation, etc. In addition, we found we needed to add new material on Intellectual Property, how to license inventions from the university, and discussions about team dynamics.  Going forward we’ll continue to offer the class in two sections with the second class focused on science and technology.
  2. Remote Discovery – As the pandemic forced teaching remotely, we’ve learned that customer discovery is actually more efficient using video conferencing. It increased the number of interviews the students were able to do each week. When Covid restrictions are over, we plan to add remote customer discovery to the students’ toolkit. It remains to be seen whether customers will remain as available on Zoom as they were during the pandemic. (See here for an extended discussion of remote customer discovery.) Remote discovery also allowed a bigger pool of potential interviews not bounded by geography. The quality of interviewees seemed to improve by this larger pool.
  3. Class size/configuration – For the past decade our class size was 8 teams of 4. This year we accepted 12 teams of 4. Previously all teams needed to sit through all 8 weekly presentations. That was tough in person and not sustainable via Zoom. This year, by moving into two breakout sections, we cut the number of presentations that each team sat through by half.  The new format allowed students and teaching staff to devote greater attention to each presentation.
  4. Adopt a team – in past years all instructors had office hours with all the teams. This year each instructor adopted three teams and saw them weekly for a half hour. Students really appreciated building a closer working relationship with one faculty member.
  5. Alumni as guest speakers – Most weeks we invited a past student to guest speak about their journey through the class, highlighting “what I wish I knew” and “what to pay attention to.”

Below are the Lessons Learned presentations from the Lean LaunchPad for deep science and technology, as well as additional learnings from the class.

During the quarter the teams spoke to 1,237 potential customers, beneficiaries, regulators – all via Zoom. Most students spent 15-20 hours a week on the class, about double that of a normal class.

Team Gloflow

Started on Week 1 as a pathology slide digitization service.
Ended in Week 10 as response prediction for cancer treatments.

If you can’t see the Gloflow video, click here

If you can’t see the Gloflow slides, click here

Team Loomia

Started on Week 1 as flexible e-textile circuit looking for a problem.
Ended in Week 10 as easy-to-integrate components for automotive suppliers.

If you can’t see the Loomia video, click here

If you can’t see the Loomia slides, click here

Team Skywalk

Started on Week 1 as wearable gesture control device for real and virtual worlds.
Ended in Week 10 as a future-proof gesture control solution for AR headsets and the Department of Defense.

If you can’t see the Skywalk video, click here

If you can’t see the Skywalk slides, click here

Team EdgeAI

Started on Week 1 as a custom silicon chip with embedded memories and a Machine Learning accelerator targeting low-power, high-throughput, and low-latency applications.
Ended in Week 10 as a chip enabling AI vision applications on next generation battery powered surveillance cameras.

If you can’t see the EdgeAI video click here

If you can’t see the EdgeAI slides, click here

Team MushroomX

Started on Week 1 as Drone pollination of crops.
Ended in Week 10 as autonomous button mushroom harvesting.

If you can’t see the MushroomX video, see here

If you can’t see the MushroomX slides, click here

Team RVEX

Started on Week 1 as a Biomimetic Sleeve as a Left Ventricular Assist Device.
Ended in Week 10 as a Platform technology as a right heart failure device.

If you can’t see the RVEX video, click here

If you can’t see the RVEX slides, click here

Team Pause

Started on Week 1 as a Menopause digital health platform that connects women to providers and other women.
Ended in Week 10 as a D2C Menopause symptom tracking app and on-demand telehealth platform that offers women a personalized and integrative approach to menopause care.

If you can’t see the Pause video, click here

If you can’t see the Pause slides, click here

Team Celsius

Started on Week 1 as an IOT hardware sensor for environmental quality and human presence.
Ended in Week 10 as hybrid work collaboration + employee engagement.

If you can’t see the Celsius video, click here

If you can’t see the Celsius slides, click here

Team TakeCare

Started on Week 1 as a platform for finding and managing at-home senior care.
Ended in Week 10 as a B2C platform for scheduling on-demand at-home senior care.

If you can’t see the TakeCare video, click here

If you can’t see the Take Care slides, click here

Team CareMatch

Started on Week 1 as AI to Match Patients to Post-Acute Care.
Ended in Week 10 as Skilled Nursing Facility-at-Home for Wound Care.

If you can’t see the CareMatch video, click here

If you can’t see the CareMatch slides, click here

Team NeuroDB

Started on Week 1 as Unstructured data Tableau-like tool.
Ended in Week 10 as Cloud-based Pandas dataframe.

If you can’t see the NeuroDB video click here

If you can’t see the NeuroDB slides, click here

Team Drova

Started on Week 1 as a provider for autonomous drone delivery for restaurants and grocery stores.
Ended in Week 10 as Fleet management software for autonomous drone delivery.

If you can’t see the Drova video click here

If you can’t see the Drova slides, click here

Student Comments
I normally don’t include student comments in these summaries, but this year’s summarized why – after a decade – we still teach the class. The students find the class hard and exhausting, and say their instructors are tough and demanding. Yet in the end, the class and the work they invest in is highly rewarding to them.

  • “Awesome course- one of the best I’ve taken so far. You get out what you put into it, but find a team you like working with, get ready to hustle and work hard, and trust the process. A must-take for entrepreneurs!”
  • “Absolutely crucial to starting a company for a first-time founder. Couldn’t imagine a better teaching team or learning environment.”
  • “Very worth taking, whether you want to do a start-up your own or not.”
  • “Recommend to everyone considering entrepreneurship or want to learn about it.”
  • “Great class if you are interested in learning about the Customer Discovery Model, but takes a lot of time and work.”
  • “Intense course where you learn through experience on how to build a startup. I came with a product and I learned to find a solution and how to build from there.”
  • “Incredible experience – really glad I took the class and happy with the outcome.”
  • “Steve Blank tells you your slides are ugly”
  • “Take this course if you get a chance, especially if you are a PhD student. Super useful and a different kind of learning than most case-based classes. Extremely experiential.”
  • “A great class to learn about customer discovery and entrepreneurship methodologies! The teaching team is incredibly experienced and very honest in their feedback. It is quite time intensive and heavily based on your team. Make sure to clarify expectations with your team beforehand and communicate.”
  • “Definitely recommend this course, it’s a great experience and will give you tools to launch your idea.”
  • “A really excellent course to take to learn about entrepreneurship! An invaluable opportunity you might not find anywhere else. The instructors are extremely knowledgeable veteran entrepreneurs who give all the support and encouragement needed.”

Diversity
In past years, the students in the class were mostly men, reflecting the makeup of the applicants. While Ann Miura-Ko was part of the original teaching team, having all male instructors for the last five years didn’t help. Mar Hershenson joined the teaching team in 2018 and made an all-out effort to recruit women to apply. In this new Spring section of the class Heidi Roizen and Jennifer Carolan joined us as instructors. Mar, Heidi and Jennifer are all successful VC’s. They sponsored lunch sessions, mixers and meetings with women entrepreneurs and alumni for female students interested in the class and for male students looking to work with a more diverse team. I am happy to report that as a result of many people’s hard work the gender balance in the class substantially changed. Our Spring cohort focused on deep science and tech had 51 students — 25 were women.

The lessons for me were: 1) the class had been unintentionally signaling a “boys-only” environment, 2) these unconscious biases were easily dismissed by assuming that the class makeup simply reflected the applicant pipeline, and 3) when in fact it required active outreach by a woman to change that perception and bring more women into the pipeline and teams.

Teaching Assistants (TAs)
Our Teaching Assistants keep all the moving parts of the class running. This year their job was even more challenging running the class virtually and they made it run like clockwork.

Each year’s TAs have continued to make the class better (although I must admit it was interesting to watch the TAs remove any student uncertainty about what they need to do week-to-week by moving to a more prescriptive syllabus. Originally, I had designed a level of uncertainty into the class to mimic what a real-world startup feel like.) However, the art of teaching this class is remembering that it wasn’t designed by a focus group.

A Great Class Endures Beyond Its Author
I’ve always believed that great classes continue to thrive after the original teachers have moved on. While I created the Lean LaunchPad methodology and pedagogy (how to teach the class), over the past decade the Stanford class has had ten additional instructors, thirty-three wonderful TA’s and ninety volunteer mentors.

In addition to myself the teaching team has been:

2011 Instructors: Ann Miura-ko, Jon Feiber
Lead TA: Thomas Haymore, TA’s: Felix Huber, Christina Cacioppo

2012 Instructors: Ann Miura-ko, Jon Feiber
Lead TA: Thomas Haymore, TA:, Stephanie Glass

2013 Instructors: Ann Miura-ko, Jon Feiber
Lead TA: Rick Barber, TA: Stephanie Glass

2014 Instructors: Jeff Epstein, Jim Hornthal
Lead TA: Soumya Mohan, TA: Stephanie Zhan, Asst: Gabriel Garza, Jennifer Tsau

2015 Instructors: Jeff Epstein, Steve Weinstein
TA’s: Stephanie Zhan, Gabriel Garza TAs: Jennifer Tsau, Akaash Nanda, Asst: Nick Hershey

2016 Instructors: Jeff Epstein, Steve Weinstein
Lead TA: Jose Ignacio del Villar TA’s: Akaash Nanda, Nick Hershey, Zabreen Khan, Asst: Eric Peter

2017 Instructors: Jeff Epstein, Steve Weinstein
Lead TA: Eric Peter TA’s: Nick Hershey, Lorel Sim Karan Singhal Asst: Jenny Xia

2018 Instructors: Jeff Epstein, Steve Weinstein, Mar Hershenson, George John
Lead TA: Jenny Xia TA’s: Anand Upender, Marco Lorenzon, Lorel Sim Asst: Parker Ence, Trent Hazy, Sigalit Perelson

2019 Instructors: Jeff Epstein, Steve Weinstein, Mar Hershenson, George John, Tom Bedecarre
Lead TA: Parker Ence, Trent Hazy TA’s: Marco Lorenzon, Sigalit Perelson, Lorel Sim Asst:, Ashley Wu

2020 Instructors: Jeff Epstein, Steve Weinstein, Mar Hershenson, George John, Tom Bedecarre
Lead TA: Marco Lorenzon, Ashley Wu TA’s: Sigalit Perelson, Gopal Raman

2021 – Winter Instructors: Jeff Epstein, Mar Hershenson, George John, Tom Bedecarre
Lead TA: Erica Meehan, Anand Lalwani, TA’s: Gopal Raman, Andrew Hojel

2021 – Spring Instructors: Steve Weinstein, Heidi Roizen, Jennifer Carolan, Tom Bedecarre
Lead TAs: Sandra Ha, Lorenz Pallhuber TA: Manan Rai

Our Decade of Mentors
The mentors (industry experts) who volunteer their time have been supported and coordinated by Tom Bedecarre and Todd Basche. Each mentor’s contribution gets graded by the student team they coached.

Bryan Stolle, Charles Hudson, Dan Martell, David Feinlab, David Stewart, Doug Camplejohn, Eric Carr, George Zachary, Gina Bianchini, Heiko Hubertz, Hiten Shah, Jason Davies, Jim Greer, Jim Smith, Jonathan Ebinger, Josh Schwarzapel, Joshua Reeves, Justin Schaffer, Karen Richardson, Marianne Wu, Masheesh Jain, Ravi Belani, Rowan Chapman, Shawn Carolan, Steve Turner, Sven Strohbad, Thomas Hessler, Will Harvey, Ashton Udall, Ethan Bloch, Jonathan Abrams, Nick O’Connor, Pete Vlastellica, Steve Weinstein, Adi Bittan, Alan Chiu, George Zachary, Jeff Epstein, Kat Barr, Konstantin Guericke, Michael Borrus, Scott Harley, Jorge Heraud, Bob Garrow, Eyal Shavit, Ethan Kurzweil, Jim Anderson, George John, Dan Manian, Lee Redden, Steve King, Sunil Nagaraj, Evan Rapoport, Haydi Danielson, Nicholas O’Connor, Jake Seid, Tom Bedecarre, Lucy Lu, Adam Smith, Justin Wickett, Allan May, Craig Seidel, Rafi Holtzman, Roger Ross, Danielle Fong, Mar Hershenson, Heather Richman, Jim Cai, Siqi Mou, Vera Kenehan, Phil Dillard, Susan Golden, Todd Basche, Robert Locke, Maria Amundson, Freddy Dopfel, Don Peppers, Rekha Pai, Radhika Malpani, Michael Heinrich, MariaLena Popo, Jordan Segall, Mike Dorsey, Katie Connor, Anmol Madan, Kira Makagon, Andrew Westergren, Wendy Tsu, Teresa Briggs, Pradeep Jotwani.

And thanks to the continued support of Tom Byers, Tina Seelig, Kathy Eisenhardt, Ritta Katilla, Bob Sutton and Chuck Eesly at Stanford Technology Ventures Program (the entrepreneurship center in the Stanford Engineering School).

Hacking for Defense @ Stanford 2021 Lessons Learned Presentations

We just finished our 6th annual Hacking for Defense class at Stanford.

What a year. With the pandemic winding down it finally feels like the beginning of the end.

This was my sixth time teaching a virtual class during the lockdown – and for our students likely their 15th or more. Hacking for Defense has teams of students working to understand and solve national security problems. Although the class was run completely online, and even though they were suffering from Zoom fatigue, the 10 teams of 42 students collectively interviewed 1,142 beneficiaries, stakeholders, requirements writers, program managers, industry partners, etc. – while simultaneously building a series of minimal viable products.

At the end of the quarter, each of the teams gave a final “Lessons Learned” presentation. Unlike traditional demo days or Shark Tanks which are, “Here’s how smart I am, and isn’t this a great product, please give me money,” a Lessons Learned presentation tells the story of a team’s 10-week journey and hard-won learning and discovery. For all of them it’s a roller coaster narrative describing what happens when you discover that everything you thought you knew on day one was wrong and how they eventually got it right.

Here’s how they did it and what they delivered.


How Do You Get Out of the Building When You Can’t Get Out of the Building?
This class is built on conducting in-person of interviews with customers/ beneficiaries and stakeholders, but due to the pandemic, teams now had to do all their customer discovery via a computer screen. This would seem to be a fatal stake through the heart of the class. How would customer interviews work via video?  After teaching remotely for the last year, we’ve learned that customer discovery is actually more efficient using video conferencing. It increased the number of interviews the students were able to do each week.

Many of the people the students needed to talk to were sheltering at home, which meant they weren’t surrounded by gatekeepers. While the students missed gaining the context of standing on a navy ship or visiting a drone control station or watching someone try their app or hardware, the teaching team’s assessment was that remote interviews were more than an adequate substitute. When Covid restrictions are over, we plan to add remote customer discovery to the students’ toolkit. (See here for an extended discussion of remote customer discovery.)

We Changed The Class Format
While teaching remotely we made two major changes to the class. Previously, each of the teams presented a weekly ten-minute summary consisting of “here’s what we thought, here’s what we did, here’s what we found, here’s what we’re going to do next week.”  While we kept that cadence, it was too exhausting for all the other teams to stare at their screen watching every other team present. So we split the weekly student presentations into thirds – three teams presented to the entire class then three teams each went into two Zoom breakout rooms. During the quarter we rotated the teams and instructors through the main room and breakout sessions.

The second change was the addition of alumni guest speakers – students who had taken the class in the past. They offered insights about what they got right and wrong and what they wished they had known.

Lessons Learned Presentation Format
For the final Lessons Learned presentation many of the eight teams presented a 2-minute video to provide context about their problem. This was followed by an 8-minute slide presentation describing their customer discovery journey over the 10 weeks. While all the teams used the Mission Model Canvas, (videos here), Customer Development and Agile Engineering to build Minimal Viable Products, each of their journeys was unique.

By the end the class all the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

All the presentations are worth a watch.

Team Fleetwise – Vehicle Fleet Management

If you can’t see the Fleetwise 2-minute video, click here

If you can’t see the Fleetwise slides, click here

Mission-Driven Entrepreneurship
This class is part of a bigger idea – Mission-Driven Entrepreneurship. Instead of students or faculty coming in with their own ideas, we ask them to work on societal problems, whether they’re problems for the State Department or the Department of Defense or non-profits/NGOs  or the Oceans and Climate or for anything the students are passionate about. The trick is we use the same Lean LaunchPad / I-Corps curriculum — and the same class structure – experiential, hands-on– driven this time by a mission-model not a business model. (The National Science Foundation, National Security Agency and the Common Mission Project have helped promote the expansion of the methodology worldwide.)

Mission-driven entrepreneurship is the answer to students who say, “I want to give back. I want to make my community, country or world a better place, while being challenged to solve some of the toughest problems.”

Project Agrippa – Logistics and Sustainment in IndoPacific

If you can’t see the Project Agrippa 2-minute video, click here

If you can’t see the Project Agrippa slides, click here

It Started With An Idea
Hacking for Defense has its origins in the Lean LaunchPad class I first taught at Stanford in 2011. I observed that teaching case studies and/or how to write a business plan as a capstone entrepreneurship class didn’t match the hands-on chaos of a startup. Furthermore, there was no entrepreneurship class that combined experiential learning with the Lean methodology. Our goal was to teach both theory and practice.

The same year we started the class, it was adopted by the National Science Foundation to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 9 regional locations supporting 98 universities and has trained over 1500 science teams. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

Team Silknet – Detecting Ground Base Threats

If you can’t see the Silknet 2-minute video, click here

If you can’t see the Silknet slides, click here

Origins Of Hacking For Defense
In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford, we observed that students in our research universities had little connection to the problems their government was trying to solve or the larger issues civil society was grappling with. As we thought about how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. That year we launched both Hacking for Defense and Hacking for Diplomacy (with Professor Jeremy Weinstein and the State Department) at Stanford.

Team Flexible Fingerprints – Improve Cybersecurity

If you can’t see the Flexible Fingerprints 2-minute video, click here

If you can’t see the Flexible Fingerprints slides, click here

Goals for the Hacking for Defense Class
Our primary goal was to teach students Lean Innovation while they engaged in national public service. Today if college students want to give back to their country, they think of Teach for America, the Peace Corps, or AmeriCorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence community or other government agencies.

In the class we saw that students could learn about the nation’s threats and security challenges while working with innovators inside the DoD and Intelligence Community. At the same time the experience would introduce to the sponsors, who are innovators inside the Department of Defense (DOD) and Intelligence Community (IC), a methodology that could help them understand and better respond to rapidly evolving threats. We wanted to show that if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, defense acquisition programs could operate at speed and urgency and deliver timely and needed solutions.

Finally, we wanted to familiarize students with the military as a profession and help the better understand its expertise, and its proper role in society. We hoped it would also show our sponsors in the Department of Defense and Intelligence community that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions to real-world problems.

Team Neurosmart –  Optimizing Performance of Special Operators

If you can’t see the Neurosmart 2-minute video, click here

If you can’t see the Neurosmart slides, click here

Mission-Driven in 50 Universities and Continuing to Expand in Scope and Reach
What started as a class is now a movement.

From its beginning with our Stanford class, Hacking for Defense is now offered in over 50 universities in the U.S., as well as in the UK and Australia. Steve Weinstein started Hacking for Impact (Non-Profits) and Hacking for Local (Oakland) at U.C. Berkeley, and Hacking for Oceans at both Scripps and UC Santa Cruz.  Hacking for Homeland Security launched last year at the Colorado School of Mines and Carnegie Mellon University. A version for NASA is coming up next.

And to help businesses recover from the pandemic, the teaching team taught a series of Hacking For Recovery classes last summer.

Our Hacking for Defense team continues to look for opportunities to adapt and apply the course methodology for broader impact and public good. Project Agrippa, for example, piloted a new “Hacking for Strategy” initiative inspired by their experience in Stanford’s “Technology, Innovation and Modern War” class that Raj Shah, Joe Felter and I taught last fall. This all-star team of 4 undergraduates and a JD/MBA developed new ways to provide logistical support to maritime forces in the Indo-Pacific region. Their recommendations drew on insights gleaned from over 242! interviews (a national H4D class record.) After in-person briefings to Marine Corps and Navy commanders and staff across major commands from California to Hawaii, they received interest in establishing a future collaboration, validating our hypothesis that Hacking for Strategy would be a welcome addition to our course offerings. Its premise is that keeping America safe not only requires us maintaining a technological edge but also using these cutting edge technologies to develop new operational concepts and strategies. Stay tuned.

Team AngelComms – Rescuing Downed Pilots

If you can’t see the AngelComms 2-minute video, click here

If you can’t see the AngelComms slides, click here

Team Salus – Patching Operational Systems to Keep them Secure

If you can’t see the Salus 2-minute video, click here

If you can’t see the Salus slides, click here

Team Mongoose – Tracking Hackers Disposable Infrastructure

If you can’t see the Mongoose 2-minute video, click here

If you can’t see the Mongoose slides, click here

Team Engage – Preparing Aviators to Make Critical High Stakes Decisions

If you can’t see the Engage slides, click here

What’s Next For These Teams?
When they graduate, the Stanford students on these teams have the pick of jobs in startups, companies, and consulting firms. Most are applying to H4X Labs, an accelerator focused on building dual-use companies that sell to both the government and commercial firms. Many will continue to work with their problem sponsor. Several will join the new Stanford Gordian Knot Center which is focused on the intersection of policy, operational concepts, and technology.

In our post class survey 86% of the students said that the class had impact on their immediate next steps in their career. Over 75% said it changed their opinion of working with the Department of Defense and other USG organizations.

It Takes A Village
While I authored this blog post, this class is a team project. The secret sauce of the success of Hacking for Defense at Stanford is the extraordinary group of dedicated volunteers supporting our students in so many critical ways.

The teaching team consisted of myself and:

  • Pete Newell, retired Army Colonel and ex Director of the Army’s Rapid Equipping Force, now CEO of BMNT.
  • Joe Felter, retired Army Colonel; and former deputy assistant secretary of defense for South Asia, Southeast Asia, and Oceania; and William J. Perry Fellow at Stanford’s Center for International Security and Cooperation.
  • Steve Weinstein, 30-year veteran of Silicon Valley technology companies and Hollywood media companies. Steve was CEO of MovieLabs, the joint R&D lab of all the major motion picture studios. He runs H4X Labs.
  • Tom Bedecarré, the founder and CEO of AKQA, the leading digital advertising agency.
  • Jeff Decker, a Stanford researcher focusing on dual-use research. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Our teaching assistants this year were Nick Mirda, Sally Eagen, Joel Johnson, past graduates of Hacking for Defense, and Valeria RinconA special thanks to the National Security Innovation Network (NSIN) and Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country, as well as Lockheed Martin and Northrop Grumman. And our course advisor, Tom Byers, Professor of Engineering and Faculty Director, STVP.

Thanks to Mike Brown, Director of the Defense Innovation Unit for giving an extraordinary closing keynote.

We were lucky to get a team of mentors (VCs and entrepreneurs) as well as an extraordinary force of military liaisons from the Hoover Institution’s National Security Affairs Fellows program, Stanford senior military fellowship program and other accomplished military affiliated volunteers. This diverse group of experienced experts selflessly volunteer their time to help coach the teams. Thanks to Todd Basche, Rafi Holtzman, Kevin Ray, Craig Seidel, Katie Tobin, Jennifer Quarrie, Jason Chen, Matt Fante, Richard Tippitt, Rich Lawson, Commander Jack Sounders, Mike Hoeschele, Donnie Hasseltine, Steve Skipper, LTC Jim Wiese, Col. Denny Davis, Commander Jeff Vanak, Marco Romani, Rachel Costello, LtCol Kenny Del Mazo, Don Peppers, Mark Wilson and LTC Ed Cuevas

And of course a big shout-out to our problem sponsors across the DoD and IC: MSgt Ashley McCarthy, Jason Stack, Col Sean Heidgerken, LTC Richard Barnes, George Huber, Neal Ziring, Shane Williams, Anthony Ries, Russell Hoffing, Javier Garcia, Matt Correa, Shawn Walsh, and Claudia Quigley.

You Don’t Need Permission

I was pleasantly surprised to hear from Suresh, an ex-student I’ve known for a long time. A U.S. citizen he was now the head of sales and marketing for a company in London selling medical devices to hospitals in the UK National Health Service.  His boss had identified the U.S. as their next market and wanted him to set up a U.S. salesforce. Suresh understood that the U.S. health system was very different from the system in the UK, not just the regulatory regime through the FDA, but the reimbursement process and the entire sales process.

Over a Zoom call Suresh explained, “I’m trying to push the importance of running customer discovery and testing these hypotheses before we build our U.S. product, but I’m running into a pushback from my CEO. He says, “We’re disruptors! Discovery is going to slow us down.  We need to move quickly!”

Suresh was concerned. “If we don’t test our assumptions about the market and any changes needed to our products, we’ll build something I can’t sell. Worse, given how expensive clinical trials are in medtech, I’m concerned if we build a product that isn’t commercially viable, we’ll be out of business before we even start.”

I could hear his frustration and concern when he asked, “How can I convince my boss to use customer discovery to test our hypotheses?”

That’s when I realized that Suresh was overlooking a few things.

  1. He was trying to sell the “story” of Customer Discovery as part of the Lean Methodology to his CEO by explaining how discovery worked with the business model canvas, agile engineering, pivots and MVPs.
  2. But talking about the method to others is not the same as “doing” Customer Discovery.
  3. Customer Discovery is about gathering validated evidence, not proselytizing a method.
  4. The goal of discovery is to gather evidence to test hypotheses, deeply understand the customer problem and validate a solution.
  5. As head of sales and marketing, Suresh didn’t need his CEO to buy into the process or give his permission to start the discovery process. He was in charge.
  6. Given the ubiquity of Zoom, he could use it to rapidly get out of the building to the U.S. to test some hypotheses and gather some initial insights.

I pointed out that once he had potential customer, regulator, and reimbursement data from his Discovery interviews, he could bring that data into conversations with his CEO.

Having real data turns conversations from faith-based to evidence-based.

Lessons Learned

  • Talking about the Lean method to others is not the same as doing it
  • If you’re in charge or part of a customer-facing organization, you don’t need to wait for permission to talk to customers to test hypotheses
  • Having real data turns conversations from faith-based to evidence-based

Your Product is Not Their Problem

There are no facts inside your building, so get the heck outside

I just had a call with Lorenz, a former business school student who started a job at a biotech startup making bacteria to take CO2 out of the air. His job was to find new commercial markets for this bacteria at scale.  And he wanted to chat about how to best enter a new market.

His market research found that the concrete industry contributes between 5 and 10% of the world’s carbon emissions. So it seemed logical to him that the concrete industry was going to be one the first places to approach since it was obvious that they need to reduce carbon emissions. He believed that if used as an additive to concrete, his bacteria could strengthen it while reducing CO2.

The conversation got interesting when I asked, “How are you going to describe the product to potential customers in the concrete industry?” Lorenz began a long description of the details of the bacteria, his founders’ research papers on bacteria, the scientific advisory board bacteria experts they had assembled, how the bacteria was made at scale in fluidized bed reactors, etc… This went on for at least ten more minutes. When he was done I asked him, “So why should anybody in the concrete industry care? Do you really think they’re looking for bacteria made in fluidized bed reactors? Do you think there are a significant number whose number one issue is to buy bacteria? Do you know what if any of the features you mentioned actually matter to a potential customer?” There was silence for a moment. And then he said, “I don’t know.”

I wasn’t completely surprised because as a young marketeer, I made this mistake all the time – thinking that my product was a solution to someone’s problem  – without ever understanding what problems the customers really had. And that I needed to have all the answers when in fact I didn’t even understand the questions.

I suggested that perhaps he should get out of the building and actually talk to some large-scale concrete suppliers and rather than starting with what he wanted to sell them, try to understand what their needs were. For example, how were current and upcoming green building regulations on CO2 emissions affecting the concrete industry? How are they solving that problem today? (If they weren’t solving it, it may not be a problem they’ll pay to solve.) What was the current cost of low carbon concrete? How much would they have to charge to be competitive? Were there specific use-cases that made sense for initial adoption/pilots? What additional benefits could bacteria as an concrete additive make (ie. greater strength, crack healing)?

We talked for a few minutes more and by then I could see the lightbulb going on over his head when he said, “I think I got my work cut out for me.”

Lessons learned

  • Your product is not someone’s problem
  • Start with a deep understanding of a customer problem or need before you start pitching your solution
  • Ask customers how they solve the problem today
  • Understand future regulations that might change your customer’s priorities or challenges

Back to the Classroom – The Educators Summit

Register Here

In 2020 over 1,000 educators joined us online to learn and share how to teach during the pandemic. Now we’re heading back into the classroom and the world has changed. What is the “new normal” for Lean Education? Will using video to “get out of the building” still play a role? What roles do diversity, equity and inclusion play in future syllabi?

Join me, our guest speaker Dr. Laura Tyson, our panelists Brandy Nagel, Ivy Shultz, Michael Camp and Jim Hornthal and hosts Jerry Engel, Pete Newell, Steve Weinstein and your peers for a robust discussion to these questions at the 4th edition of Lean Innovation Educators Summit on June 3rd, 1 – 4pm EDT, 10am – 1pm PDT.


Why
The Pandemic has changed everything. After a year plus in the virtual teaching environment how much will stick and what will return to pre-pandemic norms? How much of our virtual  “getting out of the classroom” teaching methods will remain? The business landscape crippled some industries while others have exploded. Which ones will rebound? The funding environment for our students continues to be on fire. Will that continue? How will the push for diversity, equity and inclusion affect educators?

What
The event will begin with a fireside chat with Dr. Laura Tyson former Dean of the Berkeley Haas School of Business and former Chairman of the National Economic Council. The respondent panel will include, Brandy Nagel (Georgia Tech), and Ivy Shultz (Columbia University), Michael Camp (Ohio State), Jim Hornthal (UC Berkeley). To learn from each other and share ideas, we’ll then go into breakout sessions so you can discuss the topics from the fireside chat and share best practices with your peer Lean educators from around the world.

How
This session is free to all but limited to Innovation educators. You can register for the event here and/or learn more on our website. We look forward to gathering as a community of educators to shape the future of Lean Innovation Education.

When
See you on June 3rd 1pm – 4pm EDT, 10am-1pm PDT.

Bring your best ideas!

Register here

These Five Principles Will Accelerate Innovation

As Director of the U.S. Army’s Rapid Equipping Force Pete Newell delivered innovation at speed and scale in the Department of Defense. Pete is now CEO of BMNT, a company that delivers innovation solutions and processes for governments.

Here are Pete’s 5 principles that will accelerate innovation.


To help a large Defense organization wrestle with how to increase the velocity of innovation in their ranks Steve Blank and I spent the better part of last week with our heads together reviewing everything we learned in the five years since we merged the concepts of problem curation and Lean while launching the innovation pipeline.

The original Innovation Pipeline sketch – 2016

I spent yesterday sifting through the most recent lessons learned and results from a series of accelerators BMNT is running for the intelligence community. Then last night I watched the final presentations from the inaugural Hacking for National Security course in Australia before jumping over to teach Stanford’s Hacking for Defense® (H4D) class.

Looking back on the week I’m blown away by how far we’ve come since we merged the two methodologies five years ago and by how fast we are discovering the pathways toward solving incredibly hard problems. Some examples:

  • In less than six weeks a Stanford H4D team has redefined a problem related to security vetting and radicalization while also describing the pathway a solution could follow to deployment within the Department of Defense and the Intelligence Community.
  • A Navy team recently sourced 80 problems, then curated down to one priority problem to solve. In less than 60 days they created 26 MVPs while interviewing over two dozen companies. They then incubated and delivered a solution that will help get large vessels back to sea faster, potentially saving the Navy $20M-$30M a year.
  • In just three weeks, the DIA MARS team sourced 100 problems from nearly 400 people, then curated and selected five priority problems to focus on. In eight weeks, five teams conducted more than 125 interviews (across 53 stakeholder organizations) and 12 experiments to deliver five validated proofs-of-concept and the evidence needed to confidently invest resources to prototype three of the five based on their user desirability, technical feasibility and organizational viability.

What I observed from the week’s deep dive: Whether it is an agency cross-functional team or a university-based “Hacking for” team we are accelerating, five key concepts drive the foundation for this increasing pace of learning and solution delivery. They are

  1. The power of Lean Methodology is supercharged when discovery begins with a well-curated and prioritized problem. Getting to one well-curated problem requires access to a source of hundreds of them.   
  2. Problem curation doesn’t stop until discovery is complete — the process of trying to discover the solution to a problem helps define the actual problem (and for .gov folks the actual requirement for the future solution).
  3. Stakeholder mapping and nailing the value propositions for beneficiaries, buyers, supporters, advocates and potential saboteurs are critical to building a pathway through the phases of the innovation pipeline and transitioning a solution to deployment. 
  4. The key to understanding value propositions is in building interviews that are based on a set of hypotheses (about the problem, the stakeholder and potential solutions to be explored) and data to be captured while using minimum viable products (just enough “product” to increase the efficacy of a conversation and increase the speed of learning).
  5. Innovation happens because of people and it takes a village. Whether academic-based like our Hacking for Defense teams or internal organizational Integrated Product or Cross-Functional Teams (IPTs/CFTs), accelerator teams perform best when supported by:
    1. Teachers – who can ground teams in a common framework and language for the discipline of innovation and entrepreneurship.
    2. Coaches – who can walk teams through the practical application of innovation tools in context with the problem they are trying to solve.
    3. Mentors – who will provide relentlessly direct feedback to teams and challenge them on the quality of the hypotheses, MVPs and the analysis of what they’ve learned, while driving them through each pivot rather than letting them get bogged down in “analysis paralysis.”
    4. Advisors – who will provide alternative viewpoints that will enable teams to see clearer pathways through bureaucracies.
    5. Connectors – who will help teams rapidly grow their networks to gain new insights from unique partners not yet discovered.

If your organization is running innovation activities or an accelerator and these five principles aren’t part of their program they are likely wasting your organization’s time and resources, and contributing to Innovation Theater instead of deploying solutions to real problems.

—–

Next up we’ll dig into how the innovation pipeline serves a parallel process for managing innovation

Enterprise Innovation for the 21st Century

Today’s environment requires separate systems for innovation and execution that operate with parallel and sometimes overlapping processes


Why Defense Could Now Be a Market for Startups

The U.S. Department of Defense is coming to grips with the idea that the technologies it needs to keep the country safe and secure are no longer exclusively owned by the military or its prime contractors. AI, machine learning, autonomy, cyber, quantum, access to space, semiconductors, biotech are all being driven by commercial companies.  At the front-end of these innovations are startups – organizations the Department of Defense hasn’t previously dealt with at scale.

They’re now learning how.

Mrinal Menon is one of my Hacking for Defense students at Stanford where he’s currently in the MBA program. He’s a former U.S. Navy Surface Warfare Officer. Jeff Decker is a co-instructor of the Hacking for Defense class and the Stanford program director. Jeff is an Army Second Ranger Battalion veteran.

Mrinal and Jeff’s article below explains how startups can adapt and thrive while working with the Defense Department. And how the Department of Defense is learning to work with startups.



This article previously appeared in Fast Company.

At a time when young companies struggle to find technology sectors not dominated by Silicon Valley’s giants, most startups remain oblivious to one of the largest markets in the world, the U.S. Defense Department. The military awarded $445 billion in contracts in 2020. By comparison, last year’s global market for software-as-a-service, one of the hottest sectors for startup creation and investment, was estimated at $104 billion.

There’s a willing market here, too. The Pentagon is eager for help from the nation’s innovators. The military is clamoring for cutting-edge technologies in areas like artificial intelligence, machine learning, and autonomy. To attract interest the Defense Department is handing out unprecedented numbers of small contracts and in 2020 seeded 1,635 firms with more than $1.5 billion in early funding. Dozens of outreach programs across the military now offer quick revenue to early-stage companies. A startup could land a contract worth up to $3 million within months of entering the defense market.

This emerging opportunity reflects the urgency of keeping pace with rivals like China and Russia, who are furiously integratingcommercial technologies like AI, quantum computing, and unmanned systems into their armed forces. If former Google CEO Eric Schmidt’s prediction about China overtaking the United States in AI comes to pass, the People’s Liberation Army could be better prepared than the U.S. military for future conflicts in which cyberattacks and drone swarms play a larger role than warships and fighter jets. Without adopting cutting-edge commercial technologies, the U.S. military may be rendered obsolete.

Yet, for every newly minted success like Palantir or Anduril, thousands of companies struggle to find follow-on opportunities after receiving early innovation contracts. Historically, more than 80% of new entrants exit the defense market before they have a chance to see recurring revenue. Anduril founder Palmer Luckey has sardonically noted that the only defense companies to reach unicorn status in the past 30 years have all been founded by billionaires able to endure long gaps in funding. To truly compete with technologically advanced rivals, the Defense Department will ultimately need to enable a broad pool of innovative founders to succeed. Until this ecosystem materializes, however, building a successful defense business is likely to take years of grueling effort. Before committing to work with the military, startups must look past the allure of early money and carefully assess whether the defense market is right for them.

The Key Questions For Startup
Despite the Pentagon’s newfound enthusiasm for innovation, startups in the defense market continue to encounter longstanding obstacles. Most companies face uncertainty when trying to bridge prototyping awards with more permanent follow-on contracts. Startups that successfully fulfill innovation contracts don’t automatically scale. Instead, their solutions must still compete for funding in a formal budgeting and acquisition process before being eligible for widespread adoption . This ordeal takes two years or more, during which startups should anticipate little or no revenue as they try to push their product through the bureaucracy.

While earning the first $1 million in the defense market can be relatively easy, landing the next $10 million is extremely difficult. Before committing to work with the military, startups should carefully consider whether the defense market is right for them by asking three questions.

Is there strong interest for the product?
Those that offer solutions to high-priority problems have the best chance of sourcing funding and follow-on awards. Startups should assess how well their technologies align with the needs outlined in the defense budget, various modernization strategies, and problems posed by defense innovation hubs. For example, “flying taxi” startup Joby Aviation aligned its electric vertical takeoff and landing aircraft with the Air Force’s stated interest in advancing the market for air mobility vehicles. The 3D mapping company Hivemapper found traction with the Army by targeting its need to autonomously resupply field artillery units.

Is the company prepared to endure revenue gaps?
As noted, companies may go years without defense revenue while trying to bridge early contracts with larger opportunities. During this period, startups will need to sustain business development activities and may incur additional regulatory and compliance costs to ensure their product is usable by the military. Investors are also likely to express impatience with the long timelines of defense sales. The best prepared companies can either raise sufficient funds to keep operating or count on revenue from other sources. Palmer Luckey’s Anduril has aggressively pursued financingto sustain growth. Recently public enterprise-AI firm C3.ai entered the defense market having already built a successful commercial business in the energy vertical. Similarly, 3D-printing homebuilder Icon’s technologies have applications that extend well beyond national security use cases to commercial housing markets.

Can I balance the needs of my defense and commercial customers?
The best-positioned startups solve fundamentally similar problems for both military and commercial users. Satellite imagery firm Orbital Insight’s Go platform, for example, automates imagery analysis for defense analysts, investors, and agribusiness alike. Some amount of customization is inevitable given that military users often have specific mission requirements or technical constraints. However, companies that can serve military customers without fundamentally altering their existing product roadmap are likely to be most successful.

Incidents like Google’s withdrawal from Project Maven have highlighted the potential for conflict when commercial companies decide to work with the military. This tension tends to be lower at startups where an early team willingly signs on to pursue defense business. Regardless, companies should be mindful of the reputational impacts working with the DoD could have on their existing business.

The Case for Optimism
Companies eyeing the defense market have reason to be optimistic. There are a growing number of efforts to remove obstacles to partnerships between high-tech startups and the military. For example, the Air Force last year created the Strategic Financing (STRATFI) program. STRATFI provides the Air Force’s most promising commercial innovators, a list which includes Icon and Orbital Insight, quick access to large contracts worth up to $15M. Another positive sign is President Biden’s nomination of former Symantec CEO Mike Brown to oversee the Pentagon’s purchases of new technologies. As director of the Defense Innovation Unit (DIU), Brown has spearheaded investments in commercial technologies and helped C3.ai, Joby Aviation, and other Silicon Valley standouts navigate funding gaps. If confirmed, Brown could accelerate much needed reforms to military acquisition processes.

A startup that makes it through the hurdles may find a massive market with a stable set of customers eager for better technology solutions. The U.S. government pays its bills on time, defense budgets are shielded from short-term volatility, and contracts may be guaranteed for multiple years. Once onboarded, military users can quickly become long-term customers given the challenges involved with switching to another product.

A Path to the Minimum Viable Product

I first met Shawn Carolan and his wife Jennifer at the turn of the century at 11,000 feet. I was hiking with my kids between the Yosemite High Sierra camps. Having just retired from a career as an entrepreneur I had started thinking about why startups were different from large companies. The ideas were bouncing around my head so hard that I shared them with these strangers around a campfire, drawing out the four steps with a stick in the dirt. Shawn immediately said the name I had given the four steps was confusing – I had called it market development – he suggested that I call it Customer Development – and the name stuck. What I didn’t realize was that both were graduate students at Stanford and later both would become great VCs – Shawn at Menlo Ventures and Jennifer at Reach Capital. (And Jennifer is now my co-instructor in the Stanford Lean LaunchPad class.)

The MVP Tree
Over the last two decades Shawn has seen hundreds of startups use the Lean Methodology. Many of them get hung up on understanding how to select the right minimal viable product. He came up with the concept to simply the search for product/market fit by using an MVP Tree.

Shawns guest blog post describing describing the MVP Tree is below.

(Note that if you’re familiar with the business model canvas, Steps 1-4 below are equivalent to a visual map of the choices a founder makes as they develop a business model canvas. Step 5 and 6 leads you to selecting the right MVPs.)


It’s commonly believed that the top two reasons startups fail is because “there’s no market need” and “they ran out of cash.”  These reasons are mental gymnastics to avoid a plain truth: startups fail when they don’t build a simple solution to a problem many people have.

startup_fails.pngMany startups fall into the trap of building toward a “mission” rather than a minimum viable product (MVP).

Your mission is your baby. It’s the North Star that got your people on board and inspires them daily. However, solely focusing on your mission is the same as being unfocused.

Bridging the gap between a big, ambitious dream and the reality of what you can accomplish with limited resources isn’t fun because it requires saying no (for now) to a lot of things that have you excited. Paradoxically, resource limitations are the secret to success; they teach lessons that experienced entrepreneurs have harnessed. Constraints force you to pause—or even permanently shelve—certain aspects of your mission in favor of proving that you can deliver one specific thing that really matters to customers.

During the crucial mission-to-MVP planning phase, the objective of a startup is to solve one job for one customer group such that customers consistently use your minimally viable product for an important part of their work or personal lives. In other words, you prove retention. It’s really all that matters at the earliest stage.

The tool for doing this efficiently and effectively? We call it an MVP tree.

Solution: Build an MVP Tree
Company missions tend to fall to the extremes: either the mission isn’t ambitious enough or it’s too ambitious to build with your current resources.

It bears repeating: an early-stage startup must focus on making one customer group excited by a mission-aligned product. Doing so is usually a long way away from realizing your full mission statement, and that’s okay.

An MVP tree is a way of methodically breaking your mission into smaller components and formulating MVP candidates that may get your company sustainable and scalable. Using the tree structure outlined below increases the chance that your first step forward (the MVP) will be successful with a small team, while taking you in the right direction to achieve your company’s big mission. An MVP tree has three main components—customer archetypes, jobs to be done, and execution—and we’ll walk you through them step by step with a case study of Roku, a former portfolio company we’ve been fortunate to see execute from a $20 million first VC round to what is now a $50 billion public company.

(To get started, we draw our tree out using Whimsical, a product that I’m a big fan of.)

Step 1: Define Your Big Mission in a Simple Statement

In the day-to-day fray of prioritizing features, considering customer input, and handling people issues, it’s easy for a team to lose their bearings. The world is full of great products and it’s essential to be crystal clear on your reason for being; to avoid wandering in circles, you need a mission statement. A few examples:

  • Roku: “To be the TV streaming platform that connects the entire TV ecosystem around the world”

  • Uber: “To bring transportation—for everyone, everywhere”

  • Chime: “We believe everyone deserves financial peace of mind”

Your mission statement needs to stand for something specific and impactful. It may change as you build and learn from your customers, but aim for it to conjure up an image of a better future with your product at the center.

Your mission statement is the center of your MVP Tree.

Screen Shot 2021-03-17 at 11.21.46 AM.png

A motivating mission statement inspires action. It may be tempting to jump right into coding, but slow down—remember, the goal of this exercise is to determine how little you can do to acquire and retain customers. With both growth and retention, you earn the right to build more.

Step 2: Customer Archetypes

A customer archetype is a category of similar people with similar needs (e.g., segmented by age, gender, profession, personality type, etc.). While it may be tempting to want to build a product for everyone, everywhere (7 billion TAM, right?!), doing so will distract you from building exactly what one group needs. Each new segment you attempt to serve can increase scope, adding to your workload and demanding more of your limited resources.

The intent of this branch of the tree is to identify one group of potential customers who would be highly motivated to fix their pain as part of your MVP. Draw out a few customer archetypes that might be a fit for your startup, (we’ll discuss later how to prioritize the one group to tackle first.)

Screen Shot 2021-03-17 at 11.22.16 AM.png

Step 3: Jobs to Be Done

Clayton Christensen’s Jobs to Be Done framework proposes that focusing solely on customer data leads founders on a wild goose chase. Founders should understand what the customer hopes to accomplish, or what their job to be done is.

Break down your mission statement into different “jobs”; this itemization will likely narrow your product scope considerably, while still allowing you to create something of significant value.

When consumers have a job to be done, they’ll look around at their choices and select the best tool for the job. Your goal is to build the best tool for meaningful, reoccurring jobs that people face. Map out the jobs that you believe exist for the customer archetypes identified in step 2.

The more common jobs may apply to several of the archetypes, while the more esoteric jobs may apply to just one. There will be plenty of overlap, which is why jobs have their own branch of the MVP tree rather than attaching directly to the archetype branches.

The more common jobs may apply to several of the archetypes, while the more esoteric jobs may apply to just one. There will be plenty of overlap, which is why jobs have their own branch of the MVP tree rather than attaching directly to the archetype branches.

Step 4: Execution Branches 

Execution branches will vary based on the company, but think of them as the components of what gets built and where it gets sold. These branches of the MVP tree comprise the components of market expansion that urge founders to explore the tactical side of getting a solid product in front of the right people. This first product launch can either be a costly mistake or an ingenious first step that shows traction with early customers and gives your team and investors confidence. Map these out now as part of the tree and reduce the odds of a headache down the road.

In the Roku case study we’ve chosen three execution branches: (1) delivery platforms, (2) sales channels, and (3) chip platforms.

Delivery platforms
Delivery platforms are the vehicles through which customers come to interact with your product. For software products, they would be the operating systems with big market shares: iOS, Android, web, Mac, and PC. While supporting each additional platform expands your addressable market or breadth of customer touchpoints, it can dramatically increase scope. Developing for multiple platforms at once spreads already-thin resources, which ultimately harms the creation of the best product possible for a specific customer segment.

The fewer platforms you choose to support, the smaller the scope. Pick one delivery platform to save resources and prove your value. Investors will recognize that a successful app on iOS will also work on Android with more capital. Focus your precious time on making one platform sing.

Clubhouse, a recent startup darling, has quietly grown to over 2M users exclusively on Apple. Some early users might be frustrated because they can’t invite their Android peers, but the strategy to focus on iOS helped Team Clubhouse minimize their initial scope and meticulously learn from early users without the distraction that may come from opening the floodgates.

For hardware companies, delivery platforms are essentially the potential SKUs you might consider shipping. Steve Jobs once said that if you are really serious about software, you should build your own hardware. You can think of these hardware form factors as software delivery vehicles. Most people only know Roku for their TV devices, but Roku initially shipped an audio device for Internet radio stations and a PhotoBridge product to get digital photo libraries to the TV. Even when moving to video, consumers had the choice between a stand-alone box or a smaller plug-in stick. Today, Roku has become an operating system embedded in other brands’ TVs.

Sales channels
Sales channels are the paths through which your product lands in customers’ hands. For software products, the channels are typically through the mobile app stores or directly over the Internet. For hardware products, it’s D2C e-commerce, online retailers, or physical retail.

Some sales channels may behave similarly; more often than not, they each pose unique challenges. Every additional sales channel costs resources and increases scope. Pick one channel, prove traction, then experiment with the next.

Roku_MVP4.png

Chip platforms
Unique to hardware companies is the choice of which chip to use, and it’s a big one. The choice of semiconductor has profound implications on system requirements like how much memory is needed, how much power is required—and ultimately, the end system cost. Owners of the new Mac M1 laptops are taking advantage of a decade of Apple’s mobile chip development, which is finally robust enough to run the MacOS. The Roku OS has come to run on several different chipsets over time, but in the beginning they had to choose one.

Roku_MVP5.png

Other branches
There are several other execution branches that may be relevant to your business. If you are an office productivity tool, the data ecosystem you pick is a big one: Google or Microsoft? It’s lazy thinking (and expensive engineering) to try to build for both at the same time. Pick one, show success, then raise more money to address the other half of the market. Again, the fewer of these branches (ecosystems, etc.) you choose to support, the smaller the scope. If your startup doesn’t need to access customer data, the choice of consumer-grade vs. enterprise-grade is a big one that adds scope. The theme remains the same: be selective and pick the branch you have the best shot at success with before adding more.

Roku_MVP6.png

Step 5: Scope Out Your Candidate MVPs

If your map looks like Roku’s, you’re probably now staring at something that has grown to become quite complex, with many “leaves” at the ends of your branches. A single leaf chosen on each branch is what makes up an MVP. The reasonable permutations are your candidate MVPs.

Remember, an MVP is the minimally scoped product that gets some job done for your chosen customer archetype.

Step 6: Evaluate Your Candidate MVPs

How do you know which MVP to build first? Choose wisely: This will be the next several months of your life.
A successful MVP satisfies three criteria:

  1. It addresses a meaningful job to be done. A customer spending their own time or money to do a job chooses your solution as the best option for them. Pick a meaningful job—the more frequently occurring the better—and offer a significant advantage (better, faster, or cheaper).

  2. It has a growth engine. Build or price growth into your product. There are two viable growth engines for tech companies:

    1. Viral, or “inherently viral” growth: Customers either intentionally or unintentionally recruit new customers by using your product. Social platforms use intentional virality; this occurs when users get a more fulfilling experience as more of  their friends join the same platform. Unintentional virality happens when customers inadvertently introduce others to the product experience, similar to how shared bike or scooter riders serve as mobile billboards for the experience.

    2. Economic paid acquisition: Contribution margins from customers are recycled into advertising, marketing, and other PR activities that successfully drive additional customers. Note well, though: this engine is “economic” because it must fuel itself. It’s easy to simply buy customers, but only real value makes them stay. Your product must collect far more value over the lifetime of the customer relationship than the cost of acquiring that customer in the first place.

  3. It has a rapid time-to-value: How long must customers wait for the “aha” moment? With my first Uber ride in late 2011, it took about two minutes for that moment to arrive: I installed the app, entered my credit card, ordered a car, and it was waiting for me by the time I walked down one flight of stairs. Aha! I knew I’d never wait for another taxi in San Francisco again. The faster and more simply your product can prove its worth, the higher rate of conversion from tire-kickers to retained customers. For software startups, ask yourselves this question: What’s the one screen that will make your customers get it? 

Step 7: Pick, Beta, Ship

Now’s the time to let the rubber meet the road and get a minimal product into a customer’s hands. Can it do the job better than their prior solution? Keep iterating until you’re getting that one specific job done. 

Roku’s first two MVPs weren’t a success (sound bridge + photo bridge). It was however, through the process of mapping out MVPs candidates, testing, and learning that brought intense clarity to and laid out the infrastructure for what would ultimately work — a delivery platform for Netflix. Even if the first MVP isn’t a hit, you’ll be building the muscle needed for the company such as making key hires.

Don’t pivot to a different job unless you’ve learned something new that causes you to reconsider your initial hypothesis. This may take a lot of time, but that’s perfectly fine. Stay focused on solving that job until you prove your hypothesis right or wrong. The world is littered with failed companies that never got a product right. Where there’s a job to be done, you can build a solution with enough time, talent, and focus.

Roku’s Winning MVP:

Roku_MVP7.png

Step 8: Double-Down

When you finally find your product getting a recurring job done better than any other tool, stick with it. Don’t take your eye off the prize and move onto new things too quickly.

Earn the right to increase scope and move on to other jobs, platforms, and customer archetypes after solidifying your position among this first set of customers―and creating a sustainable growth engine.

Final Thoughts

Founders become infatuated with a bold and ambitious mission—as they should. However, what separates a startup that actually brings its mission to life from one that doesn’t is the ability to shed the rose-colored glasses and solve for a small job to be done.

A proper MVP framework, such as our MVP Tree presented here, is a critical first step in fulfilling your mission, even though it might seem like you are selling it short. Be patient. It won’t be easy realizing your mission, and it shouldn’t be. If your mission were easy, it would already be done by someone else!

Want to see an MVP Tree for another startup? Tweet @Shawnvc to nominate a company!


E Pluribus Unum – A Rallying Cry for National Service

This post previously appeared in Real Clear Defense.

 

The Latin phrase E Pluribus Unum – Out of Many, One – is our de facto national motto. It was a rallying cry of our founders as they built a single unified nation from a collection of states. It’s a good reminder of where we need to go.

Today as our country struggles to find the common threads that bind us, we need unifying, cohesive, collective, and shared national experiences to bring the country together again.

Here’s what we’ve done to get started.

And why I did it.

————-

Pete Newell, Joe Felter and I met over coffee in 2016 to discuss our common goal – how to get students in research universities who would never consider working on national security problems engaged in keeping the country safe and secure.

Today, our contribution to national service, Hacking for Defense, turned five years old. In this class, students learn about the nation’s emerging threats and security challenges while working with innovators inside the Department of Defense.

The result? The class teaches students entrepreneurship while they engage in what amounts to national public service. From our single class at Stanford, Hacking for Defense is now taught at 47 U.S. universities having graduated 500 teams and 2,000+ students.

Why Serve?
My interest in starting Hacking for Defense was rooted in my long belief in service – not just paying taxes or voting, but actual service. I had a great career as an entrepreneur, but always believed that at some point in your life you need to serve others – whether it’s God, country, community, or family. And I did so in my stints in the military and public service and as an educator.

Disconnection
Looking back it’s clear that our country was far more cohesive when millions of us had to physically share space and live and work with others who didn’t think like us or talk like us. The Air Force turned out to be the first melting pot I would encounter (Silicon Valley the next) where individuals from different classes and culture had the opportunity to share a common goal and move beyond the environment they grew up in. At each base I was stationed in, I hung out with a group that tutored each other, read books together, went on adventures together and learned together. And while most of us came from totally different backgrounds (before the Air Force, I never knew you put salt on watermelon, that Spam was food or muffuletta was a sandwich), as far as the military was concerned, we were all the same.

But a half-century ago, the country started to disconnect from each other and our government when we eliminated national service. In 1973, near the end of the Vietnam War, the U.S. ended compulsory military service and has since depended on an all-volunteer military.

One result of this experiment: the risk for the sons and daughters being sent into harm’s way is no longer evenly distributed across all segments of society. Many American families no longer have a personal vested interest in our nation’s decisions about foreign policy.

The unintended consequence of this decoupling is seemingly perpetual wars (we’ve been in Afghanistan for two decades). And with our country focused for two decades on fighting non-nation states – Al-Qaida and Isis – Russia re-armed and China has built weapons that have negated our strengths, matched our military, and threaten democracy around the world.

Even more corrosive to the nation is that without any type of mandatory national or public service – not just military service – we eliminated any unifying, cohesive, collective, shared national experience, or shared values.

Values
Instead our values are shaped by what we read on social media, where we find an echo-chamber of others who think like we do. Technology that was supposed to bring us together has instead sold out the country for partisanship and division, for profit over national interest. Others found it politically and/or financially profitable to create distrust in the government institutions that protect and bind us. The result is that we’re easy targets for disinformation by adversaries intent on undermining our government and its institutions.

The world isn’t a benign place. Our freedoms and values need to be defended. Throughout history the capacity of human beings to think up new ways to kill one another has proved endless. My parents, along with millions of others, lost parents, siblings, and extended family in Nazi-occupied Europe. Volunteering for national service for me was a partial payback for the country that welcomed them, sheltered them, adopted them, and allowed them to become Americans. And as much as we wish it and try, we will not eradicate violent conflict in our lifetimes or even in our children’s lifetimes. Today, the struggle for freedom and human rights continues across the globe. Ask the Uighurs, or the people in Hong Kong or Tibet what happens when their freedom is extinguished.

A Contribution to National Service
Five years ago, listening to Pete and Joe talk about the problems the Department of Defense (DoD) faced reminded me of what I noticed inside the parts of the government where I was now spending time. While there were smart, dedicated people serving their country, few of students from the schools I was teaching at were there. Few of my students knew what the DoD or other branches of government did. It just wasn’t part of their lives.

It dawned on us that building on the last national curriculum I created – the National Science Foundation I-Corps, we could hit the ground running and create our own version of a national service. We envisioned a national Hacking for Defense program across 50 universities.

It’s taken five years, but I’m proud we’ve accomplished just that. The class is now adding 1,000+ students a year, many of them choosing to change career paths to work in national service or the public sector after graduation.

Still, there’s much more we can and must do.

While my entrepreneurial career allowed me to work with people who built great products and companies, my national and public service careers connected me to those who’ve dedicated their lives to serving others. And I’ve concluded that a life lived in full measure will do both.

We need to scale the existing national and public service initiatives –AmeriCorps, YouthBuild, PeaceCorps,U.S. Digital Service, Defense Digital Service, and conservation corps– that today only reach 100,000 people. We need to offer every high school and college graduate – all 4 million of them – a shared national experience.

In the face of forces working to tear us apart, we must remember that we are stronger together, more resilient together, more successful together, than we are apart. Our challenge is to bring unity back to a nation that is built on different backgrounds and beliefs. E Pluribus Unum – Out of Many, One.

Hacking for Defense is our contribution to what hopefully will be a much larger effort to help unify the country.

Lessons Learned

  • E Pluribus Unum– Out of Many, One
  • We need to find the common threads that bind us
  • We need unifying, cohesive, collective, and shared national experiences and values will help bring the country together again
  • Hacking for Defense is our contribution

Software Once Led Us to the Precipice of Nuclear War. What Will AI Do?

A version of this story previously appeared in Defense One.

The story of RYAN and Able Archer is an oft-told lesson of a U.S. intelligence failure, miscalculation, and two superpowers unaware they were on the brink of an accidental nuclear war — all because the Soviet Union relied on a software program to make predictions that were based on false assumptions.

As more of our weapons systems and analytical and predictive systems become enabled by AI and Machine Learning, the lessons of RYAN and Able Archer is a cautionary tale for the DoD.


In 1983, the world’s superpowers drew near to accidental nuclear war, largely because the Soviet Union relied on software to make predictions that were based on false assumptions. Today, as the Pentagon moves to infuse artificial-intelligence tools into just about every aspect of its workings, it’s worth remembering the lessons of RYAN and Able Archer.

Two years earlier, the Soviet Union had deployed a software program dubbed RYAN, for Raketno Yadernoye Napadenie, or sudden nuclear missile attack. Massive for its time, RYAN sought to compute the relative power of the two superpowers by modeling 40,000 military, political, and economic factors, including 292 “indicators” reported from agents (spies) abroad. It was run by the KGB, which employed more than 200 people just to input the data.

The Soviets built RYAN to warn them when their country’s relative strength had declined to a point that the U.S. might launch a preemptive first strike on the Soviet Union. Leaders decided that if Soviet power was at least 70 percent of that of the United States the balance of power was stable. As the months went by, this number plummeted. By 1983, RYAN reported that Soviet power had declined to just 45 percent of that of the United States.

This amplified Soviet leaders’ paranoia. After 25 years of back-and-forth in the nuclear arms race, the Peacekeeper ICBM and the Trident SLBM were tipping the balance in favor of the United States. Responding to the Soviet introduction of SS-20 intermediate-range ballistic missiles to Eastern Europe in 1983, the U.S. deployed Pershing II and Ground Launched Cruise Missiles missiles to Western Europe, which reduced warning time of attack on Moscow to less than eight minutes. And in March 1983, President Reagan announced the Strategic Defense Initiative – “Star Wars” – to intercept Soviet ICBMs, then piled on just weeks later by publicly labeling the Soviet Union “the Evil Empire.” And to cap off a very bad year in the Cold War, in September 1983 the Soviets accidentally shot down a civilian 747 airliner—KAL 007—killing all 269 aboard.

By 1983, Soviet political and military leaders truly believed a nuclear war was coming. The RYAN program took on even greater importance. To feed RYAN, the KGB made its top priority to collect indicators of anything that might precede a potential surprise nuclear missile attack. They were looking for direct indicators—had the U.S. Continuity of Government program (doomsday planes) been activated? Had the U.S. given advance warning to launch our strategic nuclear forces? They also collected secondary indicators. Their agents inside the U.S. and allied countries watched for heightened activities in and around Washington offices (White House, Pentagon, State Dept, CIA, etc.), including the White House parking lot, places of evacuation and shelter, the level of blood held in blood banks, observation of where nuclear weapons were stored, etc. Some of the indicators were based on a mirror-image of how the Warsaw Pact would prepare for war. Soviet case officers were instructed to look for deviations in the behavior of people in possession of classified information suddenly moving into specially equipped secure accommodations.

While most of the KGB station chiefs and case officers thought Moscow was being paranoid, they dutifully reported what they thought their leaders wanted to hear.

By November 1983, Soviet military and political leaders had convinced themselves that a nuclear first strike from the United States was probable. The RYAN program told them that the odds favored the U.S., and the war indicators in Moscow were flashing red.

That month, NATO ran a highly realistic set of wargames in Europe called Able Archer 83. These included an airlift of 19,000 U.S. soldiers in 170 aircraft under radio silence to Europe, the shifting of commands from Permanent War Headquarters to the Alternate War Headquarters, and practicing nuclear weapons release procedures.

In reaction, the Chief of the Soviet Air Forces ordered all units of the Soviet 4th Air Army on alert which included preparations for immediate use of nuclear weapons. It appears that at least some Soviet forces were preparing to preempt or counterattack a NATO strike launched under cover of Able Archer.

Luckily, no one overreacted. The Able Archer 83 exercise passed.

For years, the U.S had no idea that the Soviet Union had believed the exercise was a cover to launch a nuclear first strike. The Berlin Wall had fallen by the time information from a defector and an end-of-tour letter from the U.S. general responsible for Air Force Intelligence in Europe prompted presidential intelligence board to revisit what the Soviets had thought. In hindsight, RYAN and Able Archer took the Cold War to the brink of Armageddon.

Even when RYAN was reporting that the U.S. had a decisive military advantage, what made the Soviets believe that we would launch a first-nuclear strike? No one knows. However, given Nazi Germany’s surprise attack on the Soviet Union in WWII, resulting in 25 million dead and the extreme devastation inflicted on their country, the Soviet Union had reason to be paranoid. Some have suggested that the Soviets had interpreted President Carter’s 1980 Presidential Directive 59 Nuclear Weapon Employment Policy as preparation for a nuclear first strike. Perhaps the Soviet Union ascribed their own plans for a first strike on the U.S. to their Cold War enemy. Or perhaps the U.S. actually did have a first-strike option in one of our operational plans that the Soviets discovered via espionage.

Why were the Soviets convinced that a war would start with a war game? Several months after Able Archer, the Soviet Minister of Defense publicly acknowledged his country’s inability to tell a big NATO exercise from an actual attack: “It was difficult to catch the difference between working out training questions and actual preparation of large-scale aggression.” It’s quite likely that the Soviets’ own plans for launching a war in Europe would have been as part of a war game.

Certainly the Soviets, believing the signals of the RYAN alert system, were primed to assume a U.S. attack. In attempting to automate military policy and potential actions, the Soviets had amplified their existing paranoia. (A movie called War Games came out that year with some of the same themes.)

A Cautionary Tale for Automating Policy and Prediction
Forty years ago RYAN attempted to automate military policy and potential actions. But in the end, RYAN failed in actually predicting U.S. intent. Instead, RYAN reinforced existing fears, and accidently created its own paranoia.

While the intelligence lessons of RYAN and Able Archer have been rehashed for decades, as our own AI initiatives scale no one is asking what lessons RYAN/Able Archer should have taught us about building predictive models and what happens when our adversaries rely on them.

Which leads to the question: What could happen when we start using Artificial Intelligence and Machine Learning to shape policy?

  • What could happen when we start using artificial intelligence and machine learning to shape policy?
  • Will AI/ML actually predict human intent?
  • What happens when the machines start seeing patterns that aren’t there?
  • How do we ensure that unintentional bias doesn’t creep into the model?
  • How much will we depend on an AI that can’t explain how it reached its decision?
  • How do we deconflict and deescalate machine-driven conclusions? Where and when should the humans be in the loop?
  • How do we ensure foreign actors can’t pollute the datasets and sensors used to drive the model and/or steal the model and look for its vulnerabilities?
  • How do we ensure that those with a specific agenda (i.e. Andropov, chairman of the KGB) don’t bias the data?
  • How do we ensure we aren’t using a software program that misleads our own leaders?

The somewhat-comforting news is that others have been thinking about these problems for a while. In 2020, the Defense Department formally adopted five AI ethical principles recommended by the Defense Innovation Board for the development of artificial intelligence capabilities: AI projects need to be Responsible, Equitable, Traceable, Reliable and Governable. The Joint Artificial Intelligence Center appointed a head of ethics policy to translate these principles into practice. Under JAIC’s 2.0 mission, they are no longer the sole developer of AI projects, but instead providing services and common software platforms. Now it’s up to the JAIC ethics front office to ensure that the hundreds of mission areas and contractors across the DoD adhere to these standards.

Here’s hoping they all remember the lessons of RYAN.

Lessons Learned

  • RYAN amplified the paranoia the Soviet leadership already had
  • The assumptions and beliefs of people who create the software shape the outcomes
  • Using data to model an adversary’s potential actions is limited by your ability to model its leaderships intent
  • Your planning and world view are almost guaranteed not to be the same as those of your adversary
  • Having an overwhelming military advantage may force an adversary into a corner. They may act in ways that seem irrational
  • Responsible, Equitable, Traceable, Reliable and Governable are great aspirational goals
%d bloggers like this: