The Lean LaunchPad Class: It’s the same, but different

It’s the same, but different

We just finished the 8th annual Lean LaunchPad class at Stanford. The team presentations are at the end of this post.

It’s hard to imagine, but only a decade ago, the capstone entrepreneurship class in most universities was how to write – or pitch- a business plan. As a serial entrepreneur turned educator, this didn’t make sense to me. In my experience, I saw that most business plans don’t survive first contact with customers.

So in 2011, with support from the Stanford Technology Ventures Program (the entrepreneurship center in the Stanford Engineering School), we created a new capstone entrepreneurship class – the Lean LaunchPad. The class was unique in that it was 1) team-based, 2) experiential, 3) lean-driven (hypothesis testing/business model/customer development/agile engineering). This new class aimed to mimic the uncertainty all startups face as they search for a business model while imparting an understanding of all the components of a business model, not just how to give a pitch or a demo.

(It’s worth reading the blog post that became the manifesto of the class here as well as what we learned when we first taught it- here.)

Ninety days after we first offered this class at Stanford, the National Science Foundation adopted the class calling it the NSF I-Corps (the Innovation Corps) to train our country’s top scientists how to commercialize their inventions. I-Corps is now offered in 88 universities. The National Institute of Health teaches its version in the National Cancer Institute. (I-Corps @ NIH). (The NIST report on Unleashing Innovation recommended expanding I-Corps and the House just passed the Innovators to Entrepreneurs Act to do just that.) The Lean LaunchPad/I-Corps syllabus is the basis for a series of Mission-Driven Entrepreneurship classes; Hacking for Diplomacy, Defense, Energy, Oceans, non-profits and cities.

If you had dropped by in 2011, the first time I taught the class, and then stuck your head in today, you’d say it was the same class. The syllabus is almost identical, the teams still get out of the building to do customer discovery every week, then come back to class and present what they learned weekly, etc.

But while it’s the same, it’s different.

After thousands of students taking this class, here are a few ways the class has changed.

—-

A Great Class Endures Beyond Its Author
I’ve always believed that great classes continue to thrive after the original teachers have moved on. While I created the Lean LaunchPad methodology and pedagogy (how to teach the class) and the train-the-trainer course for the NSF I-Corps, the sheer scale and success of the class is due to the efforts of the 100’s of National Science Foundation instructors and the NSF. And while I created the original course, the Stanford class is now led by Jeff Epstein and Steve Weinstein.

To be honest, as I watch other instructors now run these classes, I feel a proud “passing of the torch” though touched by moments of King Lear and Kurosawa’s Ran. Way past my ad hoc activities, the Stanford teaching team has thoroughly professionalized the class.

Expanded Teaching Team
In addition to the lead instructors, the Stanford teaching team now includes George John, Mar Hershenson, and Tom Bedecarre, all generously volunteering their time. Each of them brings decades of industry experience to the class. This type of teaching firepower and headcount was necessary as the teaching team expanded the class size to meet student demand.

Class Size
For the first few National Science Foundation classes, we taught 24 teams at a time with three instructors. We did it by breaking the class into three separate sections, having all teams together for our lectures and separating into sections of eight teams each when the teams presented. (After painful trial and error, we had discovered that the teaching team could listen to 8 teams present before our brains melted down.)

At Stanford we limited the class to 8 teams – four students per team. However, this year, the class was so oversubscribed, and the quality of the teams applying was so high, the teaching team admitted 14 teams and reverted to the original NSF model of separating into sections. The additional teaching team members made it possible.

Class Velocity/Depth
When we started this class, the concept of Lean (business models, customer development, agile, pivots, mvp’s) was new to everyone. Now they’re common buzzwords, and most of the students come in with an understanding of Lean. This head start has allowed the teaching team to accelerate the velocity and depth of learnings past the basics.

Women
In past years, the student teams in the Stanford classes were weighted toward men, reflecting the makeup of the applicants. While Ann Miura-Ko was part of the original teaching team, having all male instructors for the last five years didn’t help. After Mar Hershenson joined the teaching team last year, she made an all-out effort to recruit women to apply. A role model as a successful CEO and VC, Mar successfully sparked interest in women students and sponsored women-only lunch sessions, mixers and meetings to introduce them to the class. As you’ll notice from the presentations below, the result was that this year 50% of the applicants and accepted teams were women.

The lessons for me were: 1) the class had been unintentionally signaling a “boys-only” environment, 2) these unconscious biases were easily dismissed by assuming that the class makeup simply reflected the applicant pipeline, and 3) when in fact it required active outreach by a woman to change that perception and bring more women into the pipeline and subsequent teams.

Product/Market Fit Versus The Business Model Canvas
My original vision for the class was to use the business model canvas as a framework to teach engineering students all the nine elements of the business model: customer, distribution channel, revenue, get/keep/grow, value proposition, activities, resources, partners and costs. And instead of the traditional income statement, balance sheet and cash flow, discover the key “metrics that matter” for their business model.

While students want to spend their time focusing on product/market fit (who’s the customer and what should we build for them) and building product-centric minimum viable products, I thought that Y-Combinator and other accelerators already did an excellent job of that. My goal was to use the canvas to expose engineering students to other essential aspects of a successful business they may be less familiar with (sales, marketing, finance, operations.)

Admittedly this was tough to do, because in one quarter teams haven’t yet found product/market fit and are loath to move off it until they do. But since my goal was to teach a methodology rather than to run an accelerator, I traded off time on product/market fit for exposure to the rest of the canvas.

If we were designing a curriculum rather than just a single class, we’d offer it as two semesters/quarters – the first searching for problem/solution and product/market fit, and the second half focusing on the rest of the canvas testing feasibility and viability.

As you look at this year’s presentations, you can see the presentations still tend to focus on product/market fit. Obviously, there is no right answer to what and how to teach, and the answer may change over time.

TAs/ Diagnostics/Mentors
Our Teaching Assistants keep all the moving parts of the class running. Each years TAs have continued to make the class better (although I must admit it was interesting to watch the TAs remove any uncertainty from what students need to do week-to-week, as I had designed a level of uncertainty into the class to mimic what a real-world startup would feel like.) The teaching team and TA’s have added an enormous number of useful diagnostics to measure student reactions to each part of the pedagogy and the overall value of the class. However, the real art of teaching is to remember that the class wasn’t designed by a focus group.

Finally, the mentors (unpaid industry advisors) who volunteer their time have been professionalized and managed by Tom Bedecarre. Each mentor’s contribution gets graded by the students in the team they coached.

Things That Needed Constant Reminders
Every time we slipped up and admitted an all engineering or all MBA team we were reminded by their struggles that successful teams need to be diverse – that they include both innovators and entrepreneurs (typically engineers and MBA’s.)

The same holds true for pushing the students. Every time we slacked off relentlessly direct feedback we saw a commensurate drop in the quality of the teams output.

The Teams
In the end, this class is not only about what the instructors try to teach the students but also about whether students processed what we intended for them to learn. Over time, two of our major insights were: 1) teams needed a week to process all they learned, and 2) we needed to teach them how to turn that learning into a story of their journey.

This year all our teams accomplished that and much, much more.

And after 9 years of classes, students still find that this class is the closest thing to being in a real startup.

Take a look at their presentations below.

AgAI

If you can’t see the presentation click here

If you can’t see the video click here

BeaconsAI

If you can’t see the presentation click here

If you can’t see the video click here

Equify

If you can’t see the presentation click here

If you can’t see the video click here

Equipped

If you can’t see the presentation click here

If you can’t see the video click here

HardHats

If you can’t see the presentation click here

If you can’t see the video click here

Lemnos

If you can’t see the presentation click here

If you can’t see the video click here

NanoSense

If you can’t see the presentation click here

If you can’t see the video click here

Neuro

If you can’t see the presentation click here

If you can’t see the video click here

NeuroDiversity Nerds

If you can’t see the presentation click here

If you can’t see the video click here

 

Praxis

If you can’t see the presentation click here

If you can’t see the video click here

Promote.It

If you can’t see the presentation click here

If you can’t see the video click here

RightFoot

If you can’t see the presentation click here

If you can’t see the video click here

Topt

If you can’t see the presentation click here

If you can’t see the video click here

Wanderwell

If you can’t see the presentation click here

If you can’t see the video click here

We Have A Moral Obligation

I was in Boston and was interviewed by The Growth Show about my current thinking about innovation in companies and government agencies.The interviewer was great and managed to get me to summarize several years of learning in one podcast.

It’s worth a listen.

At the end of the interview I got surprised by a great question – “What’s the Problem that Still Haunts You?”  I wasn’t really prepared for the question but gave the best answer I could on the fly.

Part of the answer is the title of this blog post.

Listen to the entire interview here:
Taking the Lean Startup From Silicon Valley to Corporations and the State and Defense Department

Or just parts of the interview:
1:20  Failure and Lessons Learned

Hacking for Defense @ Stanford 2017 – Lessons Learned Presentations

We just finished our second Hacking for Defense class at Stanford. Eight teams presented their Lessons Learned presentations.

Hacking for Defense is a battle-tested problem-solving methodology that runs at Silicon Valley speed. It combines the same Lean Startup Methodology used by the National Science Foundation to commercialize science, with the rapid problem sourcing and curation methodology developed on the battlefields in Afghanistan and Iraq by Colonel Pete Newell and the US Army’s Rapid Equipping Force.

Goals for the Hacking for Defense Class
Our primary goal was to teach students entrepreneurship while they engaged in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community or other government agencies.

Our second goal was to teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there is a methodology that can help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Finally, we also wanted to show our sponsors in the Department of Defense and Intelligence community that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions to real-world problems.

The Class
Here’s a brief description of the Lean Methodology our students used:

If you can’t see the video click here

Our mantra to the students was that we wanted them to learn about “Deployment not Demos.” Our observation is that the DOD has more technology demos than they need, but often lack deep problem understanding.  Our goal was to have the students first deeply understand their sponsors problem – before they started building solutions. As you can imagine with a roomful of technologists this was tough. Further we wanted the students to understand all parts of the mission model canvas, not just the beneficiaries and the value proposition. We wanted them to learn what it takes to get their product/service deployed to the field, not give yet another demo to a general. This meant that the minimal viable products the students built were focused on maximizing their learning of what to build, not just building prototypes.

(Our sponsors did remind us, that at times getting a solution deployed meant that someone did have to see a demo!)

The Hacking for Defense class was designed as “fundamental research” to be shared broadly and the results are not subject to restriction for proprietary or national security reasons. In the 10 weeks the students have, Hacking for Defense hardware and software prototypes don’t advance beyond a Technology Readiness Level 4 and remain outside the scope of US export control regulations and restrictions on foreign national participation.

Results

  • Eight teams spoke to over 800 beneficiaries, requirements writers, program managers, warfighters, legal, security, customers, etc.
  • Seven out of the eight teams realized that the problem as given by the sponsor really wasn’t the problem. Their sponsors agreed.
  • Received from a problem sponsor mid-live stream broadcast “we are working funding for this team now.”
  • Over half the student teams have decided to continue working on national security projects after this class.

This is the End
Each of the eight teams presented a 2-minute video to provide context about their problem and then gave an 8-minute presentation of their Lessons Learned over the 10-weeks. Each of their slide presentation follow their customer discovery journey. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

The teams presented in front of several hundred people in person and online.

21st Century Frogman

If you can’t see the video click here

 

The video of the team presenting is below.  You can see all their slides right below this video.

If you can’t see the video click here

 


If you can’t see the presentation slides click here

.

.

VA Companion

If you can’t see the video click here

 

The video of the team presenting is below.  You can see all their  slides right below this video

If you can’t see the video click here

 


If you can’t see the presentation slides click here

.

.

Austra Lumina

If you can’t see the video click here

 

The video of the team presenting is below.  You can see all their  slides right below this video

If you can’t see the video click here

 


If you can’t see the presentation slides  click here

.

.

Xplomo

If you can’t see the video click here

 

The video of the team presenting is below.  You can see all their slides right below this video

If you can’t see the video click here

 


If you can’t see the presentation slides click here

.

.

Seacurity

If you can’t see the video click here

The video of the team presenting is below.  You can see all their slides right below this video

If you can’t see the video slides click here

 

If you can’t see the presentation click here

.

.

Surgency

If you can’t see the video click here

The video of the team presenting is below.  You can see all their slides right below this video

If you can’t see the slides click here

 

 


If you can’t see the presentation slides click here

.

.

Broadcom

If you can’t see the video click here

 

The video of the team presenting is below.  You can see all their slides right below this video

If you can’t see the slides click here

 


If you can’t see the presentation slides click here

The Innovation Insurgency Spreads
Hacking for Defense is now offered at eight universities in addition to Stanford – Georgetown,  University of Pittsburgh, Boise State, UC San Diego, James Madison University, University of Southern Mississippi, and later this year University of Southern California and Columbia University. We established Hacking for Defense.org a non-profit, to train educators and to provide a single point of contact for connecting the DOD/IC sponsor problems to these universities.

The Department of Defense has expanded their use of Hacking for Defense to include a classified version, and corporate partners are expanding their efforts to support the course and to create their own internal Hacking for Defense courses.

Another surprise was how applicable the “Hacking for X…” methodology is for other problems. Working with the State Department we offered a Hacking for Diplomacy class at Stanford.

Both the Defense and Diplomacy classes created lots of interest from organizations that have realized that this “Hacking for X…” problem-solving methodology is equally applicable to solving public safety, energy, policy, community and social issues internationally and within our own communities. This fall a series of new “Hacking for X…” classes will address these deserving communities. These include:

If you’re interested in learning how to apply a “Hacking for X…” class in your workplace or school we’ve partnered with the 1776 incubator in Washington DC to offer a 2-day “Hacking for X…” certification course 26-27 July for those interested in learning how. Sign up here.

It Takes a Village
While I authored this blog post, these classes are a team project. The teaching team consisted of:

  • Joe Felter a retired Army Special Forces Colonel with research and teaching appointments at Stanford’s Center for International Security and Cooperation (CISAC), the Hoover Institution, and the dept. of Management Science and Engineering. Joe is the incoming Deputy Assistant Secretary of Defense for South and Southeast Asia.
  • Pete Newell is a retired Army Colonel currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and CEO of BMNT Partners.
  • Steve Weinstein a 30-year veteran of Silicon Valley technology companies and Hollywood media companies.  Steve is CEO of MovieLabs the joint R&D lab of all the major motion picture studios.

Our teaching assistants were all prior students: Issac Matthews our lead TA, and Melisa Tokmak, Jared Dunnmon, and Darren Hau.

We were lucky to get a team of 25 mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams. Thanks to the team Lean Startup mentors: Paul Dawes, Tom Bedecarre, Kevin Ray, Craig Seidel, Daniel Bardenstein, Roi Chobadi, Donna Slade, and Rafi Holtzman and other advisors; Lisa Wallace, Peter Higgins, Steve Hong, Robert Medve.

We were privileged to have the support of an extraordinary all volunteer team of professional senior military officers representing all branches of service attending fellowship programs at Stanford’s Hoover Institution, and Center for International Security and Cooperation (CISAC) and Asia Pacific Research Center (APARC) at the Freeman Spogli Institute (FSI). These included: Colonel Lincoln Bonner (US Air Force), Colonel Curtis Burns (US Army), Captain Kurt Clark (US Coast Guard), Lieutenant Colonel Kurt Helphinstine (US Air Force), Colonel Seth Krummrich (US Army)), Commander Leo Leos (US Navy), Lieutenant Colonel Eric Reid (US Marine Corps), Colonel Mike Turley (US Army), and Colonel Dave Zinn US Army.  Additional volunteers from the active duty military providing support to our teams included  Lieutenant Colonel Donny Haseltine (US Marine Corps), Captain Jason Rathje (US Air Force), Major Dave Ahern US Army) and, Major Kevin Mott (US Army).

And finally a special thanks to our course advisor Bill Perry, former Secretary of Defense and Professor Emeritus, and Tom Byers, Professor of Engineering and Faculty Director, STVP.

Herding Cats – Using Lean to Work Together

When Colonel Peter Newell headed up the Army’s Rapid Equipping Force (REF) he used lean methods on the battlefields of Iraq and Afghanistan to provide immediate technology solutions to urgent problems.

Today, his company BMNT does for government and commercial customers what the Rapid Equipping Force did for the U.S. Army.

Pete and I created the Hacking for Defense class (with Joe Felter and Tom Byers.) One of the problems our students run into is that there are always multiple beneficiaries and stakeholders associated with a problem, often with conflicting value propositions and missions.  So how do you figure out whose needs to satisfy?

Here’s Pete’s view of how you do it.


Unlike businesses, government organizations don’t sell products, and they don’t earn revenue. Instead, they have missions to accomplish and very hard problems to solve.  They use a variant of the Business Model Canvas –  the Mission Model Canvas – to map their hypotheses, and they get of the building to do beneficiary discovery. (A beneficiary can be a soldier, program manager, commanding general, government contractor, stakeholder, customer, etc.)  And just like in a commercial business they are trying to determine whether the value proposition solves the problem and helps the beneficiary accomplish their mission.

Discovery for both business and government is similar in that the only way to do it is to turn assumptions into facts by generating hypotheses, developing Minimum Viable Products and getting out of the building to test those MVP’s in the trenches where the customers and beneficiaries work. Early in the discovery process, teams are faced with a cacophony of personalities and organizations. Often, they struggle with understanding which person or group represents a beneficiary, supporter, advocate or potential key partner. It’s only through repetitive hypothesis testing that they begin to sort them all out.

It’s in the trenches however, where things become different.

Multiple Beneficiaries, Multiple Conflicts
Unlike their commercial counterparts, government problem solvers are often faced with multiple beneficiaries associated with a problem, often with conflicting value propositions. As these differences become apparent, teams must make decisions about the value proposition trade-offs between conflicting beneficiaries – sometimes even pivoting completely in favor of one beneficiary to the detriment of another.

During last year’s Hacking for Defense class at Stanford Team Aqualink experienced the conflicting beneficiaries’ problem.  The result was a significant pivot of both beneficiary and value proposition.

Aqualink started with a problem given to them from the chief medical officer of the Navy SEALS – they had no way to understand chronic long-term health issues divers face. Divers work 60 to 200 feet underwater for 2-4 hours, but Navy doctors currently have no way to monitor divers’ core temperature, maximum dive pressure, blood pressure, pulse and the rebreather (air consumption), or the dive computer (dive profile) data.

Having all this new data would give a diver early warning of hypothermia or the bends. More importantly the data would allow the medical director to individually assess the short and long-term health of each diver. And medical researchers would have access to detailed physiological data. The medical director tasked the team with building a wearable sensor system and developing apps that would allow divers to monitor their own physiological conditions while underwater and to download it for later analysis.

In the first week of the class this team got out of the building, suited up in full Navy diving gear and did customer discovery by spending an hour in the life of the beneficiary.

But as the students on the Aqualink team spoke to the SEAL team divers, (another one of their beneficiaries), they experienced an existential crisis. Most of the divers were “ambivalent” (read hostile) about the introduction of a vitals monitoring platform, (“If you gave to us at 0900, it would end up on the bottom of the ocean by 0905.”) Having worked so hard to get into the SEALS, no diver wanted doctors telling them they could no longer dive.

After further questioning, the team discovered the reason the divers were spending so much time underwater – they often did not know where they were. To find out, they had to get a GPS fix. This meant their minisub (called the SEAL Delivery Vehicle) had to rise to within 6 feet of the ocean surface so the GPS antenna could broach the surface. And to do so they had to surface slowly to avoid giving the divers the bends.

The divers told our student team, “Screw the health sensors. Build us a GPS sensor that can be deployed from 100 feet underwater.”

Now the team had a dilemma. They would have to decide which beneficiary to focus on – the SEAL Team medical director, who was the sponsor of their problem, or the operators of the delivery vehicle and divers within SEAL Delivery Vehicle Team One, along with their immediate chains of command in SDVT-1 and Naval Special Warfare Group 3.

When they went back to the medical director with their findings, he was surprised as they were.  “Never knew that’s why they spent all that time down there.  Heck, yes, fix their problem first.”

Understanding the Problem Context and Problem Ecosystem
As Aqualink shows, getting out of the building – interviewing the beneficiaries, drawing their workflows and mapping a day-in -their-life – will give you a more complete picture of the context in which a proposed problem exists. Talking to multiple beneficiaries will lead to better understanding of the entire ecosystem of the problem. Often this will show that the problem you have been given is merely a symptom of a larger problem, or is the result of a different problem.

The solution is to:

  1. Cross check the results of your discovery between different beneficiaries. Often, you’ll find that they seldom have a complete understanding of one another’s workflows and pain points but instead are championing the solution to a mere symptom of a different problem.
  2. Share what you learned in discovery among the different beneficiaries. This will arm you with the tools needed to get them (or their leadership) to agree on the right problem that needs to be solved first. In many cases this will lead to your first pivot!

The goal is to sort out who has a value proposition that must be addressed first.

The power of beneficiaries helping one another
While discovery with multiple beneficiaries can be confusing and exhausting, there is immense power when all the beneficiaries work together. Therefore, the goal of customer discovery is not just to understand the pains and gains of individual beneficiaries, but to find a shared purpose between all of them.

Once they understand they share the same goal, they can solve pain points or create gains for each other using the resources they already control. A “shared” sense of purpose is a very powerful step in the pathway towards a deployable solution.

When the Department of Energy asked BMNT to build a training program for getting veterans into advanced manufacturing jobs, we saw the power of a shared purpose between multiple beneficiaries first hand.

The problem we were asked to solve is that of the 10,000 veterans who leave military service every month, many remain unemployed or underemployed, yet at the same time the number of unfilled advanced manufacturing jobs in the U.S. is expected to climb to over a million by 2020. From a business perspective, obtaining technically qualified talent is among the top constraints to growth in the US.

Seems like it would be a match made in heaven, right?  Not so fast…

While we initially thought the beneficiaries of the effort were the veterans, we quickly discovered there were other beneficiaries in advanced manufacturing. We found these additional beneficiaries had different pains and gains which in turn required different value propositions to solve their problems.

Our customer discovery taught us that there were three additional beneficiaries:

  • Universities needed to grow their enrollments. Our discovery showed us universities were willing to create programming for Advanced Manufacturing, but first needed to see a business case for how it would increase their enrollment to make it a worthwhile effort.
  • Industry needed to attract and hire qualified employees. We learned that technically qualified employees within industry were in such demand that the number one way to get qualified employees was to pilfer them from others.
  • Government Agencies needed to help their communities build skilled labor pools to attract new industries.

And we learned that our initial beneficiary, veterans leaving service, didn’t need internships or low-paying jobs, but needed jobs that paid enough to support their families.

We found each of these beneficiaries had a shared purpose. And each of them had a value proposition that would create a gain or relieve a pain point for another beneficiary. These were big ideas.

We found that as these overlapping value propositions emerged, we used the results to get the beneficiaries to come together in a workshop designed to jointly create a shared minimum viable product that they could then use to test within their own organizations.

Bringing the groups together in a workshop also served to align value propositions between beneficiaries by demonstrating that there was a way to create a single program that served all their needs. And we created an environment that allowed each beneficiary to discover that the other beneficiaries were partners they could work with in the future.

What was the impact of bringing the beneficiaries together in a workshop and creating this beneficiary ecosystem for advanced manufacturing?

Lawrence Livermore National Lab (LLNL) created a veterans’ jobs program. They teamed with a local college to create internships that allowed veterans to work during the summer.  In turn, the local college created additional advanced manufacturing classes to meet LLNL’s technical needs and the regional workforce investment board provided funding.

In Fort Riley, the Army base in Kansas, the military teamed with Kansas State University to create an advanced manufacturing program. Kansas State created a series of advanced manufacturing classes. Soldiers leaving the service can take these courses at a nearby campus beginning up to six months before they leave service.

An unexpected consequence is that today there are soldiers from Fort Riley using advanced manufacturing processes to create parts for vehicles and equipment at the Army base.

Lessons learned

  • Government problem solvers will often be faced with multiple beneficiaries with different value propositions. Share what you learn from different beneficiaries with each other to sort out which has a value proposition that must be addressed first.
  • The benefit of having multiple beneficiaries is that their strengths can be used to help one another create gains and relieve pains for one another. Creating a shared sense of empowerment from working together smooths the pathway towards scaling the right solution.

Don’t let process distract you from finding the strategy

When you’re up to your neck in alligators, don’t forget the goal was to drain the swamp.

I love teaching because I learn something new every class.

This time it was, “Don’t let process distract you from finding the strategy.”


The latest “aha” moment for me when I was at Columbia University teaching an intensive 5-day version of the Lean LaunchPad/I-Corps class.  The goal of the class is to expose students to the basics of the Lean MethodologyBusiness Model Design, Customer Development and Agile Engineering.

In this short version of the class, students (in teams of four) spend half their day out of the classroom testing their hypotheses by talking to customers and building minimum viable products.  The teams come back into the class and present what they found, and then they get out and talk to more customers.  Repeat for 5 days.  All teams talk to at least 50 customers/ partners/ stakeholders, and some manage to reach more than 100.

One of the teams wanted to create a new woman’s clothing brand. The good news is that they were passionate, smart and committed.  The not so good news is that other than having been customers, none of them had ever been in the fashion business. But hey, no problem.  They had the Lean Startup model to follow. They could figure it out by simply talking to customers and stores that carry unique fashion brands.  How hard can this be?!

2_8_maraBy the second day the team appeared to be making lots of progress –  they had talked to many women about their clothing line, and had marched up and down NY stores talking to buyers in clothing boutiques.  They built detailed value proposition canvases for each customer segment (young urban professional woman, students, etc.) –  trying to match customer pains, gains and jobs to be done with their value proposition (their new clothing line.) They were busily testing their hypotheses about customer segments and value proposition, seeing if they could find product/market fit.

In listening to them it dawned on me that I had fallen victim to teaching process rather than helping the teams gain insight. I asked them to remind the class what business they were in.  “We’re creating a clothing fashion brand,” was the reply.  I asked, “And how much fashion brand expertise do you have as a team?” “None, we’re using customer discovery to quickly acquire it.”  On the surface, it sounded like a good answer.

But then I asked, “Has anyone on your team asked if any of your 120 classmates are in the apparel/fashion business?”  After a moment of reflection they did just that, and eight of their classmates raised their hands. I asked, “Do you think you might want to do customer discovery first on the domain experts in your own class?”  A small lightbulb appeared over their heads.

A day later, after interviewing their classmates, the team discovered that when creating a woman’s clothing brand, the clothing itself has less to do with success than the brand does. And the one critical element in creating a brand is getting written about by a small group (less than 10) of “brand influencers” (reviewers, editors, etc.) in fashion magazines and blogs.

fashion-brandWhoah… the big insight was that how you initially “get” these key influencers – not customers or stores – is the critical part of creating a clothing fashion brand. This meant understanding these influencers was more important than anything else on the business model canvas. The team immediately added brand influencers to their business model canvas, created a separate value proposition canvas for them and started setting up customer discovery interviews.
The lessons?

  • This team was entering an existing market. (The team had already drawn the Petal Diagram mapping the competitive landscape.)petal-and-canvas
  • In an existing market there is a track record for how new entrants create a brand, get traction and scale. Many of the key insights about the business model and value proposition canvases are already known.
  • In an existing market, going through customer discovery (talking to customers, buyers, distribution channel, etc.) without first asking, “Are there any insights that can be gained by understanding the incumbent strategies, can be a trap for the unwary.”

Ironically, when I was entrepreneur I knew and practiced this. When I started a new venture in existing markets I would spend part of my initial customer discovery attending conferences, reading analysts’ reports and talking to domain experts to understand current market entry strategies. (None of this obligated me to follow the path of other companies. At times I took this information and created a different strategy to disrupt the incumbents.) But as an educator I was getting trapped in teaching the process not the strategy.

The fashion brand team’s experience was a great wake-up call.

From now on my first question to startups in an existing market is: “Tell me the critical success factors of the existing incumbents.”

Lessons Learned

  • In an existing market, draw a Petal Diagram with adjacent companies
  • Focus part of your initial customer discovery on learning competitive insights.
  • Describe how those companies entered the market. What was critical?

 

Innovation – something both parties can agree on

icorps-logoOn the last day Congress was in session in 2016, Democrats and Republicans agreed on a bill that increased innovation and research for the country.

For me, seeing Congress pass this bill, the American Innovation and Competitiveness Act, was personally satisfying. It made the program I helped start, the National Science Foundation Innovation Corps (I-Corps) a permanent part of the nation’s science ecosystem. I-Corps uses Lean Startup methods to teach scientists how to turn their discoveries into entrepreneurial, job-producing businesses.  I-Corps bridges the gap between public support of basic science and private capital funding of new commercial ventures. It’s a model for a government program that’s gotten the balance between public/private partnerships just right. Over 1,000 teams of our nation’s best scientists have been through the program.

The bill directs the expansion of I-Corps to additional federal agencies and academic institutions, as well as through state and local governments.  The new I-Corps authority also supports prototype or proof-of-concept development activities, which will better enable researchers to commercialize their innovations. The bill also explicitly says that turning federal research into companies is a national goal to promote economic growth and benefit society. For the first time, Congress has recognized the importance of government-funded entrepreneurship and commercialization education, training, and mentoring programs specifically saying that this will improve the nation’s competitiveness. And finally this bill acknowledges that networks of entrepreneurs and mentors are critical in getting technologies translated from the lab to the marketplace.

uncle-sam-2This bipartisan legislation was crafted by senators Cory Gardner (R–CO) and Gary Peters (D–MI). Senator John Thune (R–SD) chairs the Senate commerce and science committee that crafted S. 3084. After years of contention over reauthorizing the National Science Foundation, House Science Committee Chairman Lamar Smith and Ranking Member Eddie Bernice Johnson worked to negotiate the agreement that enabled both the House and the Senate to pass this bill.

While I was developing the class at Stanford, it was my counterparts at the NSF who had the vision to make the class a national program.  Thanks to Errol Arkilic, Don Millard, Babu Dasgupta, Anita LaSalle (as well as current program leaders Lydia McClure, Steven Konsek) and the over 100 instructors at the 53 universities who teach the program across the U.S.

NSF I-Corps Oct 2011But I haven’t forgotten that before everyone else thought that teaching scientists how to build companies using Lean Methods might be a good for the country, there was one congressman who got it first.  lipinskiIN 2012, Representative Dan Lipinski (D-Il), co-chair of the House STEM Education Caucus, got on an airplane and flew to Stanford to see the class first-hand.

For the first few years Lipinski was a lonely voice in Congress saying that we’ve found a better way to train our scientists to create companies and jobs.

This bill is a reauthorization of the 2010 America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Act, which set out policies that govern the NSF, the National Institute of Standards and Technology (NIST), and federal programs on innovation, manufacturing, and science and math education. Reauthorization bills don’t fund an agency, but they provide policy guidance.  It resolved partisan differences over how NSF should conduct peer review and manage research.

I-Corps is the  accelerator that helps scientists bridge the commercialization gap between their research in their labs and wide-scale commercial adoption and use.

Why This Matters
While a few of the I-Corps teams are in web/mobile/cloud, most are working on advanced technology projects that don’t make TechCrunch. You’re more likely to see their papers (in material science, robotics, diagnostics, medical devices, computer hardware, etc.) in Science or Nature.

I-Corps uses everything we know about building Lean Startups and Evidence-based Entrepreneurship to connect innovation to entrepreneurship. It’s curriculum is built on a framework of business model design, customer development and agile engineering – and its emphasis on evidence, Lessons Learned versus demos, makes it the worlds most advanced accelerator. It’s success is measured not only by the technologies that leave the labs, but how many U.S. scientists and engineers we train as entrepreneurs and how many of them pass on their knowledge to students. I-Corps is our secret weapon to integrate American innovation and entrepreneurship into every U.S. university lab.

Every time I go to Washington and spend time at the National Science Foundation or National Institute of Health I’m reminded why the U.S. leads the world in support of basic and applied science.  It’s not just the money we pour into these programs (~$125 billion/year), but the people who have dedicated themselves to make the world a better place by advancing science and technology for the common good.

Congratulations to everyone in making the Innovation Corps a national standard.

Hacking for Diplomacy @ Stanford –What We Learned With the State Department

“Being in Silicon Valley, a lot of my friends want to work for Google or Apple, but this class showed me that the problems in public service were even more challenging and rewarding. I used to watch the news about Syrian refugees and feel that I was just a bystander to a hopeless situation. But this class helped me realize I can have an impact and be part of the solution.”

Hacking for Diplomacy student

h4dip-screen-shotWe just held our final week of the Hacking for Diplomacy class, teaching students entrepreneurship and “Lean Startup” principles while they engaged in national public service applying advanced technologies to solve global challenges. Seven student teams delivered their final Lessons Learned presentations documenting their intellectual journey over just 10 short weeks in front of several hundred people in person and online. And what a journey it’s been.

In this class, we partnered with sponsors in the State Department including:

  • Office of Space and Advanced Technology
  • Bureau of Political-Military Affairs
  • Bureau of Conflict and Stabilization Operations
  • Bureau of Counterterrorism and Countering Violent Extremism
  • Bureau of Population, Refugees, and Migration
    • Office of Assistance to Europe, Central Asia, & the Americas
    • Office of Assistance to the Near East
  • Office to Monitor and Combat Trafficking in Persons

Our sponsors treated our students like serious problem solvers who could contribute unique technical skills and unfettered customer access. In exchange the sponsors got access to fresh ideas, new technology and a new perspective on serious problems.

By the end of the class our sponsors inside State had experienced a practical example of a new and powerful methodology which could help them better understand and deal with complicated international problems and apply technology where appropriate.

And finally, our students learned that they could serve their country without having to put on a uniform. Today, if college students want to give back to their country, most think of Teach for America, the Peace Corps, or Americorps or perhaps if you wanted to offer your technical skills, the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the State Department, Department of Defense, Intelligence Community or other government agencies.

(This post is a continuation of a series. See all the posts about Hacking for Diplomacy here.)

Lessons Learned – Not a Demo Day
Silicon Valley folks are familiar with Demo Days – presentations where the message is: “Here’s how smart we are right now.” That’s nice, but it doesn’t let the audience know, “Is that how smart you were three months ago, did you get smarter or dumber, what did you learn?”

Hacking for Diplomacy Lessons Learned presentations are different. Each team presents a two-minute video to provide context about their problem and then presents for eight minutes about the Lessons Learned over their ten weeks in the class.

As an example, Team Trace worked with the State Department Office to Monitor and Combat Trafficking in Persons. The team was challenged to help companies push policies of responsible business lower down the supply chain. The key thing to note in this presentation is not only that the team came up with a solution, but also how in talking to 85 people, their understanding of the problem evolved, and as it did, so did their solution. (see Slides 12 and 25).

If you can’t see the video click here.

If you can’t see the presentation click here.

Team Hacking CT was sponsored by Bureau of Counterterrorism and Countering Violent Extremism with the goal of deterring individuals from joining violent extremist groups. After 100 interviews, the team realized that a bottom-up approach, focusing on support for friends and family of those at risk for radicalization, might be effective.

If you can’t see the video click here.

If you can’t see the presentation click here.

Getting Lean
Each of the teams used the Lean Startup methodology. For those new to Lean, the process has three key components.

First, students took the problems they got from their State Department sponsors and transformed those into what we call hypotheses. For instance, one problem was: “We need to improve coordination among all the organizations trying to help Syrian refugees.” That’s a big, unwieldy problem. Students had to break it down into a series of hypotheses. They had to identify who were the beneficiaries and stakeholders, and think about what specific service they were going to provide them, how they were going to get it to them and who was going to pay for it.  To help them do that, we have them map their nine critical hypotheses onto a single sheet of paper called the Mission Model Canvas.

aggregatedb-mission-model-canvas

Then in step two, the teams got out of the classroom to test these hypotheses through interviews with people in the real world. Every team spoke to close to 100 potential “beneficiaries,” partners and stakeholders including NGOs, tech company executives, supply chain managers, foreign service officers in embassies around the world, and even refugees. While the students were interviewing, they also employed the third piece of the Lean methodology: building the solution incrementally and iteratively. These solutions, called Minimal Viable Products (MVP’s), are what allow the teams to become extremely agile and responsive.

As teams talk to stakeholders they gather evidence to either validate, invalidate or modify their hypotheses. If they find out that their assumptions are wrong (and almost all do,) they Pivot, that is, they make fundamental changes to their hypotheses, instead of blindly proceeding forward simply executing a plan. This ability to gather data, build and test MVPs, and then change course is what gives Lean it’s tremendous speed and agility to deliver rapid solutions that are needed and wanted.

As an example, Team Aggregate DB was working with the State Department Bureau of Conflict and Stabilization Operations (CSO). CSO helps embassies and diplomats to visualize, understand, and stabilize conflict. The team’s challenge was to get helps embassies and diplomats get more information about informal leader networks. Getting out of the building and talking to 87 people gave the team got a firsthand view of the downside when an embassy does not have access to the right local contacts. (Slides 3-9) 

If you can’t see the video click here.

If you can’t see the presentation click here.

As they developed MVPs, our students took these solutions out into the real world for feedback. At first the solutions were nothing more than drawings, wireframes or PowerPoint slides. As they came to understand their problems more deeply, they refined their solutions into the final products we saw.

h4dip-mvp

For example, Team 621 – Fatal Journeys worked with the State Department Bureau of Population, Refugees, and Migration. The team’s challenge: how to get more data on missing or perished refugees. In this presentation, note how the team’s understanding of the problem evolved over the course of talking to 88 people. They realized there was a missing link between key stakeholders that limited identification of perished refugees and prevented emotional and legal closure for their families. The team pivoted three times as they gained deeper and deeper insight into their problem. With each pivot, their solution radically changed. (Their first pass of problem/solution understanding is on Slides 1-29, but then they get additional insight in slides 36-50. Finally, slides 51-64 is their third and final iteration).

If you can’t see the video click here.

If you can’t see the presentation click here.

Hacking for Diplomacy was profiled this week in the L.A. Times. We’ve also had L.A. Times Beijing bureau chief Julie Makinen, who is on a JSK journalism fellowship at Stanford, helping coach students this quarter on interviewing and research techniques. Julie has been sharing her impressions of the class on this blog. Here’s her last installment:


In the Netflix age, suspense is an increasingly rare commodity. If we’re intrigued by an hour of “House of Cards,” we need not delay gratification – we can just queue up the next episode and push play. But following Stanford’s Hacking for Diplomacy class over the last 10 weeks has been like watching a TV drama the old-fashioned way. There were cliffhangers every time, and you had to wait seven days to find out what would happen next.

The class, which meets just once a week but requires massive outside work, is run not as a traditional lecture where professors drone on in front of passive students — just the opposite. It’s the students standing up in front, discussing what they’ve found out in the past seven days, what progress they’ve made, what obstacles they’ve run smack into. The teachers sit in the back row and lob questions and critiques forth — sometimes very direct critiques. That format keeps students and teachers alike on the edge of their seats.

Conflicts and misunderstandings within student teams — and between students and sponsors — cropped up as the students tried to learn about the State Department, their sponsors problem, and Lean Startup methodology all at once. Students, teachers, and interviewees said surprising, intriguing, even stunning things. Some days, you could see teams going off the rails, but instead of just shouting at your screen, “No, don’t go down that alley!” a professor would actually speak up from the back with something blunt like, “You’re way off track, and we’re firing your idea.”

And just when you thought a team had struck upon a brilliant notion for a product, they’d report back during the next session that everyone they put it in front of hated it. I started looking forward to each Thursday at 4:30 p.m. like my parents looked forward to watching “Dragnet” as kids, because the suspense was killing me.

Thursday’s season finale did not disappoint. Teams that just two or three weeks ago seemed to be foundering pulled off some amazing comebacks.

Take Team Exodus, which had spent a substantial part of the quarter focused on how to match private companies seeking to assist Syrian refugees with NGOs working in the field. Late in the term, the students scrapped that idea after finding competitors who were already deeply engaged in that space. They did a major pivot and decided to concentrate directly on refugees as customers — building on all they had learned during their first eight weeks of interviewing and research.

In week 9, they decided to build an AI chatbot on Facebook’s Messenger platform to allow refugees to ask questions like, “Where can I get clothing?” The bot will tap into a network of NGOs to source answers. A very basic prototype, built primarily by team member Kian Katanforoosh, a master’s student in computer science and management science & engineering, is already up and running.

On the eve of Thursday’s class, team members Katie Joseff and Berk Coker had a call with the United Nations’ refugee agency, UNHCR, and learned that the organization was very interested in working with the students to bring an Arabic chatbot to the field, most likely starting in Jordan.

“At the end, our team kind of came out of the weeds,” said Joseff, an undergrad majoring in human biology. “We finally got to the thing that Steve Blank talks about – where you can see the whites of a customer’s eyes and they just really want the product you’re talking about.”

Team Exodus: Coordinating information to better serve refugees

If you can’t see the video click here.

If you can’t see the presentation click here.

Watching the students’ process and progress was an eye-opener even for many State Department sponsors and private-sector mentors. Team Space Evaders, who in week 7 seemed to be vying for the title of Team Whose Proposed Products Generated the Most Yawns from Potential Customers, had an “ah-ha” moment and decided instead of focusing on tracking objects already in space, they’d pivot and concentrate on objects that will be launched in the future.

They’re proposing a “debris footprint” that would rate satellites before they’re sent into orbit on how much space junk they could generate. The team hopes that this could lead to international design standards to reduce space debris.

“They had a fundamental insight – don’t track ’em, solve it before they even get into space,” said Jonathan Margolis, deputy assistant secretary of State for science, space and health who came all the way from Washington to meet with the team and sit in on the class in Week 9. “It’s a reconceptualization of a problem we’ve really been struggling with.”

Team members Dave Gabler, a master’s student in business and public policy with an Air Force background, and Matthew Kaseman, an Army vet and freshman in aerospace engineering, said the next step is to produce a white paper that fleshes out the mathematical formulas that could underpin a ratings system, then take that to academic and industry conferences. “That would help start a public discussion and push the debate,” said Gabler.

“The math is probably the easy part,” said Pablo Quintanilla, a former Foreign Service Officer and current head of public policy for Asia for Salesforce, who served as mentor for the Space Evaders team. “There’s so much more to the behavioral side – who in the international space community will adopt this?”

Quintanilla said that working with Space Evaders drove home for him the merits of forming diverse teams to tackle problems. Besides Gabler and Kaseman, the student team included Kate Boudreau, a junior majoring in biomedical computation, and Tyler Dammann, a junior in computer science.

“This cross-functionality and working across disciplines is really effective,” Quintanilla said. “I feel like this is living proof that you should work everywhere like this.”

Team Space Evaders: Reducing space junk

If you can’t see the video click here.

If you can’t see the presentation click here.

Professor Jeremy Weinstein, a co-instructor for the class who recently served as deputy U.S. ambassador to the United Nations, acknowledged that 10 weeks is a really short time frame for the students to make any meaningful impact. But unlike an internship, where a lone student “plugs into an existing bureaucratic hierarchy and rules,” the Hacking for Diplomacy students had the advantage of being able to work in teams — and approach the problem more as outsiders.

“The students don’t have to play by the same rules [as insiders]. They can ask the non-PC questions,” said Weinstein. “To be ignorant of the rules is a blessing at times — if you can do it respectfully.”

Getting students to have a healthy appreciation for how government policy is made — sometimes painfully slowly — is part of the educational process. And so perhaps is getting bureaucrats to be more open to fresh ideas. “There’s not going to be a flip of the switch” in State as a result of this class, Weinstein said. “There is some skepticism. But I think more broadly, we’ve won some people over.”

Thursday’s wrap-up session attracted a diverse audience, including representatives from leading Silicon Valley tech companies as well as diplomats from France, Britain and Denmark. Susan Alzner, head of the U.N. Non-Governmental Liaison Service’s New York office, said after watching the student presentations, she wants to take the customer discovery and interview methodology back to her agency.

“The U.N. has lots of small teams of people who often believe they already know the solution to a problem. … And the U.N. does way too much consultation digitally. Interviews are critical. It’s so elaborate to see these students doing 100 interviews to understand a problem, but it’s so important to orient yourself before making a plan to do something.”

Team Hacking 4 Peacekeeping: Better data on, and decision-making about, peacekeeping forces

If you can’t see the video click here.

If you can’t see the presentation click here.

Weinstein noted that scaling up Hacking for Diplomacy may not be as easy as expanding Hacking for Defense, simply because “there are millions of people who work in the Department of Defense… while the size of the State Department foreign service corps is smaller than the total number of people who play in military bands.” That means there are fewer people who can serve as sponsors.

At the same time, the class could tap a wider array of sponsor organizations. “Scale maybe has to look different — we can look to the [State Department], but also UNHCR, the foreign ministry of the U.K., other international organizations,” Weinstein said. “You have to think of a different array of partners.”

Most of the State Department sponsors for this year’s class, Weinstein noted, were not political appointees but career foreign service officers or career civil servants.

“They are the glue that holds the agency together and they are key to getting anything implemented in government. And so the buy-in is there,” he said. “But they also need permission; they need a blessing to experiment with radically different ideas. And you need political cover in these bureaucracies to do this kind of work.”

“I hope,” he added, “we’ll have that cover in a subsequent administration. More than cover. Endorsement. Enthusiasm. Excitement.”


Our Teaching Team
Like the students’ efforts, the teaching of this class was also a team project. I was joined by Jeremy Weinstein, former deputy to the U.S. ambassador to the United Nations and a Stanford professor of political science; Zvika Krieger, the State Department’s representative to Silicon Valley and senior advisor for technology and innovation; retired U.S. Army Col. Joe Felter, who co-created Hacking for Defense and is a senior research scholar at the Center for International Security and Cooperation at Stanford; and Steve Weinstein, chief executive of MovieLabs who teaches entrepreneurship at Stanford and UC Berkeley.

h4dip-instructors

Our teaching assistants were Shazad Mohamed, Sam Gussman and Roland Gillah. We were fortunate to get a team of seven mentors currently or formerly served in the State Department and selflessly volunteered their time to help coach the teams. Each team also got a mentor from the tech industry who helped guide them through creating their final products. Of course, huge thanks to the Stanford students who gave their all through this class.

Going forward
While our previous Hacking for Defense class gave us a hint that doing the same for Diplomacy would work, we’re a little stunned about how well this class with the State Department went. A surprising number of students have decided to continue working on foreign policy projects after this class with the State Department or with NGO’s. Other colleges and universities have raised their hands, and said they want to offer Hacking for Diplomacy or potentially a USAID Hacking for Development class at their school.

Meanwhile our Hacking for Defense class continues to scale through H4Di.org the non profit we set up to curate the problems from our sponsors (JIDO, ARCYBER, AWG, USMC, NSA, AFNWC, SOCOM, 75th Ranger Regiment, USTRANSCOM, Cyber Force Protection Brigade, National Defense University, and the Center for Technology and National Security Policy). And H4Di.org supports the universities teaching the class this year: Stanford, UC San Diego, Georgetown, Air Force, University of Pittsburgh, James Madison University, Boise State, and RIT.

If you’re interested in offering Hacking for Diplomacy (or Defense) in your school, or if you’re a sponsor in a federal agency interested in solving problems with speed and urgency, join us at our next H4D educators class January 17-19th at Georgetown.

Lessons Learned

  • Our sponsors inside State saw examples of a new and powerful methodology – Lean which could help them better understand and deal with complicated international problems
  • Lean offers State speed and agility to deliver rapid solutions that are needed and wanted
  • Our students learned that they could serve their country without having to put on a uniform
  • Other universities are willing to have their students work on diplomacy and development problems
  • The class was a success
%d bloggers like this: