Herding Cats – Using Lean to Work Together

When Colonel Peter Newell headed up the Army’s Rapid Equipping Force (REF) he used lean methods on the battlefields of Iraq and Afghanistan to provide immediate technology solutions to urgent problems.

Today, his company BMNT does for government and commercial customers what the Rapid Equipping Force did for the U.S. Army.

Pete and I created the Hacking for Defense class (with Joe Felter and Tom Byers.) One of the problems our students run into is that there are always multiple beneficiaries and stakeholders associated with a problem, often with conflicting value propositions and missions.  So how do you figure out whose needs to satisfy?

Here’s Pete’s view of how you do it.


Unlike businesses, government organizations don’t sell products, and they don’t earn revenue. Instead, they have missions to accomplish and very hard problems to solve.  They use a variant of the Business Model Canvas –  the Mission Model Canvas – to map their hypotheses, and they get of the building to do beneficiary discovery. (A beneficiary can be a soldier, program manager, commanding general, government contractor, stakeholder, customer, etc.)  And just like in a commercial business they are trying to determine whether the value proposition solves the problem and helps the beneficiary accomplish their mission.

Discovery for both business and government is similar in that the only way to do it is to turn assumptions into facts by generating hypotheses, developing Minimum Viable Products and getting out of the building to test those MVP’s in the trenches where the customers and beneficiaries work. Early in the discovery process, teams are faced with a cacophony of personalities and organizations. Often, they struggle with understanding which person or group represents a beneficiary, supporter, advocate or potential key partner. It’s only through repetitive hypothesis testing that they begin to sort them all out.

It’s in the trenches however, where things become different.

Multiple Beneficiaries, Multiple Conflicts
Unlike their commercial counterparts, government problem solvers are often faced with multiple beneficiaries associated with a problem, often with conflicting value propositions. As these differences become apparent, teams must make decisions about the value proposition trade-offs between conflicting beneficiaries – sometimes even pivoting completely in favor of one beneficiary to the detriment of another.

During last year’s Hacking for Defense class at Stanford Team Aqualink experienced the conflicting beneficiaries’ problem.  The result was a significant pivot of both beneficiary and value proposition.

Aqualink started with a problem given to them from the chief medical officer of the Navy SEALS – they had no way to understand chronic long-term health issues divers face. Divers work 60 to 200 feet underwater for 2-4 hours, but Navy doctors currently have no way to monitor divers’ core temperature, maximum dive pressure, blood pressure, pulse and the rebreather (air consumption), or the dive computer (dive profile) data.

Having all this new data would give a diver early warning of hypothermia or the bends. More importantly the data would allow the medical director to individually assess the short and long-term health of each diver. And medical researchers would have access to detailed physiological data. The medical director tasked the team with building a wearable sensor system and developing apps that would allow divers to monitor their own physiological conditions while underwater and to download it for later analysis.

In the first week of the class this team got out of the building, suited up in full Navy diving gear and did customer discovery by spending an hour in the life of the beneficiary.

But as the students on the Aqualink team spoke to the SEAL team divers, (another one of their beneficiaries), they experienced an existential crisis. Most of the divers were “ambivalent” (read hostile) about the introduction of a vitals monitoring platform, (“If you gave to us at 0900, it would end up on the bottom of the ocean by 0905.”) Having worked so hard to get into the SEALS, no diver wanted doctors telling them they could no longer dive.

After further questioning, the team discovered the reason the divers were spending so much time underwater – they often did not know where they were. To find out, they had to get a GPS fix. This meant their minisub (called the SEAL Delivery Vehicle) had to rise to within 6 feet of the ocean surface so the GPS antenna could broach the surface. And to do so they had to surface slowly to avoid giving the divers the bends.

The divers told our student team, “Screw the health sensors. Build us a GPS sensor that can be deployed from 100 feet underwater.”

Now the team had a dilemma. They would have to decide which beneficiary to focus on – the SEAL Team medical director, who was the sponsor of their problem, or the operators of the delivery vehicle and divers within SEAL Delivery Vehicle Team One, along with their immediate chains of command in SDVT-1 and Naval Special Warfare Group 3.

When they went back to the medical director with their findings, he was surprised as they were.  “Never knew that’s why they spent all that time down there.  Heck, yes, fix their problem first.”

Understanding the Problem Context and Problem Ecosystem
As Aqualink shows, getting out of the building – interviewing the beneficiaries, drawing their workflows and mapping a day-in -their-life – will give you a more complete picture of the context in which a proposed problem exists. Talking to multiple beneficiaries will lead to better understanding of the entire ecosystem of the problem. Often this will show that the problem you have been given is merely a symptom of a larger problem, or is the result of a different problem.

The solution is to:

  1. Cross check the results of your discovery between different beneficiaries. Often, you’ll find that they seldom have a complete understanding of one another’s workflows and pain points but instead are championing the solution to a mere symptom of a different problem.
  2. Share what you learned in discovery among the different beneficiaries. This will arm you with the tools needed to get them (or their leadership) to agree on the right problem that needs to be solved first. In many cases this will lead to your first pivot!

The goal is to sort out who has a value proposition that must be addressed first.

The power of beneficiaries helping one another
While discovery with multiple beneficiaries can be confusing and exhausting, there is immense power when all the beneficiaries work together. Therefore, the goal of customer discovery is not just to understand the pains and gains of individual beneficiaries, but to find a shared purpose between all of them.

Once they understand they share the same goal, they can solve pain points or create gains for each other using the resources they already control. A “shared” sense of purpose is a very powerful step in the pathway towards a deployable solution.

When the Department of Energy asked BMNT to build a training program for getting veterans into advanced manufacturing jobs, we saw the power of a shared purpose between multiple beneficiaries first hand.

The problem we were asked to solve is that of the 10,000 veterans who leave military service every month, many remain unemployed or underemployed, yet at the same time the number of unfilled advanced manufacturing jobs in the U.S. is expected to climb to over a million by 2020. From a business perspective, obtaining technically qualified talent is among the top constraints to growth in the US.

Seems like it would be a match made in heaven, right?  Not so fast…

While we initially thought the beneficiaries of the effort were the veterans, we quickly discovered there were other beneficiaries in advanced manufacturing. We found these additional beneficiaries had different pains and gains which in turn required different value propositions to solve their problems.

Our customer discovery taught us that there were three additional beneficiaries:

  • Universities needed to grow their enrollments. Our discovery showed us universities were willing to create programming for Advanced Manufacturing, but first needed to see a business case for how it would increase their enrollment to make it a worthwhile effort.
  • Industry needed to attract and hire qualified employees. We learned that technically qualified employees within industry were in such demand that the number one way to get qualified employees was to pilfer them from others.
  • Government Agencies needed to help their communities build skilled labor pools to attract new industries.

And we learned that our initial beneficiary, veterans leaving service, didn’t need internships or low-paying jobs, but needed jobs that paid enough to support their families.

We found each of these beneficiaries had a shared purpose. And each of them had a value proposition that would create a gain or relieve a pain point for another beneficiary. These were big ideas.

We found that as these overlapping value propositions emerged, we used the results to get the beneficiaries to come together in a workshop designed to jointly create a shared minimum viable product that they could then use to test within their own organizations.

Bringing the groups together in a workshop also served to align value propositions between beneficiaries by demonstrating that there was a way to create a single program that served all their needs. And we created an environment that allowed each beneficiary to discover that the other beneficiaries were partners they could work with in the future.

What was the impact of bringing the beneficiaries together in a workshop and creating this beneficiary ecosystem for advanced manufacturing?

Lawrence Livermore National Lab (LLNL) created a veterans’ jobs program. They teamed with a local college to create internships that allowed veterans to work during the summer.  In turn, the local college created additional advanced manufacturing classes to meet LLNL’s technical needs and the regional workforce investment board provided funding.

In Fort Riley, the Army base in Kansas, the military teamed with Kansas State University to create an advanced manufacturing program. Kansas State created a series of advanced manufacturing classes. Soldiers leaving the service can take these courses at a nearby campus beginning up to six months before they leave service.

An unexpected consequence is that today there are soldiers from Fort Riley using advanced manufacturing processes to create parts for vehicles and equipment at the Army base.

Lessons learned

  • Government problem solvers will often be faced with multiple beneficiaries with different value propositions. Share what you learn from different beneficiaries with each other to sort out which has a value proposition that must be addressed first.
  • The benefit of having multiple beneficiaries is that their strengths can be used to help one another create gains and relieve pains for one another. Creating a shared sense of empowerment from working together smooths the pathway towards scaling the right solution.

Don’t let process distract you from finding the strategy

When you’re up to your neck in alligators, don’t forget the goal was to drain the swamp.

I love teaching because I learn something new every class.

This time it was, “Don’t let process distract you from finding the strategy.”


The latest “aha” moment for me when I was at Columbia University teaching an intensive 5-day version of the Lean LaunchPad/I-Corps class.  The goal of the class is to expose students to the basics of the Lean MethodologyBusiness Model Design, Customer Development and Agile Engineering.

In this short version of the class, students (in teams of four) spend half their day out of the classroom testing their hypotheses by talking to customers and building minimum viable products.  The teams come back into the class and present what they found, and then they get out and talk to more customers.  Repeat for 5 days.  All teams talk to at least 50 customers/ partners/ stakeholders, and some manage to reach more than 100.

One of the teams wanted to create a new woman’s clothing brand. The good news is that they were passionate, smart and committed.  The not so good news is that other than having been customers, none of them had ever been in the fashion business. But hey, no problem.  They had the Lean Startup model to follow. They could figure it out by simply talking to customers and stores that carry unique fashion brands.  How hard can this be?!

2_8_maraBy the second day the team appeared to be making lots of progress –  they had talked to many women about their clothing line, and had marched up and down NY stores talking to buyers in clothing boutiques.  They built detailed value proposition canvases for each customer segment (young urban professional woman, students, etc.) –  trying to match customer pains, gains and jobs to be done with their value proposition (their new clothing line.) They were busily testing their hypotheses about customer segments and value proposition, seeing if they could find product/market fit.

In listening to them it dawned on me that I had fallen victim to teaching process rather than helping the teams gain insight. I asked them to remind the class what business they were in.  “We’re creating a clothing fashion brand,” was the reply.  I asked, “And how much fashion brand expertise do you have as a team?” “None, we’re using customer discovery to quickly acquire it.”  On the surface, it sounded like a good answer.

But then I asked, “Has anyone on your team asked if any of your 120 classmates are in the apparel/fashion business?”  After a moment of reflection they did just that, and eight of their classmates raised their hands. I asked, “Do you think you might want to do customer discovery first on the domain experts in your own class?”  A small lightbulb appeared over their heads.

A day later, after interviewing their classmates, the team discovered that when creating a woman’s clothing brand, the clothing itself has less to do with success than the brand does. And the one critical element in creating a brand is getting written about by a small group (less than 10) of “brand influencers” (reviewers, editors, etc.) in fashion magazines and blogs.

fashion-brandWhoah… the big insight was that how you initially “get” these key influencers – not customers or stores – is the critical part of creating a clothing fashion brand. This meant understanding these influencers was more important than anything else on the business model canvas. The team immediately added brand influencers to their business model canvas, created a separate value proposition canvas for them and started setting up customer discovery interviews.
The lessons?

  • This team was entering an existing market. (The team had already drawn the Petal Diagram mapping the competitive landscape.)petal-and-canvas
  • In an existing market there is a track record for how new entrants create a brand, get traction and scale. Many of the key insights about the business model and value proposition canvases are already known.
  • In an existing market, going through customer discovery (talking to customers, buyers, distribution channel, etc.) without first asking, “Are there any insights that can be gained by understanding the incumbent strategies, can be a trap for the unwary.”

Ironically, when I was entrepreneur I knew and practiced this. When I started a new venture in existing markets I would spend part of my initial customer discovery attending conferences, reading analysts’ reports and talking to domain experts to understand current market entry strategies. (None of this obligated me to follow the path of other companies. At times I took this information and created a different strategy to disrupt the incumbents.) But as an educator I was getting trapped in teaching the process not the strategy.

The fashion brand team’s experience was a great wake-up call.

From now on my first question to startups in an existing market is: “Tell me the critical success factors of the existing incumbents.”

Lessons Learned

  • In an existing market, draw a Petal Diagram with adjacent companies
  • Focus part of your initial customer discovery on learning competitive insights.
  • Describe how those companies entered the market. What was critical?

 

Innovation – something both parties can agree on

icorps-logoOn the last day Congress was in session in 2016, Democrats and Republicans agreed on a bill that increased innovation and research for the country.

For me, seeing Congress pass this bill, the American Innovation and Competitiveness Act, was personally satisfying. It made the program I helped start, the National Science Foundation Innovation Corps (I-Corps) a permanent part of the nation’s science ecosystem. I-Corps uses Lean Startup methods to teach scientists how to turn their discoveries into entrepreneurial, job-producing businesses.  I-Corps bridges the gap between public support of basic science and private capital funding of new commercial ventures. It’s a model for a government program that’s gotten the balance between public/private partnerships just right. Over 1,000 teams of our nation’s best scientists have been through the program.

The bill directs the expansion of I-Corps to additional federal agencies and academic institutions, as well as through state and local governments.  The new I-Corps authority also supports prototype or proof-of-concept development activities, which will better enable researchers to commercialize their innovations. The bill also explicitly says that turning federal research into companies is a national goal to promote economic growth and benefit society. For the first time, Congress has recognized the importance of government-funded entrepreneurship and commercialization education, training, and mentoring programs specifically saying that this will improve the nation’s competitiveness. And finally this bill acknowledges that networks of entrepreneurs and mentors are critical in getting technologies translated from the lab to the marketplace.

uncle-sam-2This bipartisan legislation was crafted by senators Cory Gardner (R–CO) and Gary Peters (D–MI). Senator John Thune (R–SD) chairs the Senate commerce and science committee that crafted S. 3084. After years of contention over reauthorizing the National Science Foundation, House Science Committee Chairman Lamar Smith and Ranking Member Eddie Bernice Johnson worked to negotiate the agreement that enabled both the House and the Senate to pass this bill.

While I was developing the class at Stanford, it was my counterparts at the NSF who had the vision to make the class a national program.  Thanks to Errol Arkilic, Don Millard, Babu Dasgupta, Anita LaSalle (as well as current program leaders Lydia McClure, Steven Konsek) and the over 100 instructors at the 53 universities who teach the program across the U.S.

NSF I-Corps Oct 2011But I haven’t forgotten that before everyone else thought that teaching scientists how to build companies using Lean Methods might be a good for the country, there was one congressman who got it first.  lipinskiIN 2012, Representative Dan Lipinski (D-Il), co-chair of the House STEM Education Caucus, got on an airplane and flew to Stanford to see the class first-hand.

For the first few years Lipinski was a lonely voice in Congress saying that we’ve found a better way to train our scientists to create companies and jobs.

This bill is a reauthorization of the 2010 America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (COMPETES) Act, which set out policies that govern the NSF, the National Institute of Standards and Technology (NIST), and federal programs on innovation, manufacturing, and science and math education. Reauthorization bills don’t fund an agency, but they provide policy guidance.  It resolved partisan differences over how NSF should conduct peer review and manage research.

I-Corps is the  accelerator that helps scientists bridge the commercialization gap between their research in their labs and wide-scale commercial adoption and use.

Why This Matters
While a few of the I-Corps teams are in web/mobile/cloud, most are working on advanced technology projects that don’t make TechCrunch. You’re more likely to see their papers (in material science, robotics, diagnostics, medical devices, computer hardware, etc.) in Science or Nature.

I-Corps uses everything we know about building Lean Startups and Evidence-based Entrepreneurship to connect innovation to entrepreneurship. It’s curriculum is built on a framework of business model design, customer development and agile engineering – and its emphasis on evidence, Lessons Learned versus demos, makes it the worlds most advanced accelerator. It’s success is measured not only by the technologies that leave the labs, but how many U.S. scientists and engineers we train as entrepreneurs and how many of them pass on their knowledge to students. I-Corps is our secret weapon to integrate American innovation and entrepreneurship into every U.S. university lab.

Every time I go to Washington and spend time at the National Science Foundation or National Institute of Health I’m reminded why the U.S. leads the world in support of basic and applied science.  It’s not just the money we pour into these programs (~$125 billion/year), but the people who have dedicated themselves to make the world a better place by advancing science and technology for the common good.

Congratulations to everyone in making the Innovation Corps a national standard.

Hacking for Diplomacy @ Stanford –What We Learned With the State Department

“Being in Silicon Valley, a lot of my friends want to work for Google or Apple, but this class showed me that the problems in public service were even more challenging and rewarding. I used to watch the news about Syrian refugees and feel that I was just a bystander to a hopeless situation. But this class helped me realize I can have an impact and be part of the solution.”

Hacking for Diplomacy student

h4dip-screen-shotWe just held our final week of the Hacking for Diplomacy class, teaching students entrepreneurship and “Lean Startup” principles while they engaged in national public service applying advanced technologies to solve global challenges. Seven student teams delivered their final Lessons Learned presentations documenting their intellectual journey over just 10 short weeks in front of several hundred people in person and online. And what a journey it’s been.

In this class, we partnered with sponsors in the State Department including:

  • Office of Space and Advanced Technology
  • Bureau of Political-Military Affairs
  • Bureau of Conflict and Stabilization Operations
  • Bureau of Counterterrorism and Countering Violent Extremism
  • Bureau of Population, Refugees, and Migration
    • Office of Assistance to Europe, Central Asia, & the Americas
    • Office of Assistance to the Near East
  • Office to Monitor and Combat Trafficking in Persons

Our sponsors treated our students like serious problem solvers who could contribute unique technical skills and unfettered customer access. In exchange the sponsors got access to fresh ideas, new technology and a new perspective on serious problems.

By the end of the class our sponsors inside State had experienced a practical example of a new and powerful methodology which could help them better understand and deal with complicated international problems and apply technology where appropriate.

And finally, our students learned that they could serve their country without having to put on a uniform. Today, if college students want to give back to their country, most think of Teach for America, the Peace Corps, or Americorps or perhaps if you wanted to offer your technical skills, the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the State Department, Department of Defense, Intelligence Community or other government agencies.

(This post is a continuation of a series. See all the posts about Hacking for Diplomacy here.)

Lessons Learned – Not a Demo Day
Silicon Valley folks are familiar with Demo Days – presentations where the message is: “Here’s how smart we are right now.” That’s nice, but it doesn’t let the audience know, “Is that how smart you were three months ago, did you get smarter or dumber, what did you learn?”

Hacking for Diplomacy Lessons Learned presentations are different. Each team presents a two-minute video to provide context about their problem and then presents for eight minutes about the Lessons Learned over their ten weeks in the class.

As an example, Team Trace worked with the State Department Office to Monitor and Combat Trafficking in Persons. The team was challenged to help companies push policies of responsible business lower down the supply chain. The key thing to note in this presentation is not only that the team came up with a solution, but also how in talking to 85 people, their understanding of the problem evolved, and as it did, so did their solution. (see Slides 12 and 25).

If you can’t see the video click here.

If you can’t see the presentation click here.

Team Hacking CT was sponsored by Bureau of Counterterrorism and Countering Violent Extremism with the goal of deterring individuals from joining violent extremist groups. After 100 interviews, the team realized that a bottom-up approach, focusing on support for friends and family of those at risk for radicalization, might be effective.

If you can’t see the video click here.

If you can’t see the presentation click here.

Getting Lean
Each of the teams used the Lean Startup methodology. For those new to Lean, the process has three key components.

First, students took the problems they got from their State Department sponsors and transformed those into what we call hypotheses. For instance, one problem was: “We need to improve coordination among all the organizations trying to help Syrian refugees.” That’s a big, unwieldy problem. Students had to break it down into a series of hypotheses. They had to identify who were the beneficiaries and stakeholders, and think about what specific service they were going to provide them, how they were going to get it to them and who was going to pay for it.  To help them do that, we have them map their nine critical hypotheses onto a single sheet of paper called the Mission Model Canvas.

aggregatedb-mission-model-canvas

Then in step two, the teams got out of the classroom to test these hypotheses through interviews with people in the real world. Every team spoke to close to 100 potential “beneficiaries,” partners and stakeholders including NGOs, tech company executives, supply chain managers, foreign service officers in embassies around the world, and even refugees. While the students were interviewing, they also employed the third piece of the Lean methodology: building the solution incrementally and iteratively. These solutions, called Minimal Viable Products (MVP’s), are what allow the teams to become extremely agile and responsive.

As teams talk to stakeholders they gather evidence to either validate, invalidate or modify their hypotheses. If they find out that their assumptions are wrong (and almost all do,) they Pivot, that is, they make fundamental changes to their hypotheses, instead of blindly proceeding forward simply executing a plan. This ability to gather data, build and test MVPs, and then change course is what gives Lean it’s tremendous speed and agility to deliver rapid solutions that are needed and wanted.

As an example, Team Aggregate DB was working with the State Department Bureau of Conflict and Stabilization Operations (CSO). CSO helps embassies and diplomats to visualize, understand, and stabilize conflict. The team’s challenge was to get helps embassies and diplomats get more information about informal leader networks. Getting out of the building and talking to 87 people gave the team got a firsthand view of the downside when an embassy does not have access to the right local contacts. (Slides 3-9) 

If you can’t see the video click here.

If you can’t see the presentation click here.

As they developed MVPs, our students took these solutions out into the real world for feedback. At first the solutions were nothing more than drawings, wireframes or PowerPoint slides. As they came to understand their problems more deeply, they refined their solutions into the final products we saw.

h4dip-mvp

For example, Team 621 – Fatal Journeys worked with the State Department Bureau of Population, Refugees, and Migration. The team’s challenge: how to get more data on missing or perished refugees. In this presentation, note how the team’s understanding of the problem evolved over the course of talking to 88 people. They realized there was a missing link between key stakeholders that limited identification of perished refugees and prevented emotional and legal closure for their families. The team pivoted three times as they gained deeper and deeper insight into their problem. With each pivot, their solution radically changed. (Their first pass of problem/solution understanding is on Slides 1-29, but then they get additional insight in slides 36-50. Finally, slides 51-64 is their third and final iteration).

If you can’t see the video click here.

If you can’t see the presentation click here.

Hacking for Diplomacy was profiled this week in the L.A. Times. We’ve also had L.A. Times Beijing bureau chief Julie Makinen, who is on a JSK journalism fellowship at Stanford, helping coach students this quarter on interviewing and research techniques. Julie has been sharing her impressions of the class on this blog. Here’s her last installment:


In the Netflix age, suspense is an increasingly rare commodity. If we’re intrigued by an hour of “House of Cards,” we need not delay gratification – we can just queue up the next episode and push play. But following Stanford’s Hacking for Diplomacy class over the last 10 weeks has been like watching a TV drama the old-fashioned way. There were cliffhangers every time, and you had to wait seven days to find out what would happen next.

The class, which meets just once a week but requires massive outside work, is run not as a traditional lecture where professors drone on in front of passive students — just the opposite. It’s the students standing up in front, discussing what they’ve found out in the past seven days, what progress they’ve made, what obstacles they’ve run smack into. The teachers sit in the back row and lob questions and critiques forth — sometimes very direct critiques. That format keeps students and teachers alike on the edge of their seats.

Conflicts and misunderstandings within student teams — and between students and sponsors — cropped up as the students tried to learn about the State Department, their sponsors problem, and Lean Startup methodology all at once. Students, teachers, and interviewees said surprising, intriguing, even stunning things. Some days, you could see teams going off the rails, but instead of just shouting at your screen, “No, don’t go down that alley!” a professor would actually speak up from the back with something blunt like, “You’re way off track, and we’re firing your idea.”

And just when you thought a team had struck upon a brilliant notion for a product, they’d report back during the next session that everyone they put it in front of hated it. I started looking forward to each Thursday at 4:30 p.m. like my parents looked forward to watching “Dragnet” as kids, because the suspense was killing me.

Thursday’s season finale did not disappoint. Teams that just two or three weeks ago seemed to be foundering pulled off some amazing comebacks.

Take Team Exodus, which had spent a substantial part of the quarter focused on how to match private companies seeking to assist Syrian refugees with NGOs working in the field. Late in the term, the students scrapped that idea after finding competitors who were already deeply engaged in that space. They did a major pivot and decided to concentrate directly on refugees as customers — building on all they had learned during their first eight weeks of interviewing and research.

In week 9, they decided to build an AI chatbot on Facebook’s Messenger platform to allow refugees to ask questions like, “Where can I get clothing?” The bot will tap into a network of NGOs to source answers. A very basic prototype, built primarily by team member Kian Katanforoosh, a master’s student in computer science and management science & engineering, is already up and running.

On the eve of Thursday’s class, team members Katie Joseff and Berk Coker had a call with the United Nations’ refugee agency, UNHCR, and learned that the organization was very interested in working with the students to bring an Arabic chatbot to the field, most likely starting in Jordan.

“At the end, our team kind of came out of the weeds,” said Joseff, an undergrad majoring in human biology. “We finally got to the thing that Steve Blank talks about – where you can see the whites of a customer’s eyes and they just really want the product you’re talking about.”

Team Exodus: Coordinating information to better serve refugees

If you can’t see the video click here.

If you can’t see the presentation click here.

Watching the students’ process and progress was an eye-opener even for many State Department sponsors and private-sector mentors. Team Space Evaders, who in week 7 seemed to be vying for the title of Team Whose Proposed Products Generated the Most Yawns from Potential Customers, had an “ah-ha” moment and decided instead of focusing on tracking objects already in space, they’d pivot and concentrate on objects that will be launched in the future.

They’re proposing a “debris footprint” that would rate satellites before they’re sent into orbit on how much space junk they could generate. The team hopes that this could lead to international design standards to reduce space debris.

“They had a fundamental insight – don’t track ’em, solve it before they even get into space,” said Jonathan Margolis, deputy assistant secretary of State for science, space and health who came all the way from Washington to meet with the team and sit in on the class in Week 9. “It’s a reconceptualization of a problem we’ve really been struggling with.”

Team members Dave Gabler, a master’s student in business and public policy with an Air Force background, and Matthew Kaseman, an Army vet and freshman in aerospace engineering, said the next step is to produce a white paper that fleshes out the mathematical formulas that could underpin a ratings system, then take that to academic and industry conferences. “That would help start a public discussion and push the debate,” said Gabler.

“The math is probably the easy part,” said Pablo Quintanilla, a former Foreign Service Officer and current head of public policy for Asia for Salesforce, who served as mentor for the Space Evaders team. “There’s so much more to the behavioral side – who in the international space community will adopt this?”

Quintanilla said that working with Space Evaders drove home for him the merits of forming diverse teams to tackle problems. Besides Gabler and Kaseman, the student team included Kate Boudreau, a junior majoring in biomedical computation, and Tyler Dammann, a junior in computer science.

“This cross-functionality and working across disciplines is really effective,” Quintanilla said. “I feel like this is living proof that you should work everywhere like this.”

Team Space Evaders: Reducing space junk

If you can’t see the video click here.

If you can’t see the presentation click here.

Professor Jeremy Weinstein, a co-instructor for the class who recently served as deputy U.S. ambassador to the United Nations, acknowledged that 10 weeks is a really short time frame for the students to make any meaningful impact. But unlike an internship, where a lone student “plugs into an existing bureaucratic hierarchy and rules,” the Hacking for Diplomacy students had the advantage of being able to work in teams — and approach the problem more as outsiders.

“The students don’t have to play by the same rules [as insiders]. They can ask the non-PC questions,” said Weinstein. “To be ignorant of the rules is a blessing at times — if you can do it respectfully.”

Getting students to have a healthy appreciation for how government policy is made — sometimes painfully slowly — is part of the educational process. And so perhaps is getting bureaucrats to be more open to fresh ideas. “There’s not going to be a flip of the switch” in State as a result of this class, Weinstein said. “There is some skepticism. But I think more broadly, we’ve won some people over.”

Thursday’s wrap-up session attracted a diverse audience, including representatives from leading Silicon Valley tech companies as well as diplomats from France, Britain and Denmark. Susan Alzner, head of the U.N. Non-Governmental Liaison Service’s New York office, said after watching the student presentations, she wants to take the customer discovery and interview methodology back to her agency.

“The U.N. has lots of small teams of people who often believe they already know the solution to a problem. … And the U.N. does way too much consultation digitally. Interviews are critical. It’s so elaborate to see these students doing 100 interviews to understand a problem, but it’s so important to orient yourself before making a plan to do something.”

Team Hacking 4 Peacekeeping: Better data on, and decision-making about, peacekeeping forces

If you can’t see the video click here.

If you can’t see the presentation click here.

Weinstein noted that scaling up Hacking for Diplomacy may not be as easy as expanding Hacking for Defense, simply because “there are millions of people who work in the Department of Defense… while the size of the State Department foreign service corps is smaller than the total number of people who play in military bands.” That means there are fewer people who can serve as sponsors.

At the same time, the class could tap a wider array of sponsor organizations. “Scale maybe has to look different — we can look to the [State Department], but also UNHCR, the foreign ministry of the U.K., other international organizations,” Weinstein said. “You have to think of a different array of partners.”

Most of the State Department sponsors for this year’s class, Weinstein noted, were not political appointees but career foreign service officers or career civil servants.

“They are the glue that holds the agency together and they are key to getting anything implemented in government. And so the buy-in is there,” he said. “But they also need permission; they need a blessing to experiment with radically different ideas. And you need political cover in these bureaucracies to do this kind of work.”

“I hope,” he added, “we’ll have that cover in a subsequent administration. More than cover. Endorsement. Enthusiasm. Excitement.”


Our Teaching Team
Like the students’ efforts, the teaching of this class was also a team project. I was joined by Jeremy Weinstein, former deputy to the U.S. ambassador to the United Nations and a Stanford professor of political science; Zvika Krieger, the State Department’s representative to Silicon Valley and senior advisor for technology and innovation; retired U.S. Army Col. Joe Felter, who co-created Hacking for Defense and is a senior research scholar at the Center for International Security and Cooperation at Stanford; and Steve Weinstein, chief executive of MovieLabs who teaches entrepreneurship at Stanford and UC Berkeley.

h4dip-instructors

Our teaching assistants were Shazad Mohamed, Sam Gussman and Roland Gillah. We were fortunate to get a team of seven mentors currently or formerly served in the State Department and selflessly volunteered their time to help coach the teams. Each team also got a mentor from the tech industry who helped guide them through creating their final products. Of course, huge thanks to the Stanford students who gave their all through this class.

Going forward
While our previous Hacking for Defense class gave us a hint that doing the same for Diplomacy would work, we’re a little stunned about how well this class with the State Department went. A surprising number of students have decided to continue working on foreign policy projects after this class with the State Department or with NGO’s. Other colleges and universities have raised their hands, and said they want to offer Hacking for Diplomacy or potentially a USAID Hacking for Development class at their school.

Meanwhile our Hacking for Defense class continues to scale through H4Di.org the non profit we set up to curate the problems from our sponsors (JIDO, ARCYBER, AWG, USMC, NSA, AFNWC, SOCOM, 75th Ranger Regiment, USTRANSCOM, Cyber Force Protection Brigade, National Defense University, and the Center for Technology and National Security Policy). And H4Di.org supports the universities teaching the class this year: Stanford, UC San Diego, Georgetown, Air Force, University of Pittsburgh, James Madison University, Boise State, and RIT.

If you’re interested in offering Hacking for Diplomacy (or Defense) in your school, or if you’re a sponsor in a federal agency interested in solving problems with speed and urgency, join us at our next H4D educators class January 17-19th at Georgetown.

Lessons Learned

  • Our sponsors inside State saw examples of a new and powerful methodology – Lean which could help them better understand and deal with complicated international problems
  • Lean offers State speed and agility to deliver rapid solutions that are needed and wanted
  • Our students learned that they could serve their country without having to put on a uniform
  • Other universities are willing to have their students work on diplomacy and development problems
  • The class was a success

Machine Learning Meets the Lean Startup

We just finished our Lean LaunchPad class at UC Berkeley’s engineering school where many of the teams embedded machine learning technology into their products.

It struck me as I watched the teams try to find how their technology would solve real customer problems, is that machine learning is following a similar pattern of previous technical infrastructure innovations. Early entrants get sold to corporate acquirers at inflated prices for their teams, their technology, and their tools. Later entrants who miss that wave have to build real products that people want to buy.

—–

I’ve lived through several technology infrastructure waves; the Unix business, the first AI and VR waves in the 1980’s, the workstation wave, multimedia wave, the first internet wave. Each of those had a set of common characteristics that the Gartner Group characterizes as the Hype Cycle .

hype-cycle-gartner

The five stages of the hype cycle are:

Stage 1: The Technology Trigger: A potential technology breakthrough kicks things off. Early proof-of-concept stories and media interest trigger significant publicity. Often no usable products exist and commercial viability is unproven.

Stage 2: Peak of Inflated Expectations: Early publicity produces a number of success stories—often accompanied by scores of failures. Some companies take action; most don’t.

Stage 3: Trough of Disillusionment: Interest wanes as experiments and implementations fail to deliver. Producers of the technology shake out or fail. Investments continue only if the surviving providers improve their products to the satisfaction of early adopters.

Stage 4: Slope of Enlightenment: More instances of how the technology can benefit the enterprise start to crystallize and become more widely understood. Second- and third-generation products appear from technology providers. More enterprises fund pilots; conservative companies remain cautious.

Stage 5: Plateau of Productivity: Mainstream adoption starts to take off. Criteria for assessing provider viability are more clearly defined. The technology’s broad market applicability and relevance are clearly paying off.

Shiny Object meets First Mover Advantage
What’s become apparent in the last few technology hype cycles is that for startups and their investors there is a short multi-year window of opportunity (at the Peak of Inflated Expectations) to sell a startup at an inflated price. This occurs because large technology companies (Google, Facebook, IBM, Microsoft, Twitter, Apple, Salesforce, Intel, et al,) and increasingly other non-tech firms, are in an arms race to stay relevant. For example, according to CBInsights nearly 140 machine intelligence have been acquired since 2011, with over 40 being bought so far in 2016.

untitled-2

Most often the first acquisitions in a hype cycle are for the “shiny objects” – the technology, the team and the tools. The acquired technical teams usually start up or complement the company’s research group in a specific new technology area.hype-cycle

If you’re a startup (or their investors) getting acquired at this point in the hype cycle is exactly where you want to be – short time in business, large acquisition price, value based on a frenzy, perceived scarcity of expertise, and fear of a competitor getting the key talent.

History shows that the acquirers often overpay buying this expertise early. While these acquisitions have teams of great researchers, they rarely contribute actual revenue generating products (because most never reached that stage when they were acquired.)  The irony is that the acquisitions made later in the hype cycle – when companies have built real products that customers want, are the ones that generate revenue and profit for the acquirer.

I had all that in mind as we watched our teams present.

Machine Learning Meets Lean – Berkeley Lean LaunchPad Class

Each of our teams in this class followed the canonical Lean model:

  1. Articulate your hypotheses using the business model canvas
  2. Get of the building and test those hypotheses using customer development
  3. Validate learning by building minimal viable products and getting them in front of customers

Each week the teams got out of the classroom and talked to 10-15 customers, testing a new part of the business model canvas.  And after week two, they had to build and then update their minimal viable product weekly. And present what they learned each week in an 8-minute presentations.

The presentations below are their final Lessons Learned presentations, along with a 2-minute video summary.

SalesStash
Three Berkeley PhD computer science students and an MBA working on machine learning. How can you not hit out of the park on day one?

This team epitomized rapid learning. Once their initial assumptions ran into the wall of actual customer feedback they rapidly built multiple minimum viable products (MVPs) and kept pivoting until they found product/market fit (i.e. a customer segment that was grabbing the product out of their hands.)

If you can’t see the video click here

If you can’t see the presentation click here

Delphi
Before this class this team had spent three months in an incubator building the product after talking to only one customer. After  week two of the class they realized they had wasted three months building something no one actually wanted. What they next learned was pretty amazing.

If you can’t see the video click here

If you can’t see the presentation click here

Homeslice
Homeslice had a great journey. They came together over a personal pain – the inability to afford a house in Silicon Valley. Their initial plan was to provide fractional ownership to solve that problem. But they found that first serving an adjacent market – slices of investment properties – could serve as a launchpad for their initial idea of fractional home ownership.

If you can’t see the video click here

If you can’t see the presentation click here

Exit Strategy
Exit Strategy was building the penultimate planning tool. This teams learning that this wasn’t a business was as important as finding one that is. Really impressive process.

If you can’t see the video click here

If you can’t see the presentation click here



This class this was a team effort. Professor Kurt Keutzer and Errol Arkilic (former program director for the National Science Foundation Innovation Corps (NSF I-Corps), now founder of M34 Capital) were the lead instructors. Steve Weinstein (CEO of MovieLabs) and I assisted. Thanks to our TA Kathryn Crimmins and all the team mentors: Lev Mass, Kanu Gulati, Ewald Detjens, James Cham, Kanu Gulati, Patrick Chung, Rick Lazansky, Ashmeet Sidhana, Mike Olson, Michael Borrus, Fabrizo De Pasquale, Amit Kumar, Rob Rodick, Mar Hershenson.

There are 145 Entrepreneurship Courses at Stanford

Stanford is an incubator with dorms

Download the full text file with links to the courses here.
http://www.slideshare.net/sblank/stanford-entreprenuership-classes

stanford-ent-classes-page-1-of-8stanford-ent-classes-page-2-of-8stanford-ent-classes-page-3-of-8stanford-ent-classes-page-4-of-8stanford-ent-classes-page-5-of-8stanford-ent-classes-page-6-of-8stanford-ent-classes-page-7-of-8stanford-ent-classes-page-8-of-8stanford-ent-classes-page-9-of-9

Hacking for Diplomacy – Solving Foreign Policy Challenges with the Lean LaunchPad

Hacking for Diplomacy is a new course from the Management Science and Engineering department in Stanford’s Engineering school and Stanford’s International Policy Studies program that will be first offered in the Fall of 2016.

Join a select cross-disciplinary class that takes real problems from the U.S. State Department and asks students to use Lean Methods to test their understanding of the problem and deliver rapid-fire innovative solutions to pressing diplomacy, development and foreign policy challenges.

H4Dip home page


Syrian Refugees, Human Trafficking, Zika Virus, Illegal Fishing, Weapons of Mass Destruction Detection, ISIS on-line propaganda, Anti-Corruption…
What do all these problems have in common? The U.S. Department of State is working on all of them.

The U.S. Department of State manages America’s relationships with 180 foreign governments, international organizations, and the people of other countries with 270 embassies, consulates, and other posts. The management of all these relationships is called diplomacy. 70,000 State Department employees (46,000 Foreign Service Nationals, 14,000 Foreign Service Employees and 11,000 in the Civil Service) carry out the President’s foreign policy and help build a freer, more prosperous, and secure world.

The State Department has four main foreign policy goals:

  • Protect the United States and Americans
  • Advance democracy, human rights, and other global interests
  • Promote international understanding of American values and policies
  • Support U.S. diplomats, government officials, and all other personnel at home and abroad.

At a time of significant global uncertainty, diplomats are grappling with a set of transnational and cross-cutting challenges that defy easy solution. These include the continued pursuit of weapons of mass destruction by states and non-state groups, the outbreak of internal conflict across the Middle East and in parts of Africa, the most significant flow of refugees since World War II, and a changing climate that is beginning to affect both developed and developing countries.  And that’s just on Monday.  The rest of the week is equally busy.

State dept org chart

Hacking for Diplomacy
In a world of complex threats, dynamic opportunities, and diffuse power, effective diplomacy and development require institutions that adapt, embrace technology, and allow for experimentation to ensure continuous learning. This means developing new and innovative ways to think about, organize and build diplomatic strategies and solutions.  Stanford’s new Hacking for Diplomacy class is a part of this effort.

Hacking for Diplomacy is designed to provide students the opportunity to learn how to work with the Department of State to address urgent foreign policy challenges. While the traditional tools of statecraft remain relevant, policymakers are looking to harness the power of new technologies to rethink how the U.S. government approaches and responds to  the problems outlined above and other long-standing challenges. In this class, student teams will take actual foreign policy challenges and a hands-on approach that will require close engagement with officials in the U.S. State Department and other civilian agencies. They’ll learn how to apply “lean startup” principles, (“mission model canvas,” “customer development,” and “agile engineering”) to discover and validate agency and user needs and to continually build iterative prototypes to test whether they understood the problem and solution.

Each week, teams will use the Mission Model Canvas (a variant of the Business Model Canvas used for government agencies and non-profits) to develop a set of initial hypotheses about a solution to the problem and will get out of the building and talk to relevant stakeholders and users. As they learn, they’ll iterate and pivot on these hypotheses through customer discovery and build minimal viable prototypes (MVPs). Each team will be guided by a sponsor from the State Department bureau that proposed the problem and a second mentor from the local community.

Real Problems
In this class, student teams select from an existing set of problems provided by the Department of State. Hacking for Diplomacy is not a product incubator for a specific technology solution. Instead, as teams follow the Mission Model Canvas, they’ll discover a deeper understanding of the selected problem, the challenges of deploying the solutions, and the host of potential technological solutions that might be arrayed to solve them. Using the Lean LaunchPad Methodology the class focuses teams to:

  • Deeply understand the problems/needs of State Department beneficiaries and stakeholders
  • Rapidly iterate technology solutions while searching for product-market fit
  • Understand all the stakeholders, deployment issues, costs, resources, and ultimate mission value
  • Produce a repeatable model that can be used to launch other potential technology solutions

National Service
Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or AmeriCorps. Few consider opportunities to make the world safer with the State Department and other government agencies. Hacking for Diplomacy will promote engagement between students and the Department of State and provide a hands-on opportunity to solve real diplomacy, foreign policy and national security problems.

Like its sister class, Hacking for Defense, our goal is to open-source this class to other universities. By creating a national network of colleges and universities, Hacking for Diplomacy can scale to provide hundreds of solutions to critical diplomacy, policy, and national security problems every year.

We’re going to create a network of entrepreneurial students who understand the diplomatic, policy, and national security problems facing the country and get them engaged in partnership with islands of innovation in the Department of State. This is a first step to a more agile, responsive and resilient, approach to diplomacy and national security in the 21st century.

Lessons Learned

  • Hacking for Diplomacy is a new class that teaches students how to:
    • Use the Lean LaunchPad methodology to deeply understand the problems/needs of State Department customers
    • Deliver minimum viable products that match State Department needs in an extremely short time
  • The class will also teach the islands of innovation in the Department of State:
    • how the innovation culture and mindset operate at speed
    • how to identify potential dual-use technologies that exist outside their agencies and contractors (and are in university labs, or are commercial off-the-shelf solutions)
    • how to use an entrepreneurial mindset and Lean Methodologies to solve foreign policy problems
  • Register for the inaugural class at Stanford starting September 29th now
%d bloggers like this: