Hacking for Defense @ Stanford 2021 Lessons Learned Presentations

Posted on June 8, 2021 by steve blank

We just finished our 6th annual Hacking for Defense class at Stanford.

What a year. With the pandemic winding down it finally feels like the beginning of the end.

This was my sixth time teaching a virtual class during the lockdown – and for our students likely their 15th or more. Hacking for Defense has teams of students working to understand and solve national security problems. Although the class was run completely online, and even though they were suffering from Zoom fatigue, the 10 teams of 42 students collectively interviewed 1,142 beneficiaries, stakeholders, requirements writers, program managers, industry partners, etc. – while simultaneously building a series of minimal viable products.

At the end of the quarter, each of the teams gave a final “Lessons Learned” presentation. Unlike traditional demo days or Shark Tanks which are, “Here’s how smart I am, and isn’t this a great product, please give me money,” a Lessons Learned presentation tells the story of a team’s 10-week journey and hard-won learning and discovery. For all of them it’s a roller coaster narrative describing what happens when you discover that everything you thought you knew on day one was wrong and how they eventually got it right.

Here’s how they did it and what they delivered.

How Do You Get Out of the Building When You Can’t Get Out of the Building?
This class is built on conducting in-person of interviews with customers/ beneficiaries and stakeholders, but due to the pandemic, teams now had to do all their customer discovery via a computer screen. This would seem to be a fatal stake through the heart of the class. How would customer interviews work via video? After teaching remotely for the last year, we’ve learned that customer discovery is actually more efficient using video conferencing. It increased the number of interviews the students were able to do each week.

Many of the people the students needed to talk to were sheltering at home, which meant they weren’t surrounded by gatekeepers. While the students missed gaining the context of standing on a navy ship or visiting a drone control station or watching someone try their app or hardware, the teaching team’s assessment was that remote interviews were more than an adequate substitute. When Covid restrictions are over, we plan to add remote customer discovery to the students’ toolkit. (See here for an extended discussion of remote customer discovery.)

We Changed The Class Format
While teaching remotely we made two major changes to the class. Previously, each of the teams presented a weekly ten-minute summary consisting of “here’s what we thought, here’s what we did, here’s what we found, here’s what we’re going to do next week.” While we kept that cadence, it was too exhausting for all the other teams to stare at their screen watching every other team present. So we split the weekly student presentations into thirds – three teams presented to the entire class then three teams each went into two Zoom breakout rooms. During the quarter we rotated the teams and instructors through the main room and breakout sessions.

The second change was the addition of alumni guest speakers – students who had taken the class in the past. They offered insights about what they got right and wrong and what they wished they had known.

Lessons Learned Presentation Format
For the final Lessons Learned presentation many of the eight teams presented a 2-minute video to provide context about their problem. This was followed by an 8-minute slide presentation describing their customer discovery journey over the 10 weeks. While all the teams used the Mission Model Canvas, (videos here), Customer Development and Agile Engineering to build Minimal Viable Products, each of their journeys was unique.

By the end the class all the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

All the presentations are worth a watch.

Team Fleetwise – Vehicle Fleet Management

If you can’t see the Fleetwise 2-minute video, click here

If you can’t see the Fleetwise slides, click here

Mission-Driven Entrepreneurship
This class is part of a bigger idea – Mission-Driven Entrepreneurship. Instead of students or faculty coming in with their own ideas, we ask them to work on societal problems, whether they’re problems for the State Department or the Department of Defense or non-profits/NGOs or the Oceans and Climate or for anything the students are passionate about. The trick is we use the same Lean LaunchPad / I-Corps curriculum — and the same class structure – experiential, hands-on– driven this time by a mission-model not a business model. (The National Science Foundation, National Security Agency and the Common Mission Project have helped promote the expansion of the methodology worldwide.)

Mission-driven entrepreneurship is the answer to students who say, “I want to give back. I want to make my community, country or world a better place, while being challenged to solve some of the toughest problems.”

Project Agrippa – Logistics and Sustainment in IndoPacific

If you can’t see the Project Agrippa 2-minute video, click here

If you can’t see the Project Agrippa slides, click here

It Started With An Idea
Hacking for Defense has its origins in the Lean LaunchPad class I first taught at Stanford in 2011. I observed that teaching case studies and/or how to write a business plan as a capstone entrepreneurship class didn’t match the hands-on chaos of a startup. Furthermore, there was no entrepreneurship class that combined experiential learning with the Lean methodology. Our goal was to teach both theory and practice.

The same year we started the class, it was adopted by the National Science Foundation to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 9 regional locations supporting 98 universities and has trained over ~2,500 teams/7,500 scientists in 100 cohorts. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

Team Silknet – Detecting Ground Base Threats

If you can’t see the Silknet 2-minute video, click here

If you can’t see the Silknet slides, click here

Origins Of Hacking For Defense
In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford, we observed that students in our research universities had little connection to the problems their government was trying to solve or the larger issues civil society was grappling with. As we thought about how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. That year we launched both Hacking for Defense and Hacking for Diplomacy (with Professor Jeremy Weinstein and the State Department) at Stanford.

Team Flexible Fingerprints – Improve Cybersecurity

If you can’t see the Flexible Fingerprints 2-minute video, click here

If you can’t see the Flexible Fingerprints slides, click here

Goals for the Hacking for Defense Class
Our primary goal was to teach students Lean Innovation while they engaged in national public service. Today if college students want to give back to their country, they think of Teach for America, the Peace Corps, or AmeriCorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence community or other government agencies.

In the class we saw that students could learn about the nation’s threats and security challenges while working with innovators inside the DoD and Intelligence Community. At the same time the experience would introduce to the sponsors, who are innovators inside the Department of Defense (DOD) and Intelligence Community (IC), a methodology that could help them understand and better respond to rapidly evolving threats. We wanted to show that if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, defense acquisition programs could operate at speed and urgency and deliver timely and needed solutions.

Finally, we wanted to familiarize students with the military as a profession and help the better understand its expertise, and its proper role in society. We hoped it would also show our sponsors in the Department of Defense and Intelligence community that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions to real-world problems.

Team Neurosmart – Optimizing Performance of Special Operators

If you can’t see the Neurosmart 2-minute video, click here

If you can’t see the Neurosmart slides, click here

Mission-Driven in 50 Universities and Continuing to Expand in Scope and Reach
What started as a class is now a movement.

From its beginning with our Stanford class, Hacking for Defense is now offered in over 50 universities in the U.S., as well as in the UK and Australia. Steve Weinstein started Hacking for Impact (Non-Profits) and Hacking for Local (Oakland) at U.C. Berkeley, and Hacking for Oceans at both Scripps and UC Santa Cruz. Hacking for Homeland Security launched last year at the Colorado School of Mines and Carnegie Mellon University. A version for NASA is coming up next.

And to help businesses recover from the pandemic, the teaching team taught a series of Hacking For Recovery classes last summer.

Our Hacking for Defense team continues to look for opportunities to adapt and apply the course methodology for broader impact and public good. Project Agrippa, for example, piloted a new “Hacking for Strategy” initiative inspired by their experience in Stanford’s “Technology, Innovation and Modern War” class that Raj Shah, Joe Felter and I taught last fall. This all-star team of 4 undergraduates and a JD/MBA developed new ways to provide logistical support to maritime forces in the Indo-Pacific region. Their recommendations drew on insights gleaned from over 242! interviews (a national H4D class record.) After in-person briefings to Marine Corps and Navy commanders and staff across major commands from California to Hawaii, they received interest in establishing a future collaboration, validating our hypothesis that Hacking for Strategy would be a welcome addition to our course offerings. Its premise is that keeping America safe not only requires us maintaining a technological edge but also using these cutting edge technologies to develop new operational concepts and strategies. Stay tuned.

Team AngelComms – Rescuing Downed Pilots

If you can’t see the AngelComms 2-minute video, click here

If you can’t see the AngelComms slides, click here

Team Salus – Patching Operational Systems to Keep them Secure

If you can’t see the Salus 2-minute video, click here

If you can’t see the Salus slides, click here

Team Mongoose – Tracking Hackers Disposable Infrastructure

If you can’t see the Mongoose 2-minute video, click here

If you can’t see the Mongoose slides, click here

Team Engage – Preparing Aviators to Make Critical High Stakes Decisions

If you can’t see the Engage slides, click here

What’s Next For These Teams?
When they graduate, the Stanford students on these teams have the pick of jobs in startups, companies, and consulting firms. Most are applying to H4X Labs, an accelerator focused on building dual-use companies that sell to both the government and commercial firms. Many will continue to work with their problem sponsor. Several will join the new Stanford Gordian Knot Center which is focused on the intersection of policy, operational concepts, and technology.

In our post class survey 86% of the students said that the class had impact on their immediate next steps in their career. Over 75% said it changed their opinion of working with the Department of Defense and other USG organizations.

It Takes A Village
While I authored this blog post, this class is a team project. The secret sauce of the success of Hacking for Defense at Stanford is the extraordinary group of dedicated volunteers supporting our students in so many critical ways.

The teaching team consisted of myself and:

Pete Newell, retired Army Colonel and ex Director of the Army’s Rapid Equipping Force, now CEO of BMNT.
Joe Felter, retired Army Colonel; and former deputy assistant secretary of defense for South Asia, Southeast Asia, and Oceania; and William J. Perry Fellow at Stanford’s Center for International Security and Cooperation.
Steve Weinstein, 30-year veteran of Silicon Valley technology companies and Hollywood media companies. Steve was CEO of MovieLabs, the joint R&D lab of all the major motion picture studios. He runs H4X Labs.
Tom Bedecarré, the founder and CEO of AKQA, the leading digital advertising agency.
Jeff Decker, a Stanford researcher focusing on dual-use research. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Our teaching assistants this year were Nick Mirda, Sally Eagen, Joel Johnson, past graduates of Hacking for Defense, and Valeria Rincon. A special thanks to the National Security Innovation Network (NSIN) and Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country, as well as Lockheed Martin and Northrop Grumman. And our course advisor, Tom Byers, Professor of Engineering and Faculty Director, STVP.

Thanks to Mike Brown, Director of the Defense Innovation Unit for giving an extraordinary closing keynote.

We were lucky to get a team of mentors (VCs and entrepreneurs) as well as an extraordinary force of military liaisons from the Hoover Institution’s National Security Affairs Fellows program, Stanford senior military fellowship program and other accomplished military affiliated volunteers. This diverse group of experienced experts selflessly volunteer their time to help coach the teams. Thanks to Todd Basche, Rafi Holtzman, Kevin Ray, Craig Seidel, Katie Tobin, Jennifer Quarrie, Jason Chen, Matt Fante, Richard Tippitt, Rich Lawson, Commander Jack Sounders, Mike Hoeschele, Donnie Hasseltine, Steve Skipper, LTC Jim Wiese, Col. Denny Davis, Commander Jeff Vanak, Marco Romani, Rachel Costello, LtCol Kenny Del Mazo, Don Peppers, Mark Wilson and LTC Ed Cuevas

And of course a big shout-out to our problem sponsors across the DoD and IC: MSgt Ashley McCarthy, Jason Stack, Col Sean Heidgerken, LTC Richard Barnes, George Huber, Neal Ziring, Shane Williams, Anthony Ries, Russell Hoffing, Javier Garcia, Matt Correa, Shawn Walsh, and Claudia Quigley.

Filed under: Hacking For Defense | Leave a comment »

Hacking for Allies

Posted on February 19, 2021 by steve blank

During the Cold War U.S. diplomatic and military alliances existed to defend freedom around the world. Today, these alliances are being reshaped to respond to Russian threats to the Baltics and Eastern Europe and to China’s economic, military, and technological influence worldwide.

Hacking for Allies
The U.S. Department of Defense works with our allies to expand their industrial base. We benefit because it helps the North Atlantic Treaty Organization (NATO) standardize on equipment and our allies’ industrial capacity, capability and workforce can complement those of the United States. Allied countries benefit under the Global Capabilities Program which offers allies opportunities to partner on research and development, with the goal to build prototypes and eventually co-produce systems.

The goal of Hacking for Allies, (which will launch a second cohort next week,) is to connect dual-use startups (those that sell to companies and government agencies) in allied nations to the U.S. defense ecosystem.

Startup ecosystems in many of the smaller NATO countries don’t enjoy the long-established expertise or funding opportunities we have in Silicon Valley or other innovation clusters. For example, today it takes 7 to 10 years for a company in Norway to sell into the U.S. defense market. To shorten that time, we wanted to teach them the best practices of Hacking for Defense/Lean Startup/I-Corps (customer discovery, MVPs, pivots, business model canvas, etc.) And give them a roadmap for how to play in the U.S. defense market.

Hacking for Allies – Norway Edition
Norway is a founding member of the NATO and they are NATO’s bulwark against Russian incursion in the strategically critical “High North” region. Norway has experienced Russian simulated air attacks on Norwegian targets and jamming of GPS signals that threaten civilian aviation. Last fall, Russia conducted a cyberattack on the Norwegian parliament.

The Norwegian Defence Research Establishment, Innovation Norway, and H4XLabs (BMNT’s early stage tech accelerator) just ran the first Hacking for Allies cohort of Norwegian companies. The teams were guided using the “player-coach” approach. The program conducted weekly deep dives with each company working through their challenges. It combined this with sourcing outside experts for all cohort topics of interest. These topics included: Raising funds as a European company, what it takes to work with the DoD, customer discovery for adjacent markets, and more.

85 startups applied for this first cohort. They were down-selected to a few promising teams. Some of the teams included:

Alva Industries – making 3D printed electric motor stators 20% more efficient. That means more battery life and/or power for unmanned aerial vehicles.

Excitus: a medical device to clear blocked airways in the battlefield. Their device replaces existing suction pumps with the equivalent of a handheld vacuum cleaner with a sterile disposable cup.

Fieldmade: An additive manufacturing microfactory with a library of certified printable 3D parts, which radically reduces parts inventory.

Ubiq Aerospace: started as de-icing for drones but potentially pivoting to sensor data fusion.

The teams launched out of the program talked to tons of people in the U.S. they never would have connected with (“it would have taken us years to make these connections”), made pivots, built new product suites and capabilities around their core services – all of which made them attractive to wider markets — and raised additional funding.

Now a new cohort of the program is getting under way. Innovation offices from NATO countries and other allies who want to teach their dual-use startups how to work with the U.S. government should attend the Hacking for Allies webinar February 23^rd at 8 am Pacific, 11 am Eastern.

Register here.

Filed under: Hacking For Defense, National Security | Leave a comment »

Teaching Lean Innovation in the Pandemic

Posted on August 11, 2020 by steve blank

Remote education in the pandemic has been hard for everyone. Hard for students having to deal with a variety of remote instructional methods. Hard for parents with K through 12 students at home trying to keep up with remote learning, and hard for instructors trying to master new barely functional tools and technology while trying to keep students engaged gazing at them through Hollywood Squares-style boxes.

A subsegment of those instructors – those trying to teach Lean LaunchPad, whether in I-Corps, or Hacking for Defense – have an additional burden of figuring out how to teach a class that depends on students getting out of the building and talking to 10 to 15 customers a week.

400 Lean Educators instructors gathered online for a three-hour session to share what we’ve learned about teaching classes remotely. We got insights from each other about tools, tips, techniques and best practices.

Here’s what we learned.

—

When I designed the Lean LaunchPad/I-Corps/Hacking for Defense class, my goal was to replace the traditional method of teaching case studies and instead immerse the students in a hands-on experiential process that modeled what entrepreneurs really did. It would be guided week-to-week by using the Business Model Canvas and testing hypotheses by getting out of the building and building Minimum Viable Products (MVPs). After trial and error, we found that having eight teams presenting in a three-hour block was the maximum without exhausting the instructors and the students. That format, unwieldy as it is, remained the standard for a decade. Over time we started experimenting with breaking up the three-hour block with breakout rooms and other activities so not all students needed to sit through all the presentations.

When the pandemic forced us to shift to online teaching, that experimentation turned into a necessity. Three hours staring at a Zoom screen while listening to team after team present is just untenable and unwatchable. Customer discovery is doable remotely but different. Teams are scattered across the world. And the instructor overhead of managing all this is probably 3X what it is in person.

While we were making changes to our classes at Stanford, Jerry Engel was smart enough to point out that hundreds of instructors in every university were having the same problems in adapting the class to the pandemic. He suggested that as follow-up to our Lean Innovation Educators Summit here in Silicon Valley last December, we should create a mid-year on-line Summit so we could all get together and share what we learned and how we’re adapting. And so it began.

In July, 400 Educators from over 200 universities in 22 countries gathered online for a Lean Innovation Educators Summit to share best practices.

We began the summit with five of us sharing our experience of how we dealt with the online challenges of:

teaching an existing Lean program i.e. Hacking For Defense
creating and teaching new Lean classes i.e. Hacking For Oceans and Hacking For Recovery
creating programs for Diversity, Equity, & Inclusion e. GEM I4 / Black Learners Matter
effectively teaching Business Model Design during COVID-19

If you can’t see the presentation slides click here

But the core of the summit was gathering the collective wisdom and experience of the 400 attendees as we split into 22 breakout rooms. The one-hour discussion in each of the rooms covered:

What are your biggest challenges under COVID-19?
How is this challenge different now than during “in-person” learning?
What solutions have you tried?
What was most effective?

The output of the breakout sessions provided a firehose of data, a ton of useful suggestions, teaching tips and tools.I’ve summarized the collective notes from the breakout session.

Customer Discovery and Minimal Viable Products
The consensus was, yes you can “get out of the building” when you physically can’t. And it’s almost good enough.

Discovery can be done via Zoom or similar remote platforms and in some ways is more effective – see here
During Covid most people no longer have gatekeepers around them
- Sending lots of cold emails works (at least in COVID times)
You could find the best mentors and the best sponsor for a given project
Building and demonstrating hardware MVPs is a challenge
- One solution is to send a design file to a fab lab to be printed
- If you would normally have your potential customer hold, feel or use the product, make sure you video a demo someone doing that
For software MVPs create video demo snippets of less <1 minute to illustrate each of your features
It’s critical to offer a “How to do customer discovery remotely” and “how to build remote MVPs” workshop

Class Structure
3-hour long classes are challenging in person and require a redesign to be taught online.

Keep students engaged by having no more than four teams in a presentation room at one time
- Have other teams in breakout rooms and/or with other instructors
  - Breakout rooms must be well thought out and organized
  - They should have a task and a deliverable
Break up lectures so that they are no longer then 15 minutes
- Intersperse them with interactive exercises (Alex Osterwalder is a genius here, providing great suggestions for keeping students engaged)
- Work on an exercise in class and then talk more to it in office hours
- Avoid canned video lectures
Be more prescriptive on “what is required” in the team presentations
What’s the goal for the class?
- Do you want them to test the entire Canvas or …
- Do you want them to work on product market fit?
  - Teams will naturally gravitate to work on product/market fit
Vary the voices at the “front” of the room
Guest speakers – previously extraneous but needed now to break up the monotony
- But if you use guests have the student’s whiteboard summaries of what they learned
- And have the guests be relevant to the business model topic of the week
Understand that while students attend your class they actually pay attention to their mentors
- Recruit mentors whose first passion are helping students, not recruiting or investing in them
- Ensure that you train and onboard mentors to the syllabus
- Have the mentors sit in on the office hours and classroom
Invite lurkers, advisors, and others “invited” to show up and chime in
Be prepared for the intensity of the preparation required as compared to pre-COVID times
- Recruiting students and forming teams is especially hard remotely
- Double or triple down on the email and other outreach
- Hold on-line info sessions and mixers

Teaching Assistant
Having a Teaching Assistant is critical

If your school won’t pay for one, get some unofficial “co-instructors”
- They don’t have to be a teacher–use an admin or a student intern
They are critical to managing the admin side of marketing, recruiting, team formation, communications and overall support for the teaching team.
Team formation requires TA heavy lifting of emails/team mixers/team
- as well as match-making by TA’s and instructors
During class TA’S need to be focused on chat, breakout room and presentation logistics
Don’t assume (or let your TA assume) that prior practices will work in a virtual environment.
Be prepared to try different approaches to keep class moving and engaged
Pre-class write up a “How to TA in a Remote Class” handbook
- Go through it with your TA’s before class
Use security in advance; avoid open entry (Zoom Bombing)

Student Engagement
Zoom fatigue came up in almost every breakout session. Some of the solutions included:

Play music as students arrive and leave
Recognize that some may be in different time zones – take a poll in the first class session
Start each class session with an activity
- Summarize key insights/lessons learned from their office hours and customer discovery
- For those using Zoom – use the Whiteboard feature for these summaries
  - Other platforms for remote collaboration include Miro, Mural, Zoom, Discord, LaunchPad Central and Innovation Within etc.
    - A list of remote teaching tools suggested in the breakout sessions are here
Have students turn on their camera on to ensure the class they’re engaged
- And have their microphone off, their full name visible, and a virtual background with their team ID
Create deeper connection with the students
- ask them to anonymously submit a statement or two about what they wish you knew about them
- ask the students to bring something to class that tells us something about them
  - have them bring it to the breakout rooms to share with their teammates and others
Randomly cold call
- Don’t be afraid to call out students by name, as Zoom format makes raising hand or asking a question more awkward
- Ask their advice on what someone else just presented or what they learned from the other team
- After doing this a couple of times, everyone will become active (so not to get called on)
Require additional student feedback on chat – critical to keeping engagement high
- Focus on quality of feedback over just quantity.
- Have the students and mentors use chat during team presentations to share contacts, insights
Dial back the radical candor– take the edge off as the students are already stressed
Offer longer office hours for teams

(All the breakout session slides are here.)

Summary
When the National Science Foundation stopped holding their annual conference of I-Corps instructors, it offered us the opportunity to embrace a larger community beyond the NSF – now to include the Hacking for Defense, NSIN, and Lean LaunchPad educators.

When we decided to hold the online summit, we had three hypotheses:

Educators would not only want to attend, but to volunteer and help and learn from each other – validated
Instructors would care most about effective communication with students (not tools, or frameworks but quality of the engagement with students) – validated
Our educator community valued ongoing, recurring opportunities to collaborate and open source ideas and tools – validated

The Common Mission Project is coordinating the group’s efforts to create an open forum where these instructors can share best practices and to curate the best content and solutions.

A big thanks to Jerry Engel of U.C. Berkeley, the dean of this program. And thanks to the Common Mission Project which provided all the seamless logistical support, and every one of the breakout room leaders: Tom Bedecarré – Stanford University, John Blaho – City College of New York, Philip Bouchard – TrustedPeer, Dave Chapman – University College London, James Chung – George Washington University, Bob Dorf – Columbia University, Jeff Epstein – Stanford University, Paul Fox – LaSalle University Barcelona, Ali Hawks – Common Mission Project UK, Jim Hornthal – U.C. Berkeley, Victoria Larke – University of Toronto, Radhika Malpani – Google, Michael Marasco – Northwestern University, Stephanie Marrus – University of California, San Francisco, Pete Newell – BMNT/ Common Mission Project US, Thomas O’Neal – University of Central Florida, Alexander Osterwalder – Strategyzer, Kim Polese – U.C. Berkeley, Jeff Reid – Georgetown University, Sid Saleh – Colorado School of Mines, Chris Taylor – Georgetown University, Grant Warner – Howard University, Todd Warren – Northwestern University, Phil Weilerstein – VentureWell, Steve Weinstein – Stanford University, Naeem Zafar – U.C. Berkeley, and the 400 of you who attended.

Looking forward to our next Educator Summit, December 16^th online.

The video of the entire summit can be seen here

Filed under: Hacking For Defense, Lean LaunchPad, NSF (National Science Foundation), Teaching | 7 Comments »

Hacking for Defense @ Stanford 2020 Lesson Learned Presentations

Posted on June 12, 2020 by steve blank

We just finished our 5th annual Hacking for Defense class at Stanford.

What a year.

At the end of the quarter each of the eight teams give a final “Lessons Learned” presentation. Unlike traditional demo days or Shark Tanks which are, “here’s how smart I am, please give me money,” a Lessons Learned presentation tells the teams’ stories of a 10-week journey of hard-won learning and discovery. For all the teams in a normal year it’s a roller coaster narrative of what happens when you discover that everything you thought you knew on day one was wrong and how they eventually got it right.

But this year? This year was something different. 32 students were scattered across the globe and given a seemingly impossible assignment- they had 10 weeks to understand and then solve a real Dept of Defense problem – by interviewing 100 beneficiaries, stakeholders, requirements writers, et al while simultaneously building a series of minimal viable products – all while never leaving their room.

Watching each of the teams present I was left with wonder and awe about what they accomplished

Here’s how they did it and what they delivered.

Our keynote speaker for this last class was ex Secretary of Defense General Jim Mattis who gave an inspiring talk about service to the nation.

Part 1

Part 2

Part 3

Part 4

If you can’t see the four videos of General Mattis click here for the entire talk.

How Do You Get Out of the Building When You Can’t Get Out of the Building?
This year the teams had to overcome two extraordinary pandemic-created hurdles. First, most of the students were sequestering off campus and were scattered across 24 time zones. Each team of four students who would have spent the quarter working collaboratively in-person, instead were never once physically in the same room or location. Second, this class – which is built on the idea of interviewing customers/beneficiaries and stakeholders in person – now had to do all their customer discovery via a computer screen. At first this seemed to be a fatal stake through the heart of the class. How on earth would customer interviews work via video?

But we were in for two surprises. First, the students rose to the occasion, and in spite of time and physical distance, every one of them came together and acted as a unified team. Second, doing customer discovery via video actually increased the number of interviews the students were able to do each week. The eight teams spoke to over 945 beneficiaries, stakeholders, requirements writers, program managers, warfighters, legal, security, customers, etc.

A good number of the people the students needed to talk to were sheltering at home, and they weren’t surrounded by gatekeepers. While the students missed the context of standing on a navy ship or visiting a drone control station, or watching someone try their app or hardware, the teaching teams’ assessment was that remote interviews were more than an adequate substitute.

We Changed The Class Format
Going remotely we made two major changes to the class. Previously, each of the eight teams presented a weekly ten-minute summary of; here’s what we thought, here’s what we did, here’s what we found, here’s what we’re going to do next week. While we kept that cadence it was too exhausting for all the other teams to stare at their screen watching every other team present. So we split the class in half – four teams went into Zoom breakout rooms where they met with a peer-team to discuss common issues. The remaining four were in the main Zoom classroom; one presenting as three watched and listened to the instructor comments, critiques and suggestions. We rotated the teams through the main room and breakout sessions.

The second change was the addition of guest speakers. In the past, I viewed guest speakers as time filler/entertainment that detracted from the limited in-class time we needed to listen to and coach our students. But this year we realized that our students had been staring at their screens all day and it was going to fry their heads. They deserved some entertainment/distraction. But in true Hacking for Defense practice we were going to deliver it in the form of edification and inspiration. Joe Felter and I got out our rolodex’s and invited ten distinguished guest speakers. Their talks to this year’s Hacking for Defense class can be seen here.

Lessons Learned Presentation Format
Each of the eight teams presented a 2-minute video to provide context about their problem. This was followed by an 8-minute slide presentation describing their customer discovery journey over the 10-weeks. All the teams used the Mission Model Canvas, (videos here) Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

By the end the class all of the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

All the presentations are worth a watch.

Team Omniscient – An Unclassified Imaging Analyst Workbench

If you can’t see the Omniscient 2-minute video click here

If you can’t see the video of the Omniscient team presenting click here

If you can’t see the Omniscient slides click here

Mission-Driven Entrepreneurship
This class is part of a bigger idea – Mission-Driven Entrepreneurship. Instead of students or faculty coming in with their own ideas — we now have them working on societal problems, whether they’re problems for the State Department or the Department of Defense, or non-profits/NGOs, or for the City of Oakland or for energy or the environment, or for anything they’re passionate about. And the trick is we use the same Lean LaunchPad / I-Corps curriculum — and kept the same class structure – experiential, hands-on, driven this time by a mission-model not a business model. (The National Science Foundation, National Security Agency and the Common Mission Project have helped promote the expansion of the methodology worldwide.)

Mission-driven entrepreneurship is the answer to students who say, “I want to give back. I want to make my community, country or world a better place, while solving some of the toughest problems.”

Team Protocol One – Ensuring JTAC to Pilot Communication

If you can’t see the Protocol One 2-minute video click here

If you can’t see the video of the Protocol One team presenting click here

If you can’t see the Protocol One slides click here

It Started with an Idea
Hacking for Defense has its origins in the Lean LaunchPad class I first taught at Stanford in 2011. I observed that teaching case studies and/or how to write a business plan as a capstone entrepreneurship class didn’t match the hands-on chaos of a startup. And that there was no entrepreneurship class that combined experiential learning with the Lean methodology. Our goal was to teach both theory and practice.

The same year we started the class, it was adopted by the National Science Foundation to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 9 regional locations supporting 98 universities and has trained over 1500 science teams. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

Team SeaWatch – Maritime Security in the South China Sea

If you can’t see the SeaWatch 2-minute video click here

If you can’t see the video of the SeaWatch team presenting click here

If you can’t see the SeaWatch slides click here

Origins of Hacking For Defense
In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford we observed that students in our research universities had little connection to the problems their government was trying to solve or the larger issues civil society were grappling with. Wondering how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. That year we launched both Hacking for Defense and Hacking for Diplomacy (with Professor Jeremy Weinstein and the State Department) at Stanford.

Team TimeFlies – Automating Air Force aircrew scheduling

If you can’t see the TimeFlies 2-minute video click here

If you can’t see the video of the TimeFlies team presenting click here

If you can’t see the TimeFlies slides click here

Goals for the Hacking for Defense Class
Our primary goal was to teach students Lean Innovation while they engaged in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community or other government agencies.

Next, we wanted the students to learn about the nation’s threats and security challenges while working with innovators inside the DoD and Intelligence Community. And while doing so, teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there is a methodology that can help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Finally, we wanted to familiarize students about the military as a profession, its expertise, and its proper role in society. And conversely show our sponsors in the Department of Defense and Intelligence community that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions to real-world problems.

Team AV Combinator – Autonomous Vehicle Safety Standards

If you can’t see the AV Combinator 2-minute video click here

If you can’t see the video of the AV Combinator team presenting click here

If you can’t see the AV Combinator slides click here

Mission-driven in 35 Universities
What started as a class is now a movement.

Hacking for Defense is offered in over 35 universities, but quickly following, Orin Herskowitz started Hacking for Energy at Columbia, Steve Weinstein started Hacking for Impact (Non-Profits) and Hacking for Local (Oakland) at Berkeley. Hacking for Conservation and Development at Duke followed. Steve Weinstein subsequently spun out versions of Hacking for Oceans at both Scripps and UC Santa Cruz.

And to help businesses recover from the pandemic the teaching team will be offering a Hacking For Recovery class this summer.

Team Anthro Energy – next generation lightweight flexible batteries

If you can’t see the Anthro Energy 2-minute video click here

If you can’t see the video of the Anthro Energy team presenting click here

If you can’t see the Anthro Energy slides click here

Team Helmsman – Navigating in GPS denied areas

If you can’t see the Helmsman 2-minute video click her

If you can’t see the video of the Helmsman team presenting click here

If you can’t see the Helmsman slides click here

Team Election Watch – Open Source Tool to Track Political Influence Campaigns

If you can’t see the Election Watch 2-minute video click here

If you can’t see the Election Watch slides click here

What’s Next for These Teams?
When they graduate, the Stanford students on these teams have the pick of jobs in startups, companies and consulting firms. Recognizing the ability of these teams to produce real results, 38 members of the venture and private equity community dialed in to these presentations. Every year they fund several teams as they launch companies. This year a record 6 of the 8 teams (Anthro Energy, AV Combinator, Election Watch, Helmsman, Omniscient and Seawatch) have decided to continue with their projects to build them into dual-use companies – selling both to the Dept of Defense and commercial businesses.) Most are applying to H4X Labs, an accelerator focused on building dual-use companies.

Student Feedback
While Stanford does a formal survey of student reviews of the class, this year we wanted more granular data on how remote learning affected their class experience.

While we had heard anecdotal stories about how the class affected the students perceptions of the Department of Defense we now had first hand evidence. The same was true for the life-changing experience of actually doing customer discovery with 100 people. The results reinforced our belief that the class, scaling across the county was helping to bridge the civilian/military divide while teaching students a set of skills that will last a lifetime.

Student feedback on the class is here

It Takes a Village
While I authored this blog post, this class is a team project. The teaching team consisted of myself and:

Pete Newell retired Army Colonel and ex Director of the Army’s Rapid Equipping Force and CEO of BMNT.
Joe Felter retired Army Colonel and former deputy assistant secretary of defense for South Asia, Southeast Asia, and Oceania
Steve Weinstein 30-year veteran of Silicon Valley technology companies and Hollywood media companies. Steve was CEO of MovieLabs the joint R&D lab of all the major motion picture studios. He runs H4X Labs.
Tom Bedecarré the founder and CEO of AKQA, the leading digital advertising agency.
Jeff Decker a Stanford social science researcher. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Our teaching assistants this year were Nate Simon and Sam Lisbonne both past graduates of Hacking for Defense, and Valeria Rincon. A special thanks to the National Security Innovation Network (NSIN) and Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country as well Lockheed Martin and Northrop Grumman. And our course advisor – Tom Byers, Professor of Engineering and Faculty Director, STVP.

We were lucky to get a team of mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams. Thanks to Todd Basche, Teresa Briggs, Rachel Costello, Gus Hernandez, Rafi Holtzman, Katie Tobin, Robert Locke, Kevin Ray, Eric Schrader, Mark Rosekind, Don Peppers, Nini Moorhead, Daniel Bardenstein.

We were privileged to have the support of an extraordinary all volunteer team of professional senior military officers representing all branches of service attending fellowship programs at Stanford’s Hoover Institution, and Center for International Security and Cooperation (CISAC) and Asia Pacific Research Center (APARC) at the Freeman Spogli Institute (FSI) as well as from the Defense Innovation Unit. These included COL Smith-Heys, COL Liebreich and LTC Campbell – Army, CAPT Sharman, CAPT Romani – Navy, CDR Malzone – Coast Guard, LT COL Lawson, LT COL Hasseltine and LT COL Cook – USMC, LT COL Waters and LT COL Tuzel – Air Force and Mr. Smyth -State Dept.

And of course a big shout-out to our problem sponsors. At In-Q-Tel – Mark Breier/Zig Hampel, U.S. Army – LTC Leo Liebreich, U.S. Air Force – LTC Doug Snead/ MAJ Mike Rose, Joint Artificial Intelligence Center – Joe Murray/MAJ Dan Tadross, Special Operations Command Pacific – MAJ Paul Morton, United States Africa Command – Matt Moore, and from the Office of Chairman of the Joint Chief of Staff – MAJ Jeff Budis.

Be sure to check out the other Hacking For Defense classes in universities in the U.S. and the U.K.

Thanks to everyone!

Filed under: Hacking For Defense | 1 Comment »

Mission Model Canvas – the Videos

Posted on September 24, 2019 by steve blank

In 2016 Pete Newell, Alexander Osterwalder and I developed the Mission Model Canvas for our Hacking for Defense Class. We’ve now created a series of videos that explain how this variant of the Business Model Canvas works – 11 videos totaling 17 minutes.

Thanks to BMNT and the National Security Innovation Network for support of this project.

—

When Pete Newell, Joe Felter and I built the Hacking for Defense class we modeled the syllabus after my earlier Lean LaunchPad and NSF I-Corps classes. Both classes used Alexander Osterwalder’s Business Model Canvas to frame the hypotheses to be tested.

If you can’t see the business model canvas video click here

However, using the business model canvas inside the Dept of Defense was problematical. Teams there pointed out that the standard business model canvas didn’t fit their problem sets. For example, in the Dept of Defense you don’t measure progress by measuring revenue. Instead you mobilize resources and a budget to solve a particular problem and create value for a set of beneficiaries (customers, support organizations, warfighters, Congress, the country, etc.)

Therefore, the business model canvas box labeled Revenue Streams doesn’t make sense. The defense and intelligence community are mission-driven organizations so there is no revenue to measure. The first step in building a canvas that worked for these organizations was to change the Revenue Stream box to one that would provide a way to measure success.

We called this alternative- Mission Achievement/Success.

Now the Mission Model Canvas just needed four more tweaks.

Customer Segments was changed to Beneficiaries
Cost Structure was changed to Mission Cost/Budget
Channel was changed to Deployment
Customer Relationships was changed to Buy-in/Support

Read the full blog post on the development and use of the Mission Model Canvas here.

And watch the Mission Model Canvas videos below.

If you can’t see the Introduction to the Mission Model Canvas video click here

If you can’t see the Beneficiaries and Stakeholders video click here

If you can’t see the Value Proposition video click here

If you can’t see the Buy-In video click here

If you can’t see the Deployment video click here

If you can’t see the Mission Achievement video click here

If you can’t see the Key Activities video click here

If you can’t see the Key Resources video click here

If you can’t see the Key Partners video click here

If you can’t see the Mission Budget video click here

If you can’t see the Key Concepts video click here

You can find the entire video series collected here

Filed under: Hacking For Defense, Teaching | 3 Comments »

Hacking for Defense @ Stanford 2019

Posted on June 7, 2019 by steve blank

We just finished our 4th annual Hacking for Defense class at Stanford. At the end of each class we have each team give a Lessons Learned presentation. Unlike traditional demo days or Shark Tanks which are “here’s how smart I am, please give me money,” a Lessons Learned presentation tells a story of a journey of hard-won learning and discovery. For all the teams it’s a roller coaster narrative of what happens when you discover that everything you thought you knew was wrong and how they eventually got it right.

Watching each of the teams present I was left with wonder and awe about what they accomplished in 10 weeks.

The eight teams spoke to over 820 beneficiaries, stakeholders, requirements writers, program managers, warfighters, legal, security, customers, etc.
By the end the class all of the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

Our keynote speaker was Palmer Luckey, founder of Oculus and the designer of the Oculus Rift. He’s now the CEO/founder of the AI-focussed defense contractor Anduril Industries.

If you can’t see the video of Palmer Luckey click here

Presentation Format
Each of the eight teams presented a 2-minute video to provide context about their problem.

Followed by an 8-minute slide presentation follow their customer discovery journey over the 10-weeks. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

All the presentations are worth a watch.

Team: Panacea

If you can’t see the Panacea 2-minute video click here

If you can’t see the video of the Panacea team presenting click here

If you can’t see the Panacea slides click here

Team: Learn2Win

If you can’t see the Learn2Win 2-minute video click here

If you can’t see the video of the Leanr2Win team presenting click here

If you can’t see the Leanr2Win slides click here

The same year we started the class, it was adopted by the National Science Foundation to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 9 regional locations supporting 98 universities and has trained over 1500 science teams. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

Team: Embargo NK

If you can’t see the EmbargoNK 2-minute video click here

If you can’t see the video of the EmbargoNK team presenting click here

If you can’t see the EmbargoNK slides click here

Origins of Hacking For Defense
In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford we observed that students in our research universities had little connection to the problems their government as well as the larger issues civil society was grappling with. Wondering how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. That year Hacking for Defense and Hacking for Diplomacy (with Professor Jeremy Weinstein) with the State Department were both launched at Stanford.

Team: Common Ground

If you can’t see the Common Ground 2-minute video click here

If you can’t see the video of the Common Ground team presenting click here

If you can’t see the Common Ground slides click here

Next, we wanted the students to learn about the nation’s threats and security challenges while working with innovators inside the DoD and Intelligence Community. While doing so, also teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there is a methodology that can help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Team: Deepfakes

If you can’t see the Deepfakes 2-minute video click here

If you can’t see the video of the Deepfakes team presenting click here

If you can’t see the Deepfakes slides click here

Mission-driven in 30 Universities
Hacking for Defense is offered in over 25 universities, but quickly following, Orin Herskowitz started Hacking for Energy at Columbia, Steve Weinstein started Hacking for Impact (Non-Profits) and Hacking for Local (Oakland) at Berkeley and will be starting Hacking for Oceans at Scripps. Hacking for Conservation and Development at Duke followed.

Team: IntelliSense

If you can’t see the Intellisense 2-minute video click here

If you can’t see the video of the Intellisense team presenting click here

If you can’t see the Intellisense slides click here

Team: Gutenberg

If you can’t see the Gutenberg 2-minute video click here

If you can’t see the video of the Gutenberg team presenting click here

If you can’t see the Gutenberg slides click here

Team: PredictiMx

If you can’t see the PredictiMx 2-minute video click here

If you can’t see the video of the PredictiMX team presenting click here

If you can’t see the PredictiMx slides click here

It Takes a Village
While I authored this blog post, this classes is a team project. The teaching team consisted of:

Pete Newell is a retired Army Colonel and CEO of BMNT.
Steve Weinstein a 30-year veteran of Silicon Valley technology companies and Hollywood media companies. Steve was CEO of MovieLabs the joint R&D lab of all the major motion picture studios.
Tom Bedecarré was the founder and CEO of AKQA, the leading digital advertising agency. Four decades as part of the most successful advertising agencies in the world.
Jeff Decker is a social science researcher at Stanford. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Our teaching assistants Nate Simon, Aidan Daniel McCarty, Mackenzie Burnett and Diego Cervantes. A special thanks to the National Security Innovation Network (NSIN) and Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country. And our course advisor – Tom Byers, Professor of Engineering and Faculty Director, STVP.

We were lucky to get a team of mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams. Thanks to Kevin Ray, Lisa Wallace, Rafi Holtzman, Craig Seidel, Todd Basche, Don Peppers, Robert Locke, and Mark Clapper.

We were privileged to have the support of an extraordinary all volunteer team of professional senior military officers representing all branches of service attending fellowship programs at Stanford’s Hoover Institution, and Center for International Security and Cooperation (CISAC) and Asia Pacific Research Center (APARC) at the Freeman Spogli Institute (FSI) as well as from the Defense Innovation Unit. These included:Tim Mungie, Tim Murphy, Matt Kent, Todd Mahar, Donnie Hasseltine, Jay Garcia, Kevin Childs.

And of course a big shout-out to our sponsors. At United Nations Command Security Battalion – CPT Justin Bingham, Air Force Air Combat Command – Mr. Steven Niewiarowski, Office of the Secretary of Defense Asian & Pacific Security Affairs – Chief of Staff Julie Sheetz, U.S. Coast Guard – Security Specialist Asad Hussain, IQT – Vishal Sandesara, Veterans Adminstration – Kristopher “Kit” Teague, Chief Operating Officer, IARPA – John Beieler, DNI – Dean Souleles

Thanks!

Filed under: Customer Development, Hacking For Defense, Teaching | 3 Comments »

Hacking for Defense @ Stanford 2018 – wonder and awe

Posted on June 8, 2018 by steve blank

We just finished our 3rd annual Hacking for Defense class at Stanford. Six teams presented their Lessons Learned presentations.

Watching them I was left with wonder and awe about what they accomplished in 10 weeks.

Six teams spoke to over 600 beneficiaries, stakeholders, requirements writers, program managers, warfighters, legal, security, customers, etc.
By the end the class all of the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

Each of the six teams presented a 2-minute video to provide context about their problem and then gave an 8-minute presentation of their Lessons Learned over the 10-weeks. Each of their slide presentations follow their customer discovery journey. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

The teams presented in front of several hundred people in person and online.

Team: TrackID

If you can’t see the TrackID video click here

If you can’t see the TrackID slides click here

Team: Polaris

If you can’t see the Polaris video click here

If you can’t see the Polaris slides click here

Team: Acquiforce

If you can’t see the Acquiforce video click here

If you can’t see the Acquiforce slides click here

Team: Intelgrids

If you can’t see the Intelgrids video click here

If you can’t see the Intelgrids slides click here

Team: See++

If you can’t see the See++ video click here

If you can’t see the See++ slides click here

Team: Theia

If you can’t see the Theia video click here

If you can’t see the Theia slides click here

Video of the teams live presentation are here. Worth your time to watch.

The Class
Our mantra to the students was that we wanted them to learn about “Deployment not Demos.” Our observation is that the DOD has more technology demos than they need, but often lack deep problem understanding. Our goal was to have the students first deeply understand their sponsors problem – before they started building solutions. As you can imagine with a roomful of technologists this was tough. Further we wanted the students to understand all parts of the mission model canvas, not just the beneficiaries and the value proposition. We wanted them to learn what it takes to get their product/service deployed to the field, not give yet another demo to a general. This meant that the minimal viable products the students built were focused on maximizing their learning of what to build, not just building prototypes.

(Our sponsors did remind us, that at times getting a solution deployed meant that someone did have to see a demo!)

Note: The Hacking for Defense class was designed as “fundamental research” to be shared broadly and the results are not subject to restriction for proprietary or national security reasons. In the 10 weeks the students have, Hacking for Defense hardware and software prototypes don’t advance beyond a Technology Readiness Level 4 and remain outside the scope of US export control regulations and restrictions on foreign national participation.

Origins of the class
Hacking for Defense has its origins in the Lean LaunchPad class I first taught at Stanford in 2011. It was adopted by the National Science Foundation in 2012 to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 81 universities and has trained over 1500 science teams. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford we observed that students in our research universities had little connection to the problems their government as well as the larger issues civil society was grappling with. Wondering how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. Both Hacking for Defense and Hacking for Diplomacy with the State Department were born. Hacking for Energy at Columbia, Hacking for Impact (Non-Profits) at Berkeley and Hacking for Conservation and Development at Duke quickly followed.

The Innovation Insurgency Spreads
Hacking for Defense is now offered at eleven universities in addition to Stanford – Georgetown, University of Pittsburgh, Boise State, UC San Diego, James Madison University, University of Southern Mississippi, University of Southern California and Columbia University. Over the next year it will expand to 22 universities. Hacking for Defense.org a non-profit, was established to train educators and to provide a single point of contact for connecting the DOD/IC sponsor problems to these universities.

We’ve been surprised was how applicable the “Hacking for X…” methodology is for other problems. It’s equally applicable to solving public safety, energy, policy, community and social issues internationally and within our own communities. In the next year we’ll see three new variants of the class:

Hacking for the Environment
Hacking for Oceans
Hacking for Cities

It Takes a Village
While I authored this blog post, this classes is a team project. The teaching team consisted of:

Pete Newell is a retired Army Colonel currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and CEO of BMNT.
Steve Weinstein a 30-year veteran of Silicon Valley technology companies and Hollywood media companies. Steve is CEO of MovieLabs the joint R&D lab of all the major motion picture studios.
Jeff Decker is a social science researcher at Stanford. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Two of our teaching assistants were prior students: Samuel Jackson our lead TA, and Will Papper and Annie Shiel and Paricha Duangtaweesub also assisted.

Special thanks to our course advisors – Tom Byers, Professor of Engineering and Faculty Director, STVP, Arun Majumdar and Sally Benson Co-directors of the Stanford Precourt Energy Institute, and John Mitchell, Stanford Provost of Teaching and Learning.

A special thanks to Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country.

We were lucky to get a team of mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams. Thanks to Tom Bedecarre, Kevin Ray, Daniel Bardenstein, Rafi Holtzman, Craig Seidel, Michael Chai, Lisa Wallace and Dave Gabler.

We were privileged to have the support of an extraordinary all volunteer team of professional senior military officers representing all branches of service attending fellowship programs at Stanford’s Hoover Institution, and Center for International Security and Cooperation (CISAC) and Asia Pacific Research Center (APARC) at the Freeman Spogli Institute (FSI). These included: Colonel Bradley Boyd, Lieutenant Colonel James “Gumbo” Coughlin, Lieutenant Colonel Marcus Ferrara, Lieutenant Colonel Jer “Jay” Garcia, Lieutenant Commander Nick Hill, Commander Michael Nordeen, Commander Rebecca Ore, Commander Michael Schoonover, Colonel Jason “Shrek” Terry and Todd Forsman.

And of course a big shout-out to our sponsors. At SOCOM, Matt Leland and Angel Zajkowski, at MITRE, Suresh Damodaran, at NAVFAC, Ben Wilcox, at the 9th ISR, Ian Eishen, at AFRL, Jeff Palumbo and Mike Rottmayer at the Defense Acquistion University, Shirley Franko and at ERDC Thomas Bozada.

Thanks!

Filed under: Hacking For Defense | Leave a comment »

Hacking for Defense at Columbia University

Posted on February 17, 2018 by steve blank

Over the last year we’ve been rolling out the Hacking for X classes in universities across the U.S. – Hacking for Defense, Hacking for Diplomacy, Hacking for Energy, Hacking for Impact (non-profits), etc.

All are designed to allow students to work on some of the toughest problems the country has while connecting them to a parts of the government they aren’t familiar with. When they leave they have contributed to the country through national service and gained a deeper understanding of our country.

Here is the view from Columbia University.

If you can’t see the video click here.

Filed under: Hacking For Defense, Hacking for Diplomacy, Teaching | Leave a comment »

Innovation at Speed – when you have 2 million employees

Posted on February 13, 2018 by steve blank

Success no longer goes to the country that develops a new fighting technology first, but rather to the one that better integrates it and adapts its way of fighting…Our response will be to prioritize speed of delivery, continuous adaptation, and frequent modular upgrades. We must not accept cumbersome approval chains, wasteful applications of resources in uncompetitive space, or overly risk-averse thinking that impedes change.

If you read these quotes, you’d think they were from a CEO who just took over a company facing disruption from agile startups and a changing environment. And you’d be right. Although in this case the CEO is the Secretary of Defense. And his company has 2 million employees.

In January, Secretary of Defense Mattis released the 2018 National Defense Strategy. This document tells our military – the Department of Defense – what kind of adversaries they should plan to face and how they should plan to use our armed forces. The National Defense Strategy is the military’s “here’s what we’re going to do,” to implement the executive branch’s National Security Strategy. The full version of the National Defense Strategy is classified; but the 10-page unclassified summary of this strategic guidance document for the U.S. Defense Department is worth a read.

Since 9/11 the U.S. military focused on defeating non-nation states (ISIS, al-Qaeda, et al.) The new National Defense Strategy states that we need to prepare for competition between major powers, calling out China and Russia explicitly as adversaries, (with China appearing to be the first.) Secretary Mattis said, “Our competitive advantage has eroded in every domain of warfare.”

While the Defense Strategy recognizes the importance of new technologies e.g. autonomous systems and artificial intelligence – the search is no longer for the holy grail of a technology offset strategy. Instead the focus is on global and rapid maneuver capabilities of smaller, dispersed units to “increase agility, speed, and resiliency .. and deployment … in order to stand ready to fight and win the next conflict.” The goal is to make our military more “lethal, agile, and resilient.”

The man with a lot of fingerprints on this document is the Deputy Secretary of Defense Patrick Shanahan. Shanahan came from Boeing, and his views on innovation make interesting reading.

“The DoD must become less averse to risk” is something you’ve rarely heard in the government. Yet, Shanahan said, “Innovation is messy, if the (defense) department is really going to succeed at innovation, we’re going to have to get comfortable with people making mistakes.”

All of this means that significant changes will be needed in the Department of Defense’s culture and policies. But now the change agents and innovation insurgents who have been fighting to innovate from the bottom up at Office of Naval Research, National Geospatial-Intelligence Agency, Joint Improvised-Threat Defeat Organization, Defense Intelligence Agency, National Security Agency, Defense Innovation Unit Experimental, Intelligence Advanced Research Projects Activity, etc., will have the support all the way to the Secretary of Defense.

The innovation language in this document is pretty mind blowing – particularly the summary on page 10. It’s almost as they’ve been reading the posts Pete Newell and I have written on the Red Queen Problem and the Innovation Pipeline.

This document, combined with the split of Acquisition and Logistics, (the office responsible for buying equipment for the military), from Research and Engineering will enable the DoD to better connect with private industry to get technology integrated into the defense department. The last time the country mobilized private industry at scale was the Cold War.

As you read the excerpts below from the 2018 National Defense Strategy imagine the shockwaves this would send through any large company. This is a pretty unprecedented document for the military.

on page 3
“…Maintaining the Department’s technological advantage will require changes to industry culture, investment sources, and protection across the National Security Innovation Base. “

on page 4
“Defense objectives include: …Continuously deliver performance with affordability and speed as we change Departmental mindset, culture, and management systems;

on page 5
“Foster a competitive mindset. To succeed in the emerging security environment, our Department and Joint Force will have to out-think, out-maneuver, out-partner, and out-innovate revisionist powers, rogue regimes, terrorists, and other threat actors. We will expand the competitive space while … reforming the Department’s business practices for greater performance and affordability.

on page 7
“Cultivate workforce talent. … Cultivating a lethal, agile force requires more than just new technologies and posture changes; …it depends on the ability of our Department to integrate new capabilities, adapt warfighting approaches, and change business practices to achieve mission success. The creativity and talent of the American warfighter is our greatest enduring strength, and one we do not take for granted.
…A modern, agile, information-advantaged Department requires a motivated, diverse, and highly skilled civilian workforce. We will emphasize new skills and complement our current workforce with information experts, data scientists, computer programmers, and basic science researchers and engineers…The Department will also continue to explore streamlined, non-traditional pathways to bring critical skills into service, expanding access to outside expertise, and devising new public-private partnerships to work with small companies, start-ups, and universities.

on page 10
The current bureaucratic approach, centered on exacting thoroughness and minimizing risk above all else, is proving to be increasingly unresponsive. We must transition to a culture of performance where results and accountability matter.

Deliver performance at the speed of relevance. Success no longer goes to the country that develops a new fighting technology first, but rather to the one that better integrates it and adapts its way of fighting…. Current processes are not responsive to need; the Department is over-optimized for exceptional performance at the expense of providing timely decisions, policies, and capabilities to the warfighter. Our response will be to prioritize speed of delivery, continuous adaptation, and frequent modular upgrades. We must not accept cumbersome approval chains, wasteful applications of resources in uncompetitive space, or overly risk-averse thinking that impedes change. Delivering performance means we will shed outdated management practices and structures while integrating insights from business innovation.

Organize for innovation. The Department’s management structure and processes are not written in stone, they are a means to an end–empowering the warfighter with the knowledge, equipment and support systems to fight and win. Department leaders will adapt their organizational structures to best support the Joint Force. If current structures hinder substantial increases in lethality or performance, it is expected that Service Secretaries and Agency heads will consolidate, eliminate, or restructure as needed.

Up to now innovation within the Department of Defense has been the province of a small group of insurgents, each doing heroic efforts. Now innovation at speed has become a nation’s priority. Culture change is hardest in the middle of a large organization. It will be interesting to see how each agency in the Department of Defense (and its contractors) adopts the strategy or whether the bureaucratic middle kills it/waits it out. Time will tell whether it provides real change, but this is a great start.

Filed under: Corporate/Gov't Innovation, Hacking For Defense | 3 Comments »

The Red Queen Problem – Innovation in the DoD and Intelligence Community

Posted on October 17, 2017 by steve blank

“…it takes all the running you can do to keep in the same place. ”
The Red Queen Alice in Wonderland

Innovation, disruption, accelerators, have all become urgent buzzwords in the Department of Defense and Intelligence community. They are a reaction to the “red queen problem” but aren’t actually solving the problem. Here’s why.

In the 20th century our nation faced a single adversary – the Soviet Union. During the Cold War the threat from the Soviets was quantifiable and often predictable. We could specify requirements, budget and acquire weapons based on a known foe. We could design warfighting tactics based on knowing the tactics of our opponent. Our defense department and intelligence community owned proprietary advanced tools and technology. We and our contractors had the best technology domain experts. We could design and manufacture the best systems. We used these tools to keep pace with the Soviet threats and eventually used silicon, semiconductors and stealth to create an offset strategy to leapfrog their military.

That approach doesn’t work anymore. In the 21st century you need a scorecard to keep track of the threats: Russia, China, North Korea, Iran, ISIS in Yemen/Libya/Philippines, Taliban, Al-Qaeda, hackers for hire, etc. Some are strategic peers, some are near peers in specific areas, some are threats as non-state disrupters operating with no rules.

In addition to the proliferation of threats, most of the tools and technologies that were uniquely held by the DoD/IC or only within the reach of large nation states are now commercially available (Cyber, GPS, semiconductors, analytics, centrifuges, drones, genetic engineering, agile and lean methodologies, ubiquitous Internet, crypto and smartphones, etc.). In most industries, manufacturing is no longer a core competence of the U.S.

U.S. agencies that historically owned technology superiority and fielded cutting-edge technologies now find that off-the-shelf solutions may be more advanced than the solutions they are working on, or that adversaries can rapidly create asymmetric responses using these readily available technologies.

The result is that our systems, organizations, headcount and budget – designed for 20th century weapons procurements and warfighting tactics on a predictable basis – can’t scale to meet all these simultaneous and unpredictable challenges. Today, our DoD and national security agencies are running as hard as they can just to stay in place, but our adversaries are continually innovating faster than our traditional systems can respond. They have gotten inside our OODA loop (Observe, Orient, Decide and Act).

We believe that continuous disruption can only be met with a commitment to continuous innovation.

Pete Newell and I have spent a lot of time bringing continuous innovation to government organizations. Newell ran the U.S. Army’s Rapid Equipping Force on the battlefields of Iraq and Afghanistan finding and deploying technology solutions against agile insurgents. He’s spent the last four years in Silicon Valley out of uniform continuing that work. I’ve spent the last six years teaching our country’s scientists how to rapidly turn scientific breakthroughs into deliverable products by creating the curriculum for the National Science Foundation Innovation Corps – now taught in 53 universities. Together Pete, Joe Felter and I created Hacking for Defense, a nationwide program to teach university students how use Lean methodologies to solve defense and national security problems.

The solution to continuous disruption requires new ways to think about, organize, and build and deploy national security people, organizations and solutions.

Here are our thoughts about how to confront the Red Queen trap and adapt a government agency to infuse continuous innovation in its culture and practices.
—

Problem 1: Regardless of a high-level understanding that business as usual can’t go on, all agencies are given “guidance and metrics (what they are supposed to do (their “mission”) and how they are supposed to measure success). To no one’s surprise the guidance is “business as usual but more of it.” And to fulfill that guidance agencies create structure (divisions, directorates, etc.) designed to execute repeatable processes and procedures to deliver solutions that meet the requirements of the overall guidance.

Inevitably, while all of our defense and national security agencies will tell you that innovation is one of their pillars, innovation actually is an ill-defined and amorphous aspirational goal, while the people, budget and organization continue to flow to execution of mission (as per guidance.)

There is no guidance or acknowledgement that in our national security agencies, even as we execute our current mission, our capabilities decline every year due to security breaches, technology timing out, tradecraft obsolescence, etc. And there is no explicit requirement for creation of new capabilities that give us the advantage.

Solution 1: Extend agency guidance to include the requirements to create a continuous innovation process that a) resupplies the continual attrition of capabilities and b) creates new capabilities that gives us a mission advantage. The result will be agency leadership creating new organizational structures that make innovation a continual process rather than an ad hoc series of heroic efforts.

Problem 2: The word “Innovation” actually describes three very different types of activities.

Solution 2: Use the McKinsey Three Horizons Model to differentiate among the three types. Horizon 1 ideas provide continuous innovation to a company’s existing mission model and core capabilities. Horizon 2 ideas extend a company’s existing mission model and core capabilities to new stakeholders, customers, or targets. Horizon 3 is the creation of new capabilities to take advantage of or respond to disruptive technologies/opportunities or to counter disruption.

We’d add a new category, Horizon 0, which kills ideas that are not viable or feasible (something that Silicon Valley is tremendously efficient at doing).

These Horizons also apply to government agencies and other large organizations. Agencies and commands need to support all three horizons.

Problem 3: Risk equals failure and failure is to be avoided as it indicates a lack of competence.

Solution 3: The three-horizon model allows everyone to understand that failure in a Horizon 1/existing mission activity is different than failure in a Horizon 3 “never been done before” activity. We want to take risks in Horizon 3. If we aren’t failing with some efforts, we aren’t trying hard enough. An innovation process embraces and understands the different types of failure and risk.

Problem 4: Innovators tend to create activities rather than deployable solutions that can be used on the battlefield or by the mission. Accelerators, hubs, cafes, open-sourcing, crowd-souring, maker spaces, Chief Innovation Officers, etc. are all great but they tend to create innovation theater – lots of motion but no action. Great demos are shown and there are lots of coffee cups and posters, but if you look at the deliverables for the mission over a period of years the result is disappointing. Most of the executors and operators have seen little or no value from any of these activities. While the activities individually may produce things of value, they aren’t valued within the communities they serve because they aren’t connected to a complete pipeline that harnesses that value and turns it into a deliverable on the battlefield where it matters.

Solution 4: What we have been missing is an innovation pipeline focused on deployment not demos.

The Lean Innovation process is a self-regulating, evidence-based innovation pipeline. It is a process that operates with speed and urgency, where innovators and stakeholders curate and prioritize their own problems/Challenges/ideas/technology. It is evidence based, data driven, accountable, disciplined, rapid and mission- and deployment-focused.

The process recognizes that Innovation isn’t a single activity (an incubator, a class, etc.) it is a process from start to deployment.
The canonical innovation pipeline:

As you see in the diagram, there are 6 steps to the innovation pipeline: sourcing, challenge/curation, prioritization, solution exploration and hypothesis testing, incubation and integration.

Innovation sourcing: a list of problems/challenges, ideas, and technologies that might be worth investing in. These can come from hackathons, research groups, needs from operators in the field, etc.

Challenge/Curation: innovators get out of their own offices and talk to colleagues and customers with the goal of finding other places in the DoD where a problem or challenge might exist in a slightly different form, to identify related internal projects already in existence, and to find commercially available solutions to problems. It also seeks to identify legal issues, security issues, and support issues.

This process also helps identify who the customers for possible solutions would be, who the internal stakeholders would be, and even what initial minimum viable products might look like.

This phase also includes building initial minimal viable products (MVPs.) Some ideas drop out when the team recognizes that they may be technically, financially, or legally unfeasible or they may discover that other groups have already built a similar product.

Prioritization: Once a list of innovation ideas has been refined by curation, it needs to be prioritized using the McKinsey Three Horizons Model.

Once projects have been classified, the team prioritizes them, starting by asking: is this project worth pursing for another few months full time? This prioritization is not done by a committee of executives but by the innovation teams themselves.

Solution exploration and hypotheses testing: The ideas that pass through the prioritization filter enter an incubation process like Hacking for Defense/I-Corps, the system adopted by all U.S. government federal research agencies to turn ideas into products.

This six- to ten-week process delivers evidence for defensible, data-based decisions. For each idea, the innovation team fills out a mission model canvas. Everything on that canvas is a hypothesis. This not only includes the obvious – is there solution/mission fit? — but the other “gotchas” that innovators always seem to forget. The framework has the team talking not just to potential customers but also with people responsible for legal, support, contracting, policy, and finance. It also requires that they think through compatibility, scalability and deployment long before this gets presented to engineering. There is now another major milestone for the team: to show compelling evidence that this project deserves to be a new mainstream capability. Alternatively, the team might decide that it should be spun into its own organization or that it should be killed.

Incubation: Once hypothesis testing is complete, many projects will still need a period of incubation as the teams championing the projects gather additional data about the application, further build the minimum viable product (MVP), and get used to working together. Incubation requires dedicated leadership oversight from the horizon 1 organization to insure the fledgling project does not die of malnutrition (a lack of access to resources) or become an orphan (continue to work with no parent to guide them).

Integration and refactoring: At this point, if the innovation is Horizon 1 or 2, its time to integrate it into the existing organization. (Horizon 3 innovations are more likely set up as their own entities or at least divisions.) Trying to integrate new, unbudgeted, and unscheduled innovation projects into an engineering organization that has line item budgets for people and resources results in chaos and frustration. In addition, innovation projects carry both technical and organizational debt. This creates an impedance mismatch between the organizations that can be easily be resolved with a small dedicated refactoring team. Innovation then becomes a continuous cycle rather than a bottleneck.

Problem 5: The question being asked across the Department of Defense and national security community is, “Can we innovate like startups in Silicon Valley” and insert speed, urgency and agility into our work?

Solution 5: The reality is that the DoD/IC is not Silicon Valley. In fact, it’s much more like a large company with existing customers, existing products and the organizations built to support and service them. And much like large companies they are being disrupted by forces outside their control.

But what’s unique is, that unlike a large company that doesn’t know how to move rapidly, on the battlefields of Iraq and Afghanistan our combatant commands and national security community were more agile, creative and Lean than any startup. They wrote the book on how to collaborate (read Team of Teams) or adopt new technologies (see the Rapid Equipping Force.) The problem isn’t that these agencies and commands don’t know how to be innovative. The problem is they don’t know how to be innovative in peacetime when innovation succumbs to the daily demands of execution. Part of the reason is that large agencies are run by leaders who tend to be excellent Horizon 1 managers of existing people, process and resources but have no experience in building and leading Horizon 3 organizations.

The solution is to understand that an innovation pipeline requires different people, processes, procedures, and metrics, then execution.

Problem 6: How to get started? How to get leadership behind continuous innovation?

Solution 6: To leadership, incubators, cafes, accelerators and hackathons appear to be just background noise unrelated to their guidance and mission. Part of the problem lies with the innovators themselves. Lots of innovation activities celebrate the creation of demos, funding, new makerspaces, etc. but there is little accountability for the actual rapid deployment of useful tools. Once we can convince and demonstrate to leadership that continuous innovation can solve the Red Queen problem, we’ll have their attention and support.

We know how to do this. Our country requires it.
Let’s get started.

Lessons Learned

Organizations must constantly adapt and evolve, to survive when pitted against ever-evolving opposition in an ever-changing environment

Government agencies need to both innovate and execute

In peacetime innovation succumbs to the demands of execution

We need explicit guidance for innovation to agencies and their leadership requiring an innovation organization and process, that operates in parallel with the execution of current mission

We need an innovation pipeline that delivers rapid results, not separate, disconnected innovation activities

Filed under: Corporate/Gov't Innovation, Hacking For Defense, Science and Industrial Policy | 15 Comments »

Email Subscription

Categories

Get Steve Blank via your RSS Feed

Recent Posts

Archives

Other Stuff

Team Fleetwise – Vehicle Fleet Management

Project Agrippa – Logistics and Sustainment in IndoPacific

Team Silknet – Detecting Ground Base Threats

Team Flexible Fingerprints – Improve Cybersecurity

Team Neurosmart – Optimizing Performance of Special Operators

Team AngelComms – Rescuing Downed Pilots

Team Salus – Patching Operational Systems to Keep them Secure

Team Mongoose – Tracking Hackers Disposable Infrastructure

Team Engage – Preparing Aviators to Make Critical High Stakes Decisions

Share this:

Like this:

Register here.

Share this:

Like this:

Share this:

Like this:

Team Omniscient – An Unclassified Imaging Analyst Workbench

Team Protocol One – Ensuring JTAC to Pilot Communication

Team SeaWatch – Maritime Security in the South China Sea

Team TimeFlies – Automating Air Force aircrew scheduling

Team AV Combinator – Autonomous Vehicle Safety Standards

Team Anthro Energy – next generation lightweight flexible batteries

Team Helmsman – Navigating in GPS denied areas

Team Election Watch – Open Source Tool to Track Political Influence Campaigns

Share this:

Like this:

Share this:

Like this:

Team: Panacea

Team: Learn2Win

Team: Embargo NK

Team: Common Ground

Team: Deepfakes

Team: IntelliSense

Team: Gutenberg

Team: PredictiMx

Share this:

Like this:

Team: TrackID

Team: Polaris

Team: Acquiforce

Team: Intelgrids

Team: See++

Team: Theia

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Share this:

Like this:

Calendar

Customer Development

Secret History of Silicon Valley