Hacking for Defense in 2 minutes

Hacking for Defense in 2 minutes.

If you can’t see the video click here

Hacking for Defense class here

Hacking for Diplomacy class here

Hacking for Diplomacy – Solving Foreign Policy Challenges with the Lean LaunchPad

Hacking for Diplomacy is a new course from the Management Science and Engineering department in Stanford’s Engineering school and Stanford’s International Policy Studies program that will be first offered in the Fall of 2016.

Join a select cross-disciplinary class that takes real problems from the U.S. State Department and asks students to use Lean Methods to test their understanding of the problem and deliver rapid-fire innovative solutions to pressing diplomacy, development and foreign policy challenges.

H4Dip home page


Syrian Refugees, Human Trafficking, Zika Virus, Illegal Fishing, Weapons of Mass Destruction Detection, ISIS on-line propaganda, Anti-Corruption…
What do all these problems have in common? The U.S. Department of State is working on all of them.

The U.S. Department of State manages America’s relationships with 180 foreign governments, international organizations, and the people of other countries with 270 embassies, consulates, and other posts. The management of all these relationships is called diplomacy. 70,000 State Department employees (46,000 Foreign Service Nationals, 14,000 Foreign Service Employees and 11,000 in the Civil Service) carry out the President’s foreign policy and help build a freer, more prosperous, and secure world.

The State Department has four main foreign policy goals:

  • Protect the United States and Americans
  • Advance democracy, human rights, and other global interests
  • Promote international understanding of American values and policies
  • Support U.S. diplomats, government officials, and all other personnel at home and abroad.

At a time of significant global uncertainty, diplomats are grappling with a set of transnational and cross-cutting challenges that defy easy solution. These include the continued pursuit of weapons of mass destruction by states and non-state groups, the outbreak of internal conflict across the Middle East and in parts of Africa, the most significant flow of refugees since World War II, and a changing climate that is beginning to affect both developed and developing countries.  And that’s just on Monday.  The rest of the week is equally busy.

State dept org chart

Hacking for Diplomacy
In a world of complex threats, dynamic opportunities, and diffuse power, effective diplomacy and development require institutions that adapt, embrace technology, and allow for experimentation to ensure continuous learning. This means developing new and innovative ways to think about, organize and build diplomatic strategies and solutions.  Stanford’s new Hacking for Diplomacy class is a part of this effort.

Hacking for Diplomacy is designed to provide students the opportunity to learn how to work with the Department of State to address urgent foreign policy challenges. While the traditional tools of statecraft remain relevant, policymakers are looking to harness the power of new technologies to rethink how the U.S. government approaches and responds to  the problems outlined above and other long-standing challenges. In this class, student teams will take actual foreign policy challenges and a hands-on approach that will require close engagement with officials in the U.S. State Department and other civilian agencies. They’ll learn how to apply “lean startup” principles, (“mission model canvas,” “customer development,” and “agile engineering”) to discover and validate agency and user needs and to continually build iterative prototypes to test whether they understood the problem and solution.

Each week, teams will use the Mission Model Canvas (a variant of the Business Model Canvas used for government agencies and non-profits) to develop a set of initial hypotheses about a solution to the problem and will get out of the building and talk to relevant stakeholders and users. As they learn, they’ll iterate and pivot on these hypotheses through customer discovery and build minimal viable prototypes (MVPs). Each team will be guided by a sponsor from the State Department bureau that proposed the problem and a second mentor from the local community.

Real Problems
In this class, student teams select from an existing set of problems provided by the Department of State. Hacking for Diplomacy is not a product incubator for a specific technology solution. Instead, as teams follow the Mission Model Canvas, they’ll discover a deeper understanding of the selected problem, the challenges of deploying the solutions, and the host of potential technological solutions that might be arrayed to solve them. Using the Lean LaunchPad Methodology the class focuses teams to:

  • Deeply understand the problems/needs of State Department beneficiaries and stakeholders
  • Rapidly iterate technology solutions while searching for product-market fit
  • Understand all the stakeholders, deployment issues, costs, resources, and ultimate mission value
  • Produce a repeatable model that can be used to launch other potential technology solutions

National Service
Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or AmeriCorps. Few consider opportunities to make the world safer with the State Department and other government agencies. Hacking for Diplomacy will promote engagement between students and the Department of State and provide a hands-on opportunity to solve real diplomacy, foreign policy and national security problems.

Like its sister class, Hacking for Defense, our goal is to open-source this class to other universities. By creating a national network of colleges and universities, Hacking for Diplomacy can scale to provide hundreds of solutions to critical diplomacy, policy, and national security problems every year.

We’re going to create a network of entrepreneurial students who understand the diplomatic, policy, and national security problems facing the country and get them engaged in partnership with islands of innovation in the Department of State. This is a first step to a more agile, responsive and resilient, approach to diplomacy and national security in the 21st century.

Lessons Learned

  • Hacking for Diplomacy is a new class that teaches students how to:
    • Use the Lean LaunchPad methodology to deeply understand the problems/needs of State Department customers
    • Deliver minimum viable products that match State Department needs in an extremely short time
  • The class will also teach the islands of innovation in the Department of State:
    • how the innovation culture and mindset operate at speed
    • how to identify potential dual-use technologies that exist outside their agencies and contractors (and are in university labs, or are commercial off-the-shelf solutions)
    • how to use an entrepreneurial mindset and Lean Methodologies to solve foreign policy problems
  • Register for the inaugural class at Stanford starting September 29th now

Hacking for Defense & Hacking for Diplomacy – Educator/Sponsor Class

There is nothing more powerful than an idea whose time has come
Victor Hugo

On September 7th – 9th we are holding our first Hacking for Defense & Diplomacy – class for Educators and Sponsors, training educators how to teach these classes in their universities and sponsors how to select problem sets and manage their teams.

hacking classesSign up here.


An Idea Whose Time Has Come
Our first Hacking for Defense class was a series of experiments. And like all good experiments we tested a set of hypotheses. Surprisingly the results blew past our expectations – and we had set a pretty high bar. (see the final Hacking for Defense class presentations here). Based on those results we believe that we can do the same with Diplomacy so working with the State Department’s innovation cell in Silicon Valley we will prototype the Hacking for Diplomacy course at Stanford this fall.

A few of the student and sponsor comments about the class:

Absolutely amazing class. Experiential learning is very effective to really grasp what we claim to know intellectually.”  – computer science grad student

 “One of the best classes I’ve ever taken, and it turned me onto a whole new career path.”  – MBA student

 “We’re still blown away what students who knew nothing about our agency could learn and deliver in such a short period of time.”  – sponsor

First, would students to sign up for a class that engaged them in national service with the Department of Defense and Intelligence Community?

Result: I’ll admit my hesitancy was because I brought my memories of U.S. college campuses circa the Vietnam War (and the riots and student protests at Stanford). So I was astonished how ready and eager students were for a class that combines the toughest problems in national security, with learning Lean Innovation methods. We had more applicants (70+) for the 32 seats in this class than we usually get in our Lean LaunchPad entrepreneurship class. And early indications are that Hacking for Diplomacy will be at least as popular.

Second, could we find islands of innovation inside the DOD, the Intelligence Community and State Department willing to engage students to work on real problems? And could those sponsors work with us to scrub those problems so they were unclassified but valuable to the sponsors and the students?

Result: We solicited 8 problems for the students to work on and had to shut down the submission process after we reached 24 from the DOD.  We’ve now built a national clearing house for DOD/Intel problems that other schools can use. The Department of State has already given us 15 problems for our upcoming Diplomacy class.

Third, would students be turned off by working problems that weren’t theirs, in particular from the DOD and Intel community?

Result: We surveyed student motivations before and after the class and were surprised to find that a large percentage became more interested and engaged in national service. Over half the student teams have decided to continue working on national security projects. At the end of class two teams were funded by SOCOM (U.S. Special Operations Command) to continue prototyping over the summer. One team closed a $200k seed round in the middle of the class. Multiple teams have been engaged by government, prime contractor and VC firms for follow-on discussions/engagements.

Fourth, would the same Lean Startup methodology (business model design, customer development and agile engineering) used in the Lean LaunchPad and NSF I-Corps class work here?

Result: Hell yes.

Fifth, would other schools be interested in offering this class?

Result: Seven schools have already added Hacking for Defense classes: UC San Diego, University of Pittsburg, University of Southern California, Stanford, University of Rochester, Georgia Tech and Georgetown University. 15 more schools are in the pipeline. NDU (the National Defense University) National Security Accelerator (NSTA), the Stanford University Hacking 4 Defense Project and JIDA (the Joint Improvised-threat Defeat Agency) have all teamed up to fund the expansion of the Hacking for Defense class to other universities. The Department of Energy Advanced Manufacturing Office has also lent its support to the expansion. If our Hacking for Diplomacy goes well this fall, we intend to scale it as well.

The Educator/Sponsor Class
We learned a lot developing the Hacking for Defense class, and even more as we taught it and worked with the problem sponsors in the DOD/Intel community. Now we’ve created a ton of course materials for educators (syllabus, slides, videos) and have written a detailed educator’s guide with suggestions on how to set up and run a class along with best practices and detailed sample lessons plans for each class session. And for sponsors we have an equally robust set of tools on how to get the most out of the student teams and the university. And we’re excited it to share it all with other educators and sponsors in the DOD/Intel community.

So on September 7 through 9th at Stanford we will hold our first 2.5-day Hacking for Defense & Diplomacy Educator and Sponsor Class.

We’ll provide you all the course materials (syllabus, slides, videos) along with an educator’s guide with suggestions on how to set up and run a class along with detailed sample lessons plans for each class session. You’ll also

  • Meet with other instructors and problem sponsors and experience the Hacking for Defense/Diplomacy methodology first hand
  • Learn how to build H4D teaching teams and recruit student teams to participate
  • Learn what makes a good student problem and how problem sponsors can increase their Return On Investment for supporting the course
  • Engage with the original Stanford Hacking For Defense course authors (Pete Newell, Joe Felter and I) and students from the original H4D cohort
  • Engage directly with potential government problem sponsors

Life is series of unplanned paths and unintended consequences – Hacking For Defense
A year ago as I started helping government agencies put innovation programs in place, a student in my Stanford class who had served in the special forces pointed out that the Lean methodologies I was teaching sounded identical to what the U.S. Army had done with the Rapid Equipping Force (REF) commanded by then Colonel Pete Newell.

The REFs goal was to get out of the building and into the field to get a deep understanding of soldiers’ problems, then get technology solutions to these problems into the hands of front-line soldiers in days and weeks, instead of the military’s traditional months and years. The REF had permission to shortcut the detailed 100+ page requirements documents used by the defense acquisition process and could use existing government equipment or buy or commercial-off-the-shelf technologies purchased with a government credit card or its own budget.

When Pete Newell retired to Silicon Valley he teamed up with Joe Felter, another retired colonel, who had a career as a Special Forces and foreign area officer (among other things, Joe led the Counterinsurgency Advisory and Assistance Team (CAAT) in Afghanistan) and was now teaching at Stanford’s Center for International Security and Cooperation (CISAC). Together Pete and Joe had formed BMNT to create an “insurgency” in Silicon Valley to help accelerate the way the Department of Defense acquires new technology and ideas and integrates cutting-edge innovation into the organizations defending our country.

The Hacking for Defense & Diplomacy classes were born from the intersection of BMNTs work with the Department of Defense in Silicon Valley and my work in Lean Innovation.

We had five goals for the class:

  1. Teach students Lean Innovation – the mindset, reflexes, agility and resilience an entrepreneur needs to make decisions at speed and with urgency in a chaotic and uncertain world.
  2. Offer students an opportunity to engage in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community, State Department or other government agencies.
  3. Teach our sponsors (the innovators inside the Department of Defense (DOD), Intelligence Community (IC) and State Department) there was a methodology that could help them understand and better respond to rapidly evolving asymmetric threats. (By rapidly discovering the real problems in the field using Lean methods, and then articulating the requirements to solve them, defense acquisition programs operate at speed and urgency to deliver timely and needed solutions.)
  4. Show our DOD/IC/State sponsors that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions.
  5. Create the 21st Century version of Tech ROTC by having Hacking for Defense and Hacking for Diplomacy taught by a national network of 50 colleges and universities. This would give the Department of Defense (DOD), Intelligence Community (IC) and State Department access to a pool of previously untapped technically sophisticated talent, trained in Lean and Agile methodologies, and unencumbered by dogma and doctrine.

It looks like we’re on our way to achieving all of these goals.  Join us.

Lessons Learned

hacking classes

Hacking for Defense @ Stanford – Lessons Learned Presentations

We just held our tenth and final week of the Hacking for Defense class. Today the eight teams presented their Lessons Learned presentations.

We’re a little stunned about how well the first prototype of this class went. Over half the student teams have decided to continue working on national security projects after this class. Other colleges and universities have raised their hand and said they want to offer this at their school.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

IMG_3189


What Were Our Goals for this Class?
We had five goals for the class. First was to teach students to develop the mindset, reflexes, agility and resilience an entrepreneur needs to make decisions at speed and with urgency in a chaotic and uncertain world.

Second, we wanted to teach students entrepreneurship while they engage in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community or other government agencies.

Third was to teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there was a methodology that could help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulating the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Fourth, we wanted to show our DOD/IC sponsors that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions.

Fifth, we wanted to create the 21st Century version of Tech ROTC by having Hacking for Defense taught by a national network of 50 colleges and universities. This would give the Department of Defense (DOD) and Intelligence Community (IC) access to a pool of previously untapped technically sophisticated talent, trained in Lean and Agile methodologies, and unencumbered by dogma and doctrine. At this size the program will provide hundreds of solutions to critical national security problems every year.

The result will be a network of thousands of entrepreneurial students who understand the security threats facing the country and engaged in partnership with islands of innovation in the DOD/IC. This is a first step to a more agile, responsive and resilient, approach to national security in the 21st century.

What Did We Learn From the Class?
Not only did the students learn, but the teaching team got schooled as well.

First, we validated that students were ready and willing to sign up for a class that engaged them in national service with the Department of Defense and Intelligence Community. We had more applicants (70+) for the 32 seats in this class than we usually get in our core entrepreneurship class.

Second, we found that the islands of innovation inside the DOD and IC were willing to engage this new and eager pool of talent. We were soliciting 8 problems for the students to work on and had to shut down the submission process after we reached 25.

Third, some students took the class because they thought learning entrepreneurship with tough real-world problems would be interesting. We surveyed their motivations before and after the class and were surprised to find that a large percentage became more interested and engaged in national service. Over half the student teams have decided to continue working on national security projects.

Fourth, other schools have said they want to offer this class next year. To help kick this scale into high gear, the National Defense University will be funding Hacking for Defense at colleges and universities across the country. To train other educators and future problem sponsors we we will hold our first Hacking for Defense/Diplomacy Educators Class September 7 through 9th. Contact Pete Newell peter.newell@gc.ndu.edu to sign up.

Finally, the teaching team (instructors, TA’s, mentors) and students debriefed on our own Lessons Learned from the class. Joe Felter and his research assistants will spend the summer building out the formal educator’s guide (capturing all the “wish we would have known’s” and “here are the points you need to make in this lecture”,) sponsor guide (yep, we learned we need to train our sponsors as well), creating new DOD/IC-specific video lectures. And we will build a knowledge base of DOD/IC acquisition primers, customer development best practices, org charts, etc. Finally, for universities interested in running future courses, HackingForDefense.org will act as a central clearing house for student-ready problems that have been vetted and unclassified. While H4Di.org gets on its feet Pete Newell and his team of RA’s will continue to source problems for upcoming H4D courses.

What Surprised Us?

  1. The combination of the Mission Model Canvas and the Customer Development process was an extremely efficient template for the students to follow – even more than we expected.
  2. It drove a hyper-accelerated learning process which led the students to a “information dense” set of conclusions. (Translation: they learned a lot more, in a shorter period of time than in any other incubator, hackathon, entrepreneurship course we’ve ever taught or seen.)
  3. Insisting that the students keep a weekly blog of their customer development activities gave us insight into their progress in powerful and unexpected ways.

What Would We Change?

  1. Train the sponsors on commitment, roles, etc.
  2. Decide how we want the teams to split their time for potential dual-use products. How much time spent on focusing on the sponsors particular problem versus finding a commercial market. And what week to do so.

This is the End
Each of the eight teams presented a 2-minute video to provide context about their problem and then gave an 8-minute presentation of their Lessons Learned over the 10-weeks. Each of their slide presentation follow their customer discovery journey. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

The teams presented in front of several hundred people in person and online. You can watch the entire presentation here

https://vimeo.com/169155566

Aqualink

If you can’t see the video click here

If you can’t see the presentation click here

Capella Space

If you can’t see the video click here

If you can’t see the presentation click here

Narrative Mind

If you can’t see the video click here

If you can’t see the presentation click here

Fishreel

If you can’t see the video click here

If you can’t see the presentation click here

Sentinel

If you can’t see the video click here

If you can’t see the presentation click here

Skynet

If you can’t see the video click here

If you can’t see the presentation click here

Right of Boom

If you can’t see the video click here

If you can’t see the presentation click here

Guardian 

If you can’t see the video click here

If you can’t see the presentation click here

It Takes a Village
While I authored these blog posts, the class was truly a team project. The teaching team consisted of:

  • Tom Byers, Professor of Engineering and Faculty Director, STVP
  • Joe Felter a retired Army Special Forces Colonel with research and teaching appointments at Stanford’s Center for International Security and Cooperation (CISAC), the Hoover Institution, and the dept. of Management Science and Engineering
  • Jackie Space a former Air Force officer who as an aerospace engineer developed joint satellite and electronic warfare programs. She is currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and Managing Partner at at BMNT Partners
  • Pete Newell is a former retired Army Colonel currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and CEO of BMNT Partners.

Kim Chang was our lead teaching assistant. We were lucky to get a team of 25 mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams.

Of course, a huge thanks to the 32 Stanford students who suffered through the 1.0 version of the class.

And finally a special thanks to our course advisor Bill Perry, former Secretary of Defense and Professor Emeritus, Chris Zember, Director, National Defense University – Center for Technology & National Security Policy, Jay Harrison, Director, National Defense University – National Security Technology Accelerator, Dr Malcolm Thompson, the executive Director of NextFlex, the Flexible Hybrid Electronics Manufacturing Innovation Institute, The entire Defense Innovation Unit Experimental (DIUX), Center for International Security and Cooperation (CISAC) in the Freeman Spogli Institute for International Studies, STVP in the department of Management Science and Engineering.

Hacking for Defense will be offered again at Stanford University next Winter.  See you there!

Hacking for Defense @ Stanford – Weeks 8 and 9

We just held our eighth and ninth weeks of the Hacking for Defense class. Now with over 917 interviews of beneficiaries (users, program managers, stakeholders, etc.), the teams spent the last two weeks learning what activities, resources and partners they would need to actually deliver their solution. And they’re getting a handle on what it costs to build a company to deliver it.

Understanding the left-side of the mission model canvas (activities, resources, partners, and costs) forces all teams to ask, “Are we building a product for a DOD/IC customer only or do we have a “dual-use” product that could be sold commercially and get funded by venture capital?”

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

Next week the teams will present their final Lessons Learned presentations.

Two Items for the Bucket List
Two bucket list items got ahead of my blogging so I’ve combined the final two lecture weeks of the class into this one blog post.

Four decades ago my first job in Silicon Valley was with ESL, the first company to combine computers and signals intelligence. The founder of this 1964 Silicon Valley startup was Bill Perry. His work at ESL made him one of the 10 founders of National Reconnaissance.

Dr. Perry eventually became the 19th secretary of defense. But a decade earlier as undersecretary of defense for research and engineering, he was the father of the second offset strategy using software and semiconductors to build smart weapons, smart sensors, and stealth aircraft that helped end the Cold War.

Last week I interviewed Bill at Stanford about War and Peace, innovation and entrepreneurship.

http://ecorner.stanford.edu/videos/4255/Dedication-to-Innovation-and-Nation-Entire-Talk

Second:
I also gave the commencement speech at the NYU engineering school here.

The Left side of the Canvas
If you’ve been reading along so far, you know that this class is not an extended hackathon nor is it a 10-week long incubator. Hackathons and incubators are helpful in getting product teams focused and result in great demos, but you’re left still not knowing whether you have something beneficiaries/stakeholders/users want nor do you know what it takes to deploy the solution to the field. Ultimately you are left without a strategy to turn your idea into a solution that people will use.

Using the Lean LaunchPad methodology our teams do much more than just build a product or understand customer problems/needs. They also learn how to deploy the solution, how to get stakeholder buy-in and how to measure success. And in these last two weeks of class, they learn what activities, resources and partners they’ll need to deliver their solution and derive what it costs to build the company to deliver it.

The teams capture their work in the mission model canvas a framework for each week’s activities. The canvas illustrates the search for the unknowns that new ventures face. The 9 boxes of the canvas visualize all the components needed to turn beneficiaries needs/problems into a solution.

Mission Model Canvas by week
Each week the teams marched through another box of the canvas, testing their hypotheses in front of beneficiaries using the customer development methodology, all while building and updating their minimal viable product. It’s a ton of work. Over the course of the class, each team will have talked to 100 beneficiaries/ stakeholders/ users. The result is evidence-based entrepreneurship.

Team Presentations: Weeks 8 and 9
Over these last two weeks, teams began to figure out the activities, resources and partners their company would need to deliver their value proposition (product, service or both) to the beneficiaries in their sponsor organizations.

Activities are the expertise and resources that the company needs to deliver the value proposition. They might be hardware development, software expertise, manufacturing, launching rockets, funding, etc. Resources are the internal company-owned activities. Examples are a company-owned manufacturing facility, big data or machine learning engineers, DOD proposal writers, venture capital, etc.  Partners are the external resources (third parties) necessary to execute the Activities. i.e. outsourced manufacturing, system integrators, etc. other companies, that will provide those activities.

activities resources and partners

In addition, teams worked on understanding the costs and operations and deployment timelines for delivering the product to their sponsor.

finance and ops timeline

Team Dynamics
In these last three weeks the benefit of having a team of mixed business and technical resources becomes apparent. Teams that are just all technologists quickly grasp product/market fit (the right side of the canvas) but often have a hard time understanding the left side of the canvas (activities, resources, partners and costs.)  When the technologists work together with business focused students as a team, the learning is impressive.

However, the downside is that one of failure modes of teams (and startups) is a team that doesn’t jell. One of the symptoms is technologists going heads-down building product and features without customer input while they defer all of the left-side of the canvas to the business team. Or conversely business team members draw timelines and costs without a deep understanding of the technology hurdles.

Almost every class has a team or two that goes through team conflict – different working styles, different time commitments, pivots taking them to places where they’re no longer interested, etc. Given that 1/4 of startups meltdown over team dynamics before funding, seeing this happen to teams in the class isn’t a surprise. We treat team dynamics as a normal part of learning in the class. (Team members get to grade each other on their contributions as part of their final grade.)

Considering that none of these teams have worked together in the past, the amount of synergy and teamwork in this cohort is impressive.

Skynet

WEEK 8 Presentation

In slide 2 the Skynet team continued with customer discovery using experiments to validate or invalidate their hypotheses. Slide 5 does a good job of separating out their technical versus business activities. Slide 6 did a great job in connecting the activities to the resources and partners they’ll need.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

In Slide 2 the team made progress on developing their MVP. In slide 3 they realized some of their conclusions about DARPA partnerships from last week were wrong. Slides 5-8 continued their learning about partnerships, and slides 9-11 are a great first pass on costs and financial and operations timeline.

If you can’t see the week 9 presentation click here

Aqualink

WEEK 8 Presentation

Slide 5 is a good summary of activities/resources/partners. Slide 6 connects those to the prototyping and deployment activities by partner and sponsor. Slide 7 lays out a potential field deployment schedule to the sponsor organization. Slides 11-14 show their continued testing of their MVP underwater.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 7 and 9 is the team’s first pass in understanding costs, operations and fundraising. They continued their MVP development underwater in a pool at Stanford.

If you can’t see the week 9 presentation click here

Sentinel

WEEK 8 Presentation

The team really got out of the building and traveled to San Diego (at their own expense) and visited the USS Sampson and the 3rd Fleet headquarters. Slide 7 summarizes their activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slide 13 is an excellent example of mapping out their costs.  Slide 14 is a great example of diagramming their financial and operating milestones.

If you can’t see the week 9 presentation click here

Capella

WEEK 8 Presentation

This week Capella was so engaged in their customer discovery and pivot to illegal fishing, they missed the assignment.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 8 -12 illustrates their activities and costs. Because they missed last week’s assignment, you wouldn’t know from their presentation that they required a partnership with a space launch company :-)  The good news is this team had been distracted and will have news to share in their Lessons Learned presentation

If you can’t see the week 9 presentation click here

Guardian

WEEK 8 Presentation

Slides 4 -6 summarized Guardians activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 4 -6 summarized their costs and operating plan.

If you can’t see the week 9 presentation click here

Right of Boom

WEEK 8 Presentation

Slides 4 -6 summarized Right of Booms’ activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slide 3-5 summarizes their unique findings. This team discovered that their deliverable to the sponsor will not be a product. Instead it will be a series of recommendations on how to better utilize their existing products and data. Slides 6-8 describe the partners which can best deliver these recommendations to their sponsor.

If you can’t see the week 9 presentation click here

Narrative Mind

Slides 3 -6 summarized their activities, resources and partners

WEEK 8 Presentation

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 3 -10 further refined their partners and summarized their costs and operating plan.

If you can’t see the week 9 presentation click here

Advanced Lecture 8: Costs
In week 8 Pete Newell gave the costs lecture and put it in the context of a DOD program. Slide 3 defined what costs were, slides 4-11 tied it to a specific example.

If you can’t see the costs lecture click here

Advanced Lecture 9: Reflections
In past versions of this class teams would call on beneficiaries/customers until the last week of the class and then present their Lessons Learned. The good news is that their presentations were dramatically better than those given at demo days – they showed us what they learned over 8 weeks which gave us a clear picture of the velocity and trajectory of the teams. The bad news is since their heads were down working on customer discovery until the very end, they had no time to reflect on the experience.

We realized that we had been so focused in packing content and work into the class, we failed to give the students time to step back and think about what they actually learned.

So now we use the last week of the class as a reflection week. Our goal—to have the students extract the insights and meaning from the work they had done in the previous seven weeks.

We asked each team to prepare a draft Lessons Learned presentation telling us about their journey and showing us their:

  • Initial sponsor problem statement
  • Quotes from beneficiaries that illustrated learnings and insights
  • Pivot stories
  • Screen shots of the evolution of Minimum Viable Product (MVP)
  • Demo of final MVP

The teaching team reviewed the drafts and provided feedback to the teams and to the class as a whole. We discussed what general patterns and principles they extracted from all the customer interaction they had.

Dual-use Products
As you’ll see next week in the final presentations, some of the teams discovered that they could best serve their sponsor by building a commercial off-the-shelf product that could be sold widely and bought by the DOD/Intel community. Pete Newell came up with the best diagram I’ve seen to illustrate how the work the teams were doing in this class fit to do just that.

The diagram shows that during the class the sponsor needs drive customer discovery and product/market fit. But continued discovery would now include commercial customers and eventually those commercial customer needs would drive the feature set.

dual use trajectory

Hacking for Defense Educators Class
The H4D instructor team has been busy capturing what we learned (teams, lectures, sponsors, etc.) and we’ll incorporate the lessons from this inaugural course and revise the course materials. As part of our plan to scale this class nationwide to other schools, we’re writing an educator’s guide and offering a Hacking for Defense Educators Class Sept 7th – 9th.

Details in the next post.

Tomorrow, May 31st is the last day of class.

We’ll post the final presentations. Quite a journey for all these teams and their sponsors!

Hacking for Defense @ Stanford – Week 7

We just held our seventh week of the Hacking for Defense class. Now with over 750 interviews of beneficiaries (users, program managers, stakeholders, etc.) almost all the teams are beginning to pivot from their original understanding of their sponsor’s problem and their hypotheses about how to solve them. Minimal viable products are being demo’d to sponsors and sponsors are reacting to what the teams are learning. This week teams figured out how to measure mission achievement and success, and our advanced lectures were on activities, resources and partners.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

—-

Why Innovation in Government Is Hard
As we spend more time with the military services, commands and agencies it’s apparent that getting disruptive innovation implemented in the DOD/IC face the same barriers as large corporations (and a few more uniquely theirs.)

The first barrier to innovation is the Horizon 1 leadership conundrum. In corporations, the CEO and executives have risen through the ranks for their skill on executing existing programs/missions. The same is true in most DOD/IC organizations: leadership has been promoted through the ranks for their ability to execute existing programs/missions. By the time they reach the top, they are excellent managers of processes and procedures needed to deliver a consistent and repeatable execution of the current core mission (and typically excellent political players as well.)

These horizon 1 leaders are exactly who you want in place when the status quo prevails – and when competitors / adversaries react as per our playbook.

To these Horizon 1 leader’s, innovation is often considered an extension of what they already do today. In companies this would be product line extensions, more efficient supply chain, new distribution channels. In the DOD/IC innovation is often more technology, more planes, more aircraft carriers, more satellites, etc.

This “more and better” approach works until they meet adversaries – state and non-state – who don’t follow our game plan – adversaries who use asymmetry to offset and degrade our technological or numerical advantages – roadside bombs, cyberattacks, hybrid warfare, anti-access/area denial (A2/AD), etc.

disruption by adversaries

History tells us that what gets you promoted in peacetime
causes you to lose in wartime.

When Horizon 1 leaders set up innovation groups the innovators at the bottom of the organization start cheering. Meanwhile the middle of the organization strangles every innovation initiative. Why? Most often four points of failure occur:

  1. Horizon 1 leaders tend to appoint people who they feel comfortable with – Horizon 1 or perhaps Horizon 2 managers. This results not in innovation, but in Innovation Theater – lots of coffee cups, press releases, incubators and false hopes, but no real disruptive changes. Horizon 3 organizations require Horizon 3 leadership (with Horizon 1 second in command.)
  2. There needs to be effective communication about what being innovative means to different parts of their organizations as well as defining (and enforcing) their expectations for middle management. How do middle mangers know how to make trade-offs between the efficiency requirements of their Horizon 1 activities and the risks required of a Horizon 3 activity?
  3. They have to create incentives for middle management leaders to take on horizon three ideas
  4. They have to change the metrics across the entire organization. If not, then the effectiveness of the Horizon 3 effort will be graded using Horizon 1 metrics

Secretary of Defense Carter’s recent pivot to place the DOD’s innovation outpost – DIUx directly under his supervision after 8 months is a great example of a leader enforcing his expectations about innovation.

In peacetime Horizon 3/disruptive groups need to be led by Mavericks, sponsored and protected by Horizon 1 leadership. It is this group, challenging the dogma of the existing programs, who will come up with the disruptive/asymmetric offset technologies and strategies.

both types of leadership 2

BTW, history tells us that in war time the winners filled this innovation role with people who make most Horizon 1 leaders very uncomfortable – Churchill in WWII, Billy Mitchell, Oppenheimer on the Manhattan Project, Vannevar Bush at the OSRD, John Boyd, etc.

More next week on innovation and the intransigent middle. Now back to the class. 

Team Presentations: Week 7
In a company you know you’ve been successful when you generate revenue and profit. But in the military success has different metrics. This week the teams’ assignment was to understand what Mission Achievement and/or Mission Success looked like for each of their sponsor organizations and each of the beneficiaries inside that organization.

Later in the class some of the team will realize they can build “dual-use” products (building their product primarily for civilian use but also sold to the military.) In those case revenue will become an additional metric.

Understanding how to measure mission achievement/success for each beneficiary is the difference between a demo and a deployed solution.

Sentinel initially started by trying to use low-cost sensors to monitor surface ships for their 7th fleet sponsor in a A2/AD environment. The team pivoted and has found that their mission value is really to enable rapid, well-informed decisions by establishing a common maritime picture from heterogeneous data.

Sentinel displayOn Slide 4-5 the team continues testing their hypotheses via customer discovery. Note that they plan a trip to San Diego to visit the customer. And they realized that an unclassified proxy for their data is the IUU fishing problem. (With a great assist from the State Departments innovation outpost in Silicon Valley.) Their Minimum Viable Product can be seen on slides 12-16 using this illegal fishing data.

Slide 10 summarized what mission achievement would look like for three beneficiaries in the 7th fleet.

If you can’t see the presentation click here

Capella Space started class believing that launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging was their business. Now they’ve realized that the SAR data and analytics is the business.  Then the question was, “For whom?”

In slides 4- 11 they describe what they learned about illegal fishing in Indonesia (Thanks again to the State Departments innovation outpost.) But the big idea on slide 12 – 13 is that Capella has pivoted. The team realized that there are many countries that want to detect boats at night. And most of the countries of interest are located in the equatorial belt. Slide 14 is their rough outline of mission achievement for the key agencies/countries.

Interesting to note that Capella Space and Team Sentinel seem to be converging on the same problem space!

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing the U.S. Army Cyber Command (ARCYBER) and others to efficiently respond and mitigate threats posed by enemy messaging.

In slide 4 the team provided a textbook definition of mission achievement. They specified what success looks like for each of the beneficiaries inside of their sponsor, ARCYBER. In slide 5 they broadly outlined mission achievement for three private sector markets.

In slides 6-9 they plotted all the potential adversary communication trends on social media problems, and in slide 7 overlaid that problem space with existing commercial solutions. Slides 8 and 9 show the problems not yet solved by anyone, and slide 9 further refines the specific problems this team will solve.

NarrativeMind further refined their Minimal Viable Product to product/market fit in Slides 11-16.

If you can’t see the presentation click here

Aqualink started the class working to give Navy divers in the Naval Special Warfare Group a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. A few weeks ago they pivoted, realizing that the high-value problem the divers want solved is underwater 3-D geolocation.

Slide 2, John Boyd and the OODA Loop (finally!) makes an appearance in the class. (The OODA loops and the four steps of Customer Development and the Lean Methodology are rooted in the same “get of the building/get eyes out of your cockpit” and “speed and urgency” concepts.) In Slides 5-7 Aqualink’s two versions of their Minimum Viable Product are beginning to be outlined and in Slide 8, the team passed around physical mockups of the buoy.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones for the Army’s Asymmetric Warfare Group.

The team certainly got out of the building this week. In between their classes they flew to the east coast and attended the Army’s Asymmetric Warfare Drone Demo-Day at Fort A.P. Hill in Virginia. They spoke to lots of vendors and got a deep understanding of currently deployed tactical drones.

Slides 5-9 show their substantial progress in their Minimal Viable Product as they demo’d advanced detection and classification capabilities. They are beginning to consider whether they should pivot to become a drone software platform.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission.  Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

This week the team was actually able to talk to a key beneficiary on the front lines overseas. What they discovered is that the JIDA current technical solutions, if combined, will provide a solution of equal quality to standalone development in a shorter timeframe.

On slide 4 they outlined their Mission Achievement / Success criteria for the key JIDA beneficiaries.  Slide 9 continued to refine their understanding of the tradespace.

If you can’t see the presentation click here

Skynet is using drones to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

Mission achievement on slide 2 needs a bit of explanation; the team has met and exceeded their basic goals to reach: 80% accuracy on target identification. From SOCOM’s perspective the team has achieved their initial mission. Now Skynet has moved beyond their original scope into an interesting area. Slide 9 and 10 show their further refinement of buy in- for SOCOM and the Border Patrol.

If you can’t see the presentation click here

Advanced Lecture – Activities, Resources and Partners
Pete Newell presented the advanced lecture on Activities, Resources and Partners.

Activities are the expertise and resources that the company needs to deliver the value proposition. Resources are the internal company-owned activities. Examples are a company-owned manufacturing facility, big data or machine learning engineers, DOD proposal writers, venture capital, etc. Partners are the external resources necessary to execute the Activities. i.e. outsourced manufacturing, system integrators, etc.

If you can’t see the presentation click here

Lessons Learned

  • History tells us that what gets you promoted in peacetime causes you to lose in wartime
  • Teams are making substantive pivots on their understanding of the real sponsor problem and pivoting on their proposed solution
  • Understanding how to measure mission achievement/success for each beneficiary is the difference between a demo and a deployed solution

Hacking for Defense @ Stanford – Week 6

We just held our sixth week of the Hacking for Defense class. Now with over 660 interviews of beneficiaries (users, program managers, stakeholders, etc.) the teams are getting deep into problem understanding and their minimal viable products are getting sophisticated enough to generate detailed customer feedback; we gave them advice on how to “stand and deliver” in class; and our advanced lecture explained how to find and measure mission achievement.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)


Stand and Deliver: Preparing for Presentations
In other classes I’d normally check-in with students in the middle of the quarter / semester to hear any concerns. But in this class I don’t. Not because I don’t care, but because I know what response I’ll get in the middle of the quarter having insisted on an impossible pace while beating them with a stick. (In week 9 we’ll get the teams off the customer discovery treadmill and use that session for “reflection”. They’ll look back in awe at their own accomplishments.) This week, instead of a mid-class check-in, I give the Stand and Deliver presentation. In it I remind them what to do to prepare before each class session, tips on what to do when presenting in the class, and thoughts about opportunities after the class.

If you can’t see the presentation click here.

BTW, when I first starting teaching I noticed that teams picked the most articulate team members to give the weekly Lessons Learned presentation. And while that makes sense for a fund raising pitch, it’s the wrong model for a classroom – I want everyone to learn how to present. So each week we select a different team member to lead their team presentation. This means that even students whose first language isn’t English are up in front of the class presenting at least twice during the quarter.

Filling in the Gap: Advanced Lectures
Our advanced in-class lectures are designed to fill the knowledge gap between the on-line lectures and reading assigned for homework and the new realities of the Mission Model canvas and the DOD/IC as beneficiaries.

The goals of the weekly advanced lectures are:

  1. Define what specifically the teams need to accomplish outside of the building in the coming week to test their hypothesis for that specific part of the canvas
  2. Describe why the next part of the mission model canvas is important (to the user, organization, country, etc.)
  3. Offer specific examples of the deliverables we expect to see in their next week’s presentation as a result of their discovery

We can gauge how effective the lecture was when we see the team’s slides the next week. If the team presentations are all over the map, then our lectures were not effective. If the presentations across the teams are consistent then our lectures were on-target. This is a pretty quick way for us to tune our content.

This week some of the teams failed to present anything about last week’s buy-in lecture so it was a wakeup call that we needed to be more prescriptive in the lectures.

Pivots
A pivot is defined as a substantive change in one or more components of the mission model canvas (any of the 9 boxes). A pivot occurs after learning that your hypotheses about a specific part of the canvas are wrong. Often it’s a change in who’s the beneficiary / stakeholder / customer. Or it may be a change in the value proposition you’re delivering to those beneficiaries or it can be a substantive change in any of the 9 boxes of the canvas.

The two most important parts of a mission model canvas are the beneficiaries and the value proposition. The combination of these two is called “product/market fit.” If you’re not getting beneficiaries grabbing your value proposition out of your hands, you don’t have product/market fit.

While this sounds simple, as the teams are discovering this week, you don’t get a memo that says your hypotheses are wrong. At first you just get ambiguous data. You think hmm, perhaps I just need to talk to more people or the “right” people or just tweak the feature set. After a while you begin to realize your assumptions are incorrect, (or in this class, it’s even possible that the sponsor’s assumptions were incorrect.) It feels depressing and confusing. Finally, it dawns on you that it’s time to consider a pivot. A pivot is the lean methodology’s way to fire the plan without firing people. Pivots are what allows startups to be agile, and to move with speed and urgency.

In an actual startup, trying to complete the rest of the mission model canvas if you don’t have product/market fit is just going through the motions. Yet for the purpose of the class (versus an incubator) we do just that – we keep marching the teams through each canvas component because we want to teach them about all nine parts of the canvas. This creates cognitive dissonance for the teams – on purpose. Even though they are focused on learning about the next part of the canvas, every team continues to tenaciously search for that fit. (If we would insist they do it, it would feel like extra assigned work. When they do it on their own, it’s because it’s an obsession to solve the problem.)

This week we are seeing the typical class distribution. Several teams are in the despair, depressed and confused stage, a few are coming to the realization that it’s time to pivot, and others think they have product/market fit. It’s all part of the class. They and you will be surprised where the teams end up by the end of the class.

Team Presentations: Week 6
This week the teams’ assignment was to understand how to get “buy-in” inside their sponsors’ agency: specifically, how do they “get, keep and grow” their product inside their sponsors’ agency/command from initial interest all the way through expansion.

Aqualink started the class working to give Navy divers a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. Now they understand that the problem the divers want solved is underwater 3-D geolocation.

Week6_H4D_Aqualink buyinSlides 3-11 are a good example of what is required to go from initial Buy-in to scale in the sponsors organization.

If you can’t see the presentation click here

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gaps about surface ships from heterogeneous data.

Slide 11 is the team’s first pass at understanding what a get-keep-grow pipeline would look like. Note the details of the “get” stage – awareness, interest, consideration and purchase.

If you can’t see the presentation click here

Capella Space started class believing that launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging was their business.  Now they’ve realized that the SAR data and analytics is the business.

On slide 3 Capella gave me a reminder why Customer Discovery in this class is hard. In most other classes we insist in face-to-face interviews and if those aren’t possible high resolution video conference. This way you can read their body language and see their reactions to minimal viable products. But for some in the DOD/IC that’s not possible. The team realized that sending their MVP before the interview got them very different reactions then just conversations.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission.  Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

The team is discovering that the value proposition for the problem they are solving may not be a hardware or software solution, but perhaps could be solved by different information flows across the beneficiaries.

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing ARCYBER and others to efficiently respond and mitigate threats posed by enemy messaging.

This week the team further refined the rapid funding of R&D and prototypes through a funding mechanism called Other Transactional Authority in Slides 2-5. They further refined the org chart of who owned the problem within the DOD/IC in slide 6. They further refined their Minimal Viable Product to product/market fit in Slides 8-10.

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

Slides 3-4 are the team’s first pass at understanding what a get-keep-grow pipeline would look like. Note the details of the “get” stage – awareness, interest, consideration and purchase.  In slide 5 the team had a first demo of their MVP auto tracking of drones.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones. This week the team worked to understand what a get-keep-grow pipeline would look like in slides 5-7.  Their sponsor invited them to attend the drone conference at Fort A.P. Hill in Virginia. The team will be flying there and back in between classes.

If you can’t see the presentation click here

Advanced Lecture – Mission Achievement
Joe Felter presented the advanced lecture on Mission Achievement.

In a business the aim is to earn more money than you spend and you measure achievement/success by the revenue you bring in. In a mission-driven organization such as the defense and intelligence community, there is no revenue to measure. Instead you mobilize resources and a budget to solve a particular problem and create value for a set of beneficiaries (customers, support organizations, warfighters, Congress, the country, etc.)  So we ask the teams: how do you measure mission success/achievement for both the sum of the beneficiaries and for each individual beneficiary.

If you can’t see the presentation click here

Lessons Learned

  • The deeper teams dig into the problems some are discovering their initial hypotheses about product/market fit are wrong
    • Some are also discovering that they are adding to their sponsors understanding of the problem
  • This creates uncertainty and confusion
    • Some teams are in the “ditch of despair”
  • They all come out of it
    • with a deeper understanding of the problem and the product/market fit between the beneficiaries/value proposition
  • Many of them will pivot
    • This is what enables Lean teams to move with speed and urgency
Follow

Get every new post delivered to your Inbox.

Join 209,234 other followers

%d bloggers like this: