Hacking for Defense @ Stanford – Lessons Learned Presentations

We just held our tenth and final week of the Hacking for Defense class. Today the eight teams presented their Lessons Learned presentations.

We’re a little stunned about how well the first prototype of this class went. Over half the student teams have decided to continue working on national security projects after this class. Other colleges and universities have raised their hand and said they want to offer this at their school.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

IMG_3189


What Were Our Goals for this Class?
We had five goals for the class. First was to teach students to develop the mindset, reflexes, agility and resilience an entrepreneur needs to make decisions at speed and with urgency in a chaotic and uncertain world.

Second, we wanted to teach students entrepreneurship while they engage in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community or other government agencies.

Third was to teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there was a methodology that could help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulating the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Fourth, we wanted to show our DOD/IC sponsors that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions.

Fifth, we wanted to create the 21st Century version of Tech ROTC by having Hacking for Defense taught by a national network of 50 colleges and universities. This would give the Department of Defense (DOD) and Intelligence Community (IC) access to a pool of previously untapped technically sophisticated talent, trained in Lean and Agile methodologies, and unencumbered by dogma and doctrine. At this size the program will provide hundreds of solutions to critical national security problems every year.

The result will be a network of thousands of entrepreneurial students who understand the security threats facing the country and engaged in partnership with islands of innovation in the DOD/IC. This is a first step to a more agile, responsive and resilient, approach to national security in the 21st century.

What Did We Learn From the Class?
Not only did the students learn, but the teaching team got schooled as well.

First, we validated that students were ready and willing to sign up for a class that engaged them in national service with the Department of Defense and Intelligence Community. We had more applicants (70+) for the 32 seats in this class than we usually get in our core entrepreneurship class.

Second, we found that the islands of innovation inside the DOD and IC were willing to engage this new and eager pool of talent. We were soliciting 8 problems for the students to work on and had to shut down the submission process after we reached 25.

Third, some students took the class because they thought learning entrepreneurship with tough real-world problems would be interesting. We surveyed their motivations before and after the class and were surprised to find that a large percentage became more interested and engaged in national service. Over half the student teams have decided to continue working on national security projects.

Fourth, other schools have said they want to offer this class next year. To help kick this scale into high gear, the National Defense University will be funding Hacking for Defense at colleges and universities across the country. To train other educators and future problem sponsors we we will hold our first Hacking for Defense/Diplomacy Educators Class September 7 through 9th. Contact Pete Newell peter.newell@gc.ndu.edu to sign up.

Finally, the teaching team (instructors, TA’s, mentors) and students debriefed on our own Lessons Learned from the class. Joe Felter and his research assistants will spend the summer building out the formal educator’s guide (capturing all the “wish we would have known’s” and “here are the points you need to make in this lecture”,) sponsor guide (yep, we learned we need to train our sponsors as well), creating new DOD/IC-specific video lectures. And we will build a knowledge base of DOD/IC acquisition primers, customer development best practices, org charts, etc. Finally, for universities interested in running future courses, HackingForDefense.org will act as a central clearing house for student-ready problems that have been vetted and unclassified. While H4Di.org gets on its feet Pete Newell and his team of RA’s will continue to source problems for upcoming H4D courses.

What Surprised Us?

  1. The combination of the Mission Model Canvas and the Customer Development process was an extremely efficient template for the students to follow – even more than we expected.
  2. It drove a hyper-accelerated learning process which led the students to a “information dense” set of conclusions. (Translation: they learned a lot more, in a shorter period of time than in any other incubator, hackathon, entrepreneurship course we’ve ever taught or seen.)
  3. Insisting that the students keep a weekly blog of their customer development activities gave us insight into their progress in powerful and unexpected ways.

What Would We Change?

  1. Train the sponsors on commitment, roles, etc.
  2. Decide how we want the teams to split their time for potential dual-use products. How much time spent on focusing on the sponsors particular problem versus finding a commercial market. And what week to do so.

This is the End
Each of the eight teams presented a 2-minute video to provide context about their problem and then gave an 8-minute presentation of their Lessons Learned over the 10-weeks. Each of their slide presentation follow their customer discovery journey. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

The teams presented in front of several hundred people in person and online. You can watch the entire presentation here

https://vimeo.com/169155566

Aqualink

If you can’t see the video click here

If you can’t see the presentation click here

Capella Space

If you can’t see the video click here

If you can’t see the presentation click here

Narrative Mind

If you can’t see the video click here

If you can’t see the presentation click here

Fishreel

If you can’t see the video click here

If you can’t see the presentation click here

Sentinel

If you can’t see the video click here

If you can’t see the presentation click here

Skynet

If you can’t see the video click here

If you can’t see the presentation click here

Right of Boom

If you can’t see the video click here

If you can’t see the presentation click here

Guardian 

If you can’t see the video click here

If you can’t see the presentation click here

It Takes a Village
While I authored these blog posts, the class was truly a team project. The teaching team consisted of:

  • Tom Byers, Professor of Engineering and Faculty Director, STVP
  • Joe Felter a retired Army Special Forces Colonel with research and teaching appointments at Stanford’s Center for International Security and Cooperation (CISAC), the Hoover Institution, and the dept. of Management Science and Engineering
  • Jackie Space a former Air Force officer who as an aerospace engineer developed joint satellite and electronic warfare programs. She is currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and Managing Partner at at BMNT Partners
  • Pete Newell is a former retired Army Colonel currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and CEO of BMNT Partners.

Kim Chang was our lead teaching assistant. We were lucky to get a team of 25 mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams.

Of course, a huge thanks to the 32 Stanford students who suffered through the 1.0 version of the class.

And finally a special thanks to our course advisor Bill Perry, former Secretary of Defense and Professor Emeritus, Chris Zember, Director, National Defense University – Center for Technology & National Security Policy, Jay Harrison, Director, National Defense University – National Security Technology Accelerator, Dr Malcolm Thompson, the executive Director of NextFlex, the Flexible Hybrid Electronics Manufacturing Innovation Institute, The entire Defense Innovation Unit Experimental (DIUX), Center for International Security and Cooperation (CISAC) in the Freeman Spogli Institute for International Studies, STVP in the department of Management Science and Engineering.

Hacking for Defense will be offered again at Stanford University next Winter.  See you there!

Hacking for Defense @ Stanford – Weeks 8 and 9

We just held our eighth and ninth weeks of the Hacking for Defense class. Now with over 917 interviews of beneficiaries (users, program managers, stakeholders, etc.), the teams spent the last two weeks learning what activities, resources and partners they would need to actually deliver their solution. And they’re getting a handle on what it costs to build a company to deliver it.

Understanding the left-side of the mission model canvas (activities, resources, partners, and costs) forces all teams to ask, “Are we building a product for a DOD/IC customer only or do we have a “dual-use” product that could be sold commercially and get funded by venture capital?”

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

Next week the teams will present their final Lessons Learned presentations.

Two Items for the Bucket List
Two bucket list items got ahead of my blogging so I’ve combined the final two lecture weeks of the class into this one blog post.

Four decades ago my first job in Silicon Valley was with ESL, the first company to combine computers and signals intelligence. The founder of this 1964 Silicon Valley startup was Bill Perry. His work at ESL made him one of the 10 founders of National Reconnaissance.

Dr. Perry eventually became the 19th secretary of defense. But a decade earlier as undersecretary of defense for research and engineering, he was the father of the second offset strategy using software and semiconductors to build smart weapons, smart sensors, and stealth aircraft that helped end the Cold War.

Last week I interviewed Bill at Stanford about War and Peace, innovation and entrepreneurship.

http://ecorner.stanford.edu/videos/4255/Dedication-to-Innovation-and-Nation-Entire-Talk

Second:
I also gave the commencement speech at the NYU engineering school here.

The Left side of the Canvas
If you’ve been reading along so far, you know that this class is not an extended hackathon nor is it a 10-week long incubator. Hackathons and incubators are helpful in getting product teams focused and result in great demos, but you’re left still not knowing whether you have something beneficiaries/stakeholders/users want nor do you know what it takes to deploy the solution to the field. Ultimately you are left without a strategy to turn your idea into a solution that people will use.

Using the Lean LaunchPad methodology our teams do much more than just build a product or understand customer problems/needs. They also learn how to deploy the solution, how to get stakeholder buy-in and how to measure success. And in these last two weeks of class, they learn what activities, resources and partners they’ll need to deliver their solution and derive what it costs to build the company to deliver it.

The teams capture their work in the mission model canvas a framework for each week’s activities. The canvas illustrates the search for the unknowns that new ventures face. The 9 boxes of the canvas visualize all the components needed to turn beneficiaries needs/problems into a solution.

Mission Model Canvas by week
Each week the teams marched through another box of the canvas, testing their hypotheses in front of beneficiaries using the customer development methodology, all while building and updating their minimal viable product. It’s a ton of work. Over the course of the class, each team will have talked to 100 beneficiaries/ stakeholders/ users. The result is evidence-based entrepreneurship.

Team Presentations: Weeks 8 and 9
Over these last two weeks, teams began to figure out the activities, resources and partners their company would need to deliver their value proposition (product, service or both) to the beneficiaries in their sponsor organizations.

Activities are the expertise and resources that the company needs to deliver the value proposition. They might be hardware development, software expertise, manufacturing, launching rockets, funding, etc. Resources are the internal company-owned activities. Examples are a company-owned manufacturing facility, big data or machine learning engineers, DOD proposal writers, venture capital, etc.  Partners are the external resources (third parties) necessary to execute the Activities. i.e. outsourced manufacturing, system integrators, etc. other companies, that will provide those activities.

activities resources and partners

In addition, teams worked on understanding the costs and operations and deployment timelines for delivering the product to their sponsor.

finance and ops timeline

Team Dynamics
In these last three weeks the benefit of having a team of mixed business and technical resources becomes apparent. Teams that are just all technologists quickly grasp product/market fit (the right side of the canvas) but often have a hard time understanding the left side of the canvas (activities, resources, partners and costs.)  When the technologists work together with business focused students as a team, the learning is impressive.

However, the downside is that one of failure modes of teams (and startups) is a team that doesn’t jell. One of the symptoms is technologists going heads-down building product and features without customer input while they defer all of the left-side of the canvas to the business team. Or conversely business team members draw timelines and costs without a deep understanding of the technology hurdles.

Almost every class has a team or two that goes through team conflict – different working styles, different time commitments, pivots taking them to places where they’re no longer interested, etc. Given that 1/4 of startups meltdown over team dynamics before funding, seeing this happen to teams in the class isn’t a surprise. We treat team dynamics as a normal part of learning in the class. (Team members get to grade each other on their contributions as part of their final grade.)

Considering that none of these teams have worked together in the past, the amount of synergy and teamwork in this cohort is impressive.

Skynet

WEEK 8 Presentation

In slide 2 the Skynet team continued with customer discovery using experiments to validate or invalidate their hypotheses. Slide 5 does a good job of separating out their technical versus business activities. Slide 6 did a great job in connecting the activities to the resources and partners they’ll need.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

In Slide 2 the team made progress on developing their MVP. In slide 3 they realized some of their conclusions about DARPA partnerships from last week were wrong. Slides 5-8 continued their learning about partnerships, and slides 9-11 are a great first pass on costs and financial and operations timeline.

If you can’t see the week 9 presentation click here

Aqualink

WEEK 8 Presentation

Slide 5 is a good summary of activities/resources/partners. Slide 6 connects those to the prototyping and deployment activities by partner and sponsor. Slide 7 lays out a potential field deployment schedule to the sponsor organization. Slides 11-14 show their continued testing of their MVP underwater.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 7 and 9 is the team’s first pass in understanding costs, operations and fundraising. They continued their MVP development underwater in a pool at Stanford.

If you can’t see the week 9 presentation click here

Sentinel

WEEK 8 Presentation

The team really got out of the building and traveled to San Diego (at their own expense) and visited the USS Sampson and the 3rd Fleet headquarters. Slide 7 summarizes their activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slide 13 is an excellent example of mapping out their costs.  Slide 14 is a great example of diagramming their financial and operating milestones.

If you can’t see the week 9 presentation click here

Capella

WEEK 8 Presentation

This week Capella was so engaged in their customer discovery and pivot to illegal fishing, they missed the assignment.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 8 -12 illustrates their activities and costs. Because they missed last week’s assignment, you wouldn’t know from their presentation that they required a partnership with a space launch company :-)  The good news is this team had been distracted and will have news to share in their Lessons Learned presentation

If you can’t see the week 9 presentation click here

Guardian

WEEK 8 Presentation

Slides 4 -6 summarized Guardians activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 4 -6 summarized their costs and operating plan.

If you can’t see the week 9 presentation click here

Right of Boom

WEEK 8 Presentation

Slides 4 -6 summarized Right of Booms’ activities, resources and partners.

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slide 3-5 summarizes their unique findings. This team discovered that their deliverable to the sponsor will not be a product. Instead it will be a series of recommendations on how to better utilize their existing products and data. Slides 6-8 describe the partners which can best deliver these recommendations to their sponsor.

If you can’t see the week 9 presentation click here

Narrative Mind

Slides 3 -6 summarized their activities, resources and partners

WEEK 8 Presentation

If you can’t see the week 8 presentation click here

WEEK 9 Presentation

Slides 3 -10 further refined their partners and summarized their costs and operating plan.

If you can’t see the week 9 presentation click here

Advanced Lecture 8: Costs
In week 8 Pete Newell gave the costs lecture and put it in the context of a DOD program. Slide 3 defined what costs were, slides 4-11 tied it to a specific example.

If you can’t see the costs lecture click here

Advanced Lecture 9: Reflections
In past versions of this class teams would call on beneficiaries/customers until the last week of the class and then present their Lessons Learned. The good news is that their presentations were dramatically better than those given at demo days – they showed us what they learned over 8 weeks which gave us a clear picture of the velocity and trajectory of the teams. The bad news is since their heads were down working on customer discovery until the very end, they had no time to reflect on the experience.

We realized that we had been so focused in packing content and work into the class, we failed to give the students time to step back and think about what they actually learned.

So now we use the last week of the class as a reflection week. Our goal—to have the students extract the insights and meaning from the work they had done in the previous seven weeks.

We asked each team to prepare a draft Lessons Learned presentation telling us about their journey and showing us their:

  • Initial sponsor problem statement
  • Quotes from beneficiaries that illustrated learnings and insights
  • Pivot stories
  • Screen shots of the evolution of Minimum Viable Product (MVP)
  • Demo of final MVP

The teaching team reviewed the drafts and provided feedback to the teams and to the class as a whole. We discussed what general patterns and principles they extracted from all the customer interaction they had.

Dual-use Products
As you’ll see next week in the final presentations, some of the teams discovered that they could best serve their sponsor by building a commercial off-the-shelf product that could be sold widely and bought by the DOD/Intel community. Pete Newell came up with the best diagram I’ve seen to illustrate how the work the teams were doing in this class fit to do just that.

The diagram shows that during the class the sponsor needs drive customer discovery and product/market fit. But continued discovery would now include commercial customers and eventually those commercial customer needs would drive the feature set.

dual use trajectory

Hacking for Defense Educators Class
The H4D instructor team has been busy capturing what we learned (teams, lectures, sponsors, etc.) and we’ll incorporate the lessons from this inaugural course and revise the course materials. As part of our plan to scale this class nationwide to other schools, we’re writing an educator’s guide and offering a Hacking for Defense Educators Class Sept 7th – 9th.

Details in the next post.

Tomorrow, May 31st is the last day of class.

We’ll post the final presentations. Quite a journey for all these teams and their sponsors!

Hacking for Defense @ Stanford – Week 7

We just held our seventh week of the Hacking for Defense class. Now with over 750 interviews of beneficiaries (users, program managers, stakeholders, etc.) almost all the teams are beginning to pivot from their original understanding of their sponsor’s problem and their hypotheses about how to solve them. Minimal viable products are being demo’d to sponsors and sponsors are reacting to what the teams are learning. This week teams figured out how to measure mission achievement and success, and our advanced lectures were on activities, resources and partners.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

—-

Why Innovation in Government Is Hard
As we spend more time with the military services, commands and agencies it’s apparent that getting disruptive innovation implemented in the DOD/IC face the same barriers as large corporations (and a few more uniquely theirs.)

The first barrier to innovation is the Horizon 1 leadership conundrum. In corporations, the CEO and executives have risen through the ranks for their skill on executing existing programs/missions. The same is true in most DOD/IC organizations: leadership has been promoted through the ranks for their ability to execute existing programs/missions. By the time they reach the top, they are excellent managers of processes and procedures needed to deliver a consistent and repeatable execution of the current core mission (and typically excellent political players as well.)

These horizon 1 leaders are exactly who you want in place when the status quo prevails – and when competitors / adversaries react as per our playbook.

To these Horizon 1 leader’s, innovation is often considered an extension of what they already do today. In companies this would be product line extensions, more efficient supply chain, new distribution channels. In the DOD/IC innovation is often more technology, more planes, more aircraft carriers, more satellites, etc.

This “more and better” approach works until they meet adversaries – state and non-state – who don’t follow our game plan – adversaries who use asymmetry to offset and degrade our technological or numerical advantages – roadside bombs, cyberattacks, hybrid warfare, anti-access/area denial (A2/AD), etc.

disruption by adversaries

History tells us that what gets you promoted in peacetime
causes you to lose in wartime.

When Horizon 1 leaders set up innovation groups the innovators at the bottom of the organization start cheering. Meanwhile the middle of the organization strangles every innovation initiative. Why? Most often four points of failure occur:

  1. Horizon 1 leaders tend to appoint people who they feel comfortable with – Horizon 1 or perhaps Horizon 2 managers. This results not in innovation, but in Innovation Theater – lots of coffee cups, press releases, incubators and false hopes, but no real disruptive changes. Horizon 3 organizations require Horizon 3 leadership (with Horizon 1 second in command.)
  2. There needs to be effective communication about what being innovative means to different parts of their organizations as well as defining (and enforcing) their expectations for middle management. How do middle mangers know how to make trade-offs between the efficiency requirements of their Horizon 1 activities and the risks required of a Horizon 3 activity?
  3. They have to create incentives for middle management leaders to take on horizon three ideas
  4. They have to change the metrics across the entire organization. If not, then the effectiveness of the Horizon 3 effort will be graded using Horizon 1 metrics

Secretary of Defense Carter’s recent pivot to place the DOD’s innovation outpost – DIUx directly under his supervision after 8 months is a great example of a leader enforcing his expectations about innovation.

In peacetime Horizon 3/disruptive groups need to be led by Mavericks, sponsored and protected by Horizon 1 leadership. It is this group, challenging the dogma of the existing programs, who will come up with the disruptive/asymmetric offset technologies and strategies.

both types of leadership 2

BTW, history tells us that in war time the winners filled this innovation role with people who make most Horizon 1 leaders very uncomfortable – Churchill in WWII, Billy Mitchell, Oppenheimer on the Manhattan Project, Vannevar Bush at the OSRD, John Boyd, etc.

More next week on innovation and the intransigent middle. Now back to the class. 

Team Presentations: Week 7
In a company you know you’ve been successful when you generate revenue and profit. But in the military success has different metrics. This week the teams’ assignment was to understand what Mission Achievement and/or Mission Success looked like for each of their sponsor organizations and each of the beneficiaries inside that organization.

Later in the class some of the team will realize they can build “dual-use” products (building their product primarily for civilian use but also sold to the military.) In those case revenue will become an additional metric.

Understanding how to measure mission achievement/success for each beneficiary is the difference between a demo and a deployed solution.

Sentinel initially started by trying to use low-cost sensors to monitor surface ships for their 7th fleet sponsor in a A2/AD environment. The team pivoted and has found that their mission value is really to enable rapid, well-informed decisions by establishing a common maritime picture from heterogeneous data.

Sentinel displayOn Slide 4-5 the team continues testing their hypotheses via customer discovery. Note that they plan a trip to San Diego to visit the customer. And they realized that an unclassified proxy for their data is the IUU fishing problem. (With a great assist from the State Departments innovation outpost in Silicon Valley.) Their Minimum Viable Product can be seen on slides 12-16 using this illegal fishing data.

Slide 10 summarized what mission achievement would look like for three beneficiaries in the 7th fleet.

If you can’t see the presentation click here

Capella Space started class believing that launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging was their business. Now they’ve realized that the SAR data and analytics is the business.  Then the question was, “For whom?”

In slides 4- 11 they describe what they learned about illegal fishing in Indonesia (Thanks again to the State Departments innovation outpost.) But the big idea on slide 12 – 13 is that Capella has pivoted. The team realized that there are many countries that want to detect boats at night. And most of the countries of interest are located in the equatorial belt. Slide 14 is their rough outline of mission achievement for the key agencies/countries.

Interesting to note that Capella Space and Team Sentinel seem to be converging on the same problem space!

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing the U.S. Army Cyber Command (ARCYBER) and others to efficiently respond and mitigate threats posed by enemy messaging.

In slide 4 the team provided a textbook definition of mission achievement. They specified what success looks like for each of the beneficiaries inside of their sponsor, ARCYBER. In slide 5 they broadly outlined mission achievement for three private sector markets.

In slides 6-9 they plotted all the potential adversary communication trends on social media problems, and in slide 7 overlaid that problem space with existing commercial solutions. Slides 8 and 9 show the problems not yet solved by anyone, and slide 9 further refines the specific problems this team will solve.

NarrativeMind further refined their Minimal Viable Product to product/market fit in Slides 11-16.

If you can’t see the presentation click here

Aqualink started the class working to give Navy divers in the Naval Special Warfare Group a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. A few weeks ago they pivoted, realizing that the high-value problem the divers want solved is underwater 3-D geolocation.

Slide 2, John Boyd and the OODA Loop (finally!) makes an appearance in the class. (The OODA loops and the four steps of Customer Development and the Lean Methodology are rooted in the same “get of the building/get eyes out of your cockpit” and “speed and urgency” concepts.) In Slides 5-7 Aqualink’s two versions of their Minimum Viable Product are beginning to be outlined and in Slide 8, the team passed around physical mockups of the buoy.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones for the Army’s Asymmetric Warfare Group.

The team certainly got out of the building this week. In between their classes they flew to the east coast and attended the Army’s Asymmetric Warfare Drone Demo-Day at Fort A.P. Hill in Virginia. They spoke to lots of vendors and got a deep understanding of currently deployed tactical drones.

Slides 5-9 show their substantial progress in their Minimal Viable Product as they demo’d advanced detection and classification capabilities. They are beginning to consider whether they should pivot to become a drone software platform.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission.  Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

This week the team was actually able to talk to a key beneficiary on the front lines overseas. What they discovered is that the JIDA current technical solutions, if combined, will provide a solution of equal quality to standalone development in a shorter timeframe.

On slide 4 they outlined their Mission Achievement / Success criteria for the key JIDA beneficiaries.  Slide 9 continued to refine their understanding of the tradespace.

If you can’t see the presentation click here

Skynet is using drones to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

Mission achievement on slide 2 needs a bit of explanation; the team has met and exceeded their basic goals to reach: 80% accuracy on target identification. From SOCOM’s perspective the team has achieved their initial mission. Now Skynet has moved beyond their original scope into an interesting area. Slide 9 and 10 show their further refinement of buy in- for SOCOM and the Border Patrol.

If you can’t see the presentation click here

Advanced Lecture – Activities, Resources and Partners
Pete Newell presented the advanced lecture on Activities, Resources and Partners.

Activities are the expertise and resources that the company needs to deliver the value proposition. Resources are the internal company-owned activities. Examples are a company-owned manufacturing facility, big data or machine learning engineers, DOD proposal writers, venture capital, etc. Partners are the external resources necessary to execute the Activities. i.e. outsourced manufacturing, system integrators, etc.

If you can’t see the presentation click here

Lessons Learned

  • History tells us that what gets you promoted in peacetime causes you to lose in wartime
  • Teams are making substantive pivots on their understanding of the real sponsor problem and pivoting on their proposed solution
  • Understanding how to measure mission achievement/success for each beneficiary is the difference between a demo and a deployed solution

Hacking for Defense @ Stanford – Week 6

We just held our sixth week of the Hacking for Defense class. Now with over 660 interviews of beneficiaries (users, program managers, stakeholders, etc.) the teams are getting deep into problem understanding and their minimal viable products are getting sophisticated enough to generate detailed customer feedback; we gave them advice on how to “stand and deliver” in class; and our advanced lecture explained how to find and measure mission achievement.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)


Stand and Deliver: Preparing for Presentations
In other classes I’d normally check-in with students in the middle of the quarter / semester to hear any concerns. But in this class I don’t. Not because I don’t care, but because I know what response I’ll get in the middle of the quarter having insisted on an impossible pace while beating them with a stick. (In week 9 we’ll get the teams off the customer discovery treadmill and use that session for “reflection”. They’ll look back in awe at their own accomplishments.) This week, instead of a mid-class check-in, I give the Stand and Deliver presentation. In it I remind them what to do to prepare before each class session, tips on what to do when presenting in the class, and thoughts about opportunities after the class.

If you can’t see the presentation click here.

BTW, when I first starting teaching I noticed that teams picked the most articulate team members to give the weekly Lessons Learned presentation. And while that makes sense for a fund raising pitch, it’s the wrong model for a classroom – I want everyone to learn how to present. So each week we select a different team member to lead their team presentation. This means that even students whose first language isn’t English are up in front of the class presenting at least twice during the quarter.

Filling in the Gap: Advanced Lectures
Our advanced in-class lectures are designed to fill the knowledge gap between the on-line lectures and reading assigned for homework and the new realities of the Mission Model canvas and the DOD/IC as beneficiaries.

The goals of the weekly advanced lectures are:

  1. Define what specifically the teams need to accomplish outside of the building in the coming week to test their hypothesis for that specific part of the canvas
  2. Describe why the next part of the mission model canvas is important (to the user, organization, country, etc.)
  3. Offer specific examples of the deliverables we expect to see in their next week’s presentation as a result of their discovery

We can gauge how effective the lecture was when we see the team’s slides the next week. If the team presentations are all over the map, then our lectures were not effective. If the presentations across the teams are consistent then our lectures were on-target. This is a pretty quick way for us to tune our content.

This week some of the teams failed to present anything about last week’s buy-in lecture so it was a wakeup call that we needed to be more prescriptive in the lectures.

Pivots
A pivot is defined as a substantive change in one or more components of the mission model canvas (any of the 9 boxes). A pivot occurs after learning that your hypotheses about a specific part of the canvas are wrong. Often it’s a change in who’s the beneficiary / stakeholder / customer. Or it may be a change in the value proposition you’re delivering to those beneficiaries or it can be a substantive change in any of the 9 boxes of the canvas.

The two most important parts of a mission model canvas are the beneficiaries and the value proposition. The combination of these two is called “product/market fit.” If you’re not getting beneficiaries grabbing your value proposition out of your hands, you don’t have product/market fit.

While this sounds simple, as the teams are discovering this week, you don’t get a memo that says your hypotheses are wrong. At first you just get ambiguous data. You think hmm, perhaps I just need to talk to more people or the “right” people or just tweak the feature set. After a while you begin to realize your assumptions are incorrect, (or in this class, it’s even possible that the sponsor’s assumptions were incorrect.) It feels depressing and confusing. Finally, it dawns on you that it’s time to consider a pivot. A pivot is the lean methodology’s way to fire the plan without firing people. Pivots are what allows startups to be agile, and to move with speed and urgency.

In an actual startup, trying to complete the rest of the mission model canvas if you don’t have product/market fit is just going through the motions. Yet for the purpose of the class (versus an incubator) we do just that – we keep marching the teams through each canvas component because we want to teach them about all nine parts of the canvas. This creates cognitive dissonance for the teams – on purpose. Even though they are focused on learning about the next part of the canvas, every team continues to tenaciously search for that fit. (If we would insist they do it, it would feel like extra assigned work. When they do it on their own, it’s because it’s an obsession to solve the problem.)

This week we are seeing the typical class distribution. Several teams are in the despair, depressed and confused stage, a few are coming to the realization that it’s time to pivot, and others think they have product/market fit. It’s all part of the class. They and you will be surprised where the teams end up by the end of the class.

Team Presentations: Week 6
This week the teams’ assignment was to understand how to get “buy-in” inside their sponsors’ agency: specifically, how do they “get, keep and grow” their product inside their sponsors’ agency/command from initial interest all the way through expansion.

Aqualink started the class working to give Navy divers a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. Now they understand that the problem the divers want solved is underwater 3-D geolocation.

Week6_H4D_Aqualink buyinSlides 3-11 are a good example of what is required to go from initial Buy-in to scale in the sponsors organization.

If you can’t see the presentation click here

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gaps about surface ships from heterogeneous data.

Slide 11 is the team’s first pass at understanding what a get-keep-grow pipeline would look like. Note the details of the “get” stage – awareness, interest, consideration and purchase.

If you can’t see the presentation click here

Capella Space started class believing that launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging was their business.  Now they’ve realized that the SAR data and analytics is the business.

On slide 3 Capella gave me a reminder why Customer Discovery in this class is hard. In most other classes we insist in face-to-face interviews and if those aren’t possible high resolution video conference. This way you can read their body language and see their reactions to minimal viable products. But for some in the DOD/IC that’s not possible. The team realized that sending their MVP before the interview got them very different reactions then just conversations.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission.  Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

The team is discovering that the value proposition for the problem they are solving may not be a hardware or software solution, but perhaps could be solved by different information flows across the beneficiaries.

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing ARCYBER and others to efficiently respond and mitigate threats posed by enemy messaging.

This week the team further refined the rapid funding of R&D and prototypes through a funding mechanism called Other Transactional Authority in Slides 2-5. They further refined the org chart of who owned the problem within the DOD/IC in slide 6. They further refined their Minimal Viable Product to product/market fit in Slides 8-10.

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

Slides 3-4 are the team’s first pass at understanding what a get-keep-grow pipeline would look like. Note the details of the “get” stage – awareness, interest, consideration and purchase.  In slide 5 the team had a first demo of their MVP auto tracking of drones.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones. This week the team worked to understand what a get-keep-grow pipeline would look like in slides 5-7.  Their sponsor invited them to attend the drone conference at Fort A.P. Hill in Virginia. The team will be flying there and back in between classes.

If you can’t see the presentation click here

Advanced Lecture – Mission Achievement
Joe Felter presented the advanced lecture on Mission Achievement.

In a business the aim is to earn more money than you spend and you measure achievement/success by the revenue you bring in. In a mission-driven organization such as the defense and intelligence community, there is no revenue to measure. Instead you mobilize resources and a budget to solve a particular problem and create value for a set of beneficiaries (customers, support organizations, warfighters, Congress, the country, etc.)  So we ask the teams: how do you measure mission success/achievement for both the sum of the beneficiaries and for each individual beneficiary.

If you can’t see the presentation click here

Lessons Learned

  • The deeper teams dig into the problems some are discovering their initial hypotheses about product/market fit are wrong
    • Some are also discovering that they are adding to their sponsors understanding of the problem
  • This creates uncertainty and confusion
    • Some teams are in the “ditch of despair”
  • They all come out of it
    • with a deeper understanding of the problem and the product/market fit between the beneficiaries/value proposition
  • Many of them will pivot
    • This is what enables Lean teams to move with speed and urgency

Hacking for Defense @ Stanford – Week 5

We just held our fifth week of the Hacking for Defense class. This week the teams passed the half way mark in the class. They’ve collectively talked to over 550 beneficiaries (users, program managers, stakeholders, etc.) Their focus for this week was to figure out how to get products rapidly deployed into their sponsors organization. Our advanced lecture explained how to get buy-in for your solution by creating an insurgency among your supporters and advocates.

(This post is a continuation of the series. See all the H4D posts here (and also read Pete Newell’s weekly class summaries here.) Because of the embedded presentations this post is best viewed on the website.)


This is not a typical class
If you’ve been reading these weekly blogs, you’ve seen by now this is not a typical class.

The class is a combination of theory and intensive practice. First and foremost, it is experiential and hands-on. The teams live and die by the Lean Startup credo: “There are no facts inside the building so get the hell outside.” That’s why, just halfway through the class, they’ve already talked to 550 beneficiaries (users, program managers, stakeholders, etc.)

The Lean Methodology requires teams to abandon their preconceived notions of how one builds startups and solve problems – The class is designed to break students out of that all too common mindset that they understand customer’s problems, can design a solution and want to get right to work on building it – all without contact with the stakeholders, users, decision makers, etc.

After decades of teaching I have found that getting students to really change these beliefs cannot be done with reading, case studies or in-class simulations – at least not in the short time we have them in the class. If we really want them to understand how to efficiently and rapidly understand and solve customer problems, we needed to immerse them with customers on day one.

And if we want them to understand what life outside the classroom in an early stage venture will look like, then they need to experience chaos, conflicting data, uncertainty and good-enough decision making for 10 confusing weeks.

We start by pushing the teams incredibly hard to set the pace (and wash out any of those who can’t work at this pace.) Teams hit the class running. Before the first class, each team has already spoken to 10 customers, and they are challenged to present their mission model canvases within 20 minutes of walking through the classroom door. Within 5 minutes from the first time a team starts to present, they get hit with “relentlessly direct” critiques.

If you can’t see the presentation click here

But by week 5, (this week) the teams have either embraced the Lean process or we’re not going to get through to them. So at this point in the class we begin to dial down the tone and tenor of the comments, and over the next four weeks become their cheerleaders rather than their taskmasters.

In week 9 we’ll stop and use the week and class for “reflection”. We’ve found that getting the teams off the customer discovery treadmill at this point helps them to look back and reflect on what they’ve really learned, not just about their product/customers but more importantly about the lean processes, themselves, and team work.

Team Presentations: Week 5
This week the teams were primarily trying to answer how products get from demo to deployment in their sponsors agency.

In all team presentations, note that their new learnings each week are highlighted on their Mission Model canvas in red.

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gaps about surface ships from heterogeneous data.

Slides 4-27 is one of the best illustrations (actually an animation) of how all the beneficiaries work and interact. Slide 29-33 is their detailed drill-down on how their solution could get acquired and deployed in the Navy.
Sentinel Mission Model

Slide 34 summarizes their current Mission Model canvas. Notice that each beneficiary has a matching value proposition. These relationships are expanded in detail on slides 35-38.

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing  the U.S. Army Cyber Command (ARCYBER) and others to efficiently respond and mitigate threats posed by enemy messaging.

This week, in slide 3, the team further refined where ARCYBER sat in the org chart of who owned the problem within the DOD/IC and the  acquisition process.

They learned about getting rapid funding of R&D and prototypes through a funding mechanism called Other Transactional Authority in slide 4. They further refined their Minimal Viable Product to product/market fit in Slides 7 and 8. Their Mission Model canvas in slide 9 has an updated set of beneficiaries now refined in the Value Proposition canvases in slides 10 – 17.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones. This week the team worked to understand a day-in-the-life of a Forward Operating Base (FOB) documenting the roles of the captain, lieutenant and a private on a guard tower (slides 5 and 6). They worked on understanding how they would get a counter drone solution deployed through the Army’s Asymmetric Warfare Group (slides 7-9).

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

The team validated several critical hypotheses about technology and acquisition (slide 2), further refined their Minimum Viable Product (slide 3) and really dug into the path of getting a solution acquired and deployed in the Special Operations Command (SOCOM) in slides 4 – 9.  Their summary of their learning is highlighted on their Mission Model canvas on Slide 10.  Like Fishreel’s analysis, Skynet’s detailed Value Proposition canvases (slides 11 – 16) are also case studies on how to get to a deep understanding of the problems of all the beneficiaries and stakeholders in an organization.

If you can’t see the presentation click here

Capella Space is launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging.

Capella Space is pivoting towards commercial customers based on consistent feedback from the government. They’re finding that the government doesn’t want to pay for launch and sustainment, but instead would rather be users of commercial data.

One of the previous weeks’ hypotheses was that the Coast Guard would want and need synthetic aperture radar images and data (slide 5.) The team did extensive customer discovery at the Coast Guard 11th district command center and seemed find that the problem was not all that acute.  Depressing – but great learning. They continue to believe that the National Geospatial Agency may be a potential DOD customer (slide 9), but are struggling to find the people to talk to.

If you can’t see the presentation click here

Aqualink started the class working to give Navy divers a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. Their customer discovery helped them to understand that there were three beneficiaries (slide 2): the operators (SEAL divers), medical officers and medical researchers.  Slides 4 and slide 5 show their decision process to focus on the more immediate problem – underwater geolocation. Slides 6 and 7 describe the organization of SOCOM and how products get deployed to the divers. Slide 8, their mission model canvas, highlights their new learnings about deployment and beneficiaries in red. Slides 15-17 diagrams their understanding of the product acquisition and deployment process.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission. Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

This week their customer discovery got them to the current lead in Iraq for the Joint iED Defeat Agency (JIDA) (slide 2). This helped them refine their map of how IED information flows (slide 6).  They mapped how to get a product deployed in JIDA in slide 9. This is leading them to a Minimal Viable Product neither they nor JIDA expected in slide 15.

If you can’t see the presentation click here

Advanced Presentation Supporters and Advocates- Creating an Insurgency
The advanced lecture for this week explained how to create “buy-in” among all the beneficiaries.  Pete Newell described how to use anecdotes, artifacts and story-telling to create an insurgency among the advocates for your solution.

If you can’t see the presentation click here

Team Learning Updates
A few of the teams independently starting writing weekly one-page status reports to their sponsors and mentors. This was a great idea. It keeps the sponsors and mentors informed and makes them feel they’re part of the team.

Hacking for Defense Goes National
This week three Universities announced their intention to offer Hacking For Defense this fall –  The University of California at San Diego (UCSD), the University of Pittsburg, and the University of Rochester/ Rochester Institute for Technology.

As we scale the program, the DOD/IC sponsors have requested they have a single point of contact for the soliciting and validating the problems. So we’ll have a single site acting as the “sponsor challenge clearinghouse” for the schools that will be teaching the course.

We’ll have more to say about scaling the program, funding and the Hacking for Defense educators course in later blog posts.

Lessons Learned

  • The teams are learning what it takes to turn a demo into a deployable solution that gets to the field
  • The teams and sponsors are both learning how to accurately define the problem(s)
  • This learning will save time, money and lives

Hacking for Defense @ Stanford – Week 4

We just held our fourth week of the Hacking for Defense class. This week the teams turned the corner on understanding beneficiaries and finding product/market fit. The 8 teams spoke to 115 beneficiaries (users, program managers, etc.); we sent each team a critique of their mission model canvas; we started streaming the class live to DOD/IC sponsors and other educators; our advanced lecture explained how to go from concept to deployment in the DOD/IC; and we watched as the students got closer to understanding the actual problems their customers have.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)


Beneficiaries equals all the stakeholders
In-between class sessions, we reviewed each team’s mission model canvas and sent them a detailed critique of each of the boxes on the right side of their canvas. The critiques seemed to make a difference in this week’s presentations with a noticeable improvement in teams’ beneficiaries/stakeholder understanding. The teams are beginning to understand that beneficiaries mean “Not the name of an organization but all the stakeholders in an organization (users, program managers, saboteurs, legal, finance, etc.)” and that they can’t really understand customer problems until they can diagram the relationships among all the beneficiaries. Then, and only then, can they move on to developing a detailed value proposition canvas for each of the beneficiaries.

Some of the sponsors commented that the teams had a better grasp of the problem space and a deeper understanding of the beneficiaries and their relationships to each other, than they did.

Team Presentations: Week 4
Great technical teams like often want to use the class as a product incubator when we want them to spend an equal amount of time learning about the rest of the Mission Model canvas.

What we’re trying to prevent is to have teams give the DOD/IC yet another great technology demo. They have plenty of those. What teams need to do is deeply understand all the stakeholders in their sponsor organization (analysts, seniors, finance, legal, etc.) so they can get a great product that solves real problems and can be widely deployed quickly.

Narrative Mind had an amazing week. The sponsor’s brief to the team is to figure out how to understand, disrupt, and counter adversaries’ use of social media. After 46 interviews the team could see that there were conflicting definitions of what problems needed to be solved. They realized that different beneficiaries were each describing a different part of a much bigger picture. Take a look at slide 3, as the team synthesized and then summarized the sum of the hypotheses the beneficiaries have of the problem. This was a big learning moment. Slide 4 was another insight as they mapped out who actually owned the problem across multiple DOD and Intelligence organizations. Finally, their beneficiaries on slide 6 were focused and clear. This team is learning a lot.

If you can’t see the presentation click here 

Right of Boom had an insightful week with 19 customer discovery interviews this week across a broad range of beneficiaries. (See slide 2.) These interviews led them to conclude that their initial hypotheses (slides 3-5) were wrong. In slide 6 they were able to map out the entire IED (Improvised Explosive Device) reporting information flow.

Week_4_H4D_Right_Of_Boom Info flowAnd in slide 7 the team really narrows down their beneficiaries and value proposition. They came to an interesting conclusion about how to measure success in their Mission Achievement box.

If you can’t see the presentation click here

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gap about surface ships.

The team started by diagraming the relationships among their beneficiaries (slide 2). They realized that this is just a start. Now they need to overlay the surface ships’ intelligence information flow shown in slides 16 & 19 on top of this org chart. Slides 3-6 are a good narrative of hypotheses validated, invalidated and refined during the week. Slides 8-11 are an excellent example of a deep understanding of the beneficiaries. Their Minimum Viable Product in slides 12-14 this week shows much more problem insight compared to the prior week (slides 18-21.)

If you can’t see the presentation click here

aquaLink started the week believing they were working to give Navy divers a system of wearable devices that records data critical to diver health and safety, and makes the data actionable through real-time alerts and post-dive analytics.

This was a great but painful week for the team as they experienced a bit of an existential crisis while working to drill down into who their customer truly is. The original problem statement from their sponsor asked for a wearable sensor that would monitor the physiological status of divers. As they proceeded with customer discovery, the team found that the majority of the operators who would wear these sensors were ambivalent about the introduction of a vitals monitoring platform, but were much more excited about solving geolocation problems. On the other hand, the medical professionals and some commanders were more interested in monitoring physiological metrics in order to understand chronic long-term health issues facing divers and optimize short-term performance. Slides 2-6 illustrate aquaLink’s evolving understanding of the range of customer archetypes.

Their key take-away was that they would have to decide which beneficiary to focus on. They decided to focus on the operators and divers within SEAL Delivery Vehicle Team One, along with their immediate chains of command in SDVT-1 and Naval Special Warfare Group 3. These were the beneficiaries who viewed aquaLink’s focus on geolocation as the most valuable. See slides 7, 11 and 12.  The team recognized that it was time for a pivot and aquaLink will spend the rest of the class focusing exclusively on geolocation.

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

The team made progress understanding the Special Operations Command (SOCOM) acquisitions process in slides 3-5 and mocked-up an MVP. However, they still list organizations as beneficiaries.  We asked that they dive deeper into the each of the stakeholders and create a diagram of how the beneficiaries actually interact.

If you can’t see the presentation click here

Capella Space is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

The team made progress understanding that some beneficiaries want raw SAR imagery and some want analytics. They are starting to understand the beneficiaries in the Coast Guard; however, they are stymied in trying to find the right people to talk to about commercial data acquisition at the National Geospatial Agency. We asked that they dive deeper into each of the stakeholders and diagram how the beneficiaries actually interact.

If you can’t see the presentation click here

Guardian’s problem to solve was to counter asymmetric drone activities.  This week was a big leap forward in truly understanding their problem and beneficiaries. They did a deep dive (slides 5-7) into really understanding what, exactly, is a forward operating base. They refined their options of the problem space (slide 4) and did a great job of truly understanding the workflow in slide 8. Their mission model canvas in slide 9 had a great update on their beneficiaries while the detailed value proposition canvases in slides 10-12 gave great insight about the pains/gains/jobs to be done those beneficiaries had. 

If you can’t see the presentation click here

Advanced Lecture:  Concept to Deployment in the DOD
This week Jackie Space and Lauren Schmidt gave the advanced lecture. Jackie, an exAir Force officer who spent her career managing overhead reconnaissance systems, flies up from LA every week and has now officially joined the teaching team. Lauren is a member of the Defense Innovation Unit Experimental (DIUx) based at Moffett Field in Mountain View and advises our students in the course along with multiple other members of the DIUx.

Slide 5 “purchasing authority” and Slide 6 “key activities” were real eye-openers for the team.

If you can’t see the presentation click here

Team Learnings
A few of the teams are now writing weekly one-page status reports to their sponsors and mentors. Great idea to keep them informed and make them feel they’re part of the team.

It’s been fun to watch as the teams learn from sponsors; a few teams have been broadening their sponsors understanding of the problem. (How do we know this? When the sponsors asked their team, “Can we use your slides to present to our organization?”) That’s a win for everyone.

This week we had one group of students volunteer to go to Iraq or Afghanistan to see the customer problem first-hand. Travel restrictions and other logistical challenges will likely make this trip infeasible, but the team’s genuine interest in getting to the ground level of customer discovery reflects well on their commitment to the principles of the course’s methodology.

Lessons Learned

  • Civilian students with no prior DOD experience can be taught to deeply understand military and intelligence problems and organizations in 4 weeks
  • These students are passionate and committed to solving problems that protect the homeland and keep Americans safe around the world

Hacking for Defense (H4D) @ Stanford – Week 3

We just held our third week of the Hacking for Defense class. This week the 8 teams spoke to 108 beneficiaries (users, program mangers, etc.), we held a Customer Discovery workshop, we started streaming the class live to DOD/IC sponsors and other educators, our advanced lecture was on Product/Market fit for the DOD/IC and we watched as the students solved their customer discovery obstacles and started getting closer to their customers.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

—–

Customer Discovery in the DOD/IC Workshop
We normally hold a Customer Discovery workshop during the evening the first week of the class. But spring break and the “How to Work with the DOD” workshop got in the way. So we inserted an abbreviated version at the front of this week’s class.

When working with the DOD/IC there are some unique obstacles of “getting out of the building and talking to customers”. For example, members of the DOD will not respond to ”cold calls” and those in the Intel community won’t even tell you their names. In addition, most of the sponsors are working on classified problems. So how do teams understand the customer when the customer can’t tell you what they do? The Workshop talked about how to address those and other Discovery issues.

If you can’t see presentation click here

Team Presentations: Week 3
After the Customer Discovery workshop the 8 teams presented what their hypotheses, experiments and what they learned outside the building this week.

Team Right of Boom (previously named Live Tactical Threat Toolkit) is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission. The team originally was developing tech-centric tools for foreign teams to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.  Now they are honing in providing accurate high-volume post-incident IED reporting.

Last week this team was floundering. They had confused getting interviews and building minimal viable products with truly trying to “become the customer.” We strongly suggested that there was no way that could understand the day in the life of an explosive ordnance disposal expert by just listening to them – they needed to stand in their shoes. So to their immense credit the team suited up in full bomb disposal gear and got of the building. They earned our respect (and a name change for the team.)

If you can’t see the Right to Boom video click here (turn up the volume!)

If you can’t see the Right to Boom presentation click here

 

Team Capella Space
  is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

This week the team learned a ton. They mapped out competitive offerings, found that Government funding is not the proper channel for Capella, but did find that the Coast guard is currently in dire need of situational awareness at high resolution and that military customers want access to raw data; commercial customers highly value processed data for actionable insights

If you can’t see the Team Capella Space presentation click here

Team aquaLink is working to give Navy divers a way to monitor their own physiological conditions while underwater (core temperature, maximum dive pressure, blood pressure and pulse.) Knowing all of this would give divers early warning of hypothermia or the bends.

This week they validated that divers will want real-time alerts regarding vitals (and put up with the additional gear/procedures) of issues that threaten mission success. The found that navy medical researchers want data on vitals, the rebreather (air consumption), and the dive computer (dive profile). Their hypotheses going forward are that a heads up display is the ideal form of information transmission during a dive and system should be modular to allow for the integration of evolving technology (geolocation and communication)

If you can’t see the Team aquaLink presentation click here

Team Guardian is working to protect soldiers from cheap, off-the-shelf commercial drones. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons?

Guardians current hypotheses is that they have to provide drone detection, identification and protection against attacks from drones or swarm of drones. And that the user will be a 19 solider not trained to use complex equipment.

If you can’t see the Team Guardian presentation click here

Team Narrative Mind is trying to understand, disrupt, and counter adversaries’ use of social media. Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

The team is coalescing around the idea that the two minimal viable products for their sponsor are, 1) automatically generate an organizational chart of a target terrorist groups over time, and 2) generate a social network map of how terrorist groups interact with each other.

If you can’t see the Team Narrative Mind presentation click here

Team Sentinel initially started by trying to use low cost sensors to monitor surface ships in a A2/AD environment.

The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gap about surface ships in an A2/AD environment via:

  1. Increased number of data streams (i.e. incorporate open source data)
  2. Automated data aggregation (i.e. from disparate sources) and analysis
  3. Enhanced intel through contextualization
  4. Improved UI/UX

If you can’t see the Sentinel presentation click here

Team Skynet is also using drones to to provide ground troops situational awareness. (Almost the inverse of Team Guardian.)

The team invalidated the hypotheses that military/commercial systems exist that could already solve the problem. In addition, they originally believed that soldiers on foot needed a deployable drone system. They discovered that drones are best used with teams with vehicles or for short ranged dismounted operations.

If you can’t see the Team Skynet presentation click here

Advanced Lecture: Product/Market fit in the DOD/IC
The advanced lecture for week 3 was on the unique needs of finding Product/Market fit in the DOD/IC. Pete Newell described why a solutions in the DOD fails and then described “battlefield calculus” – how two identical sounding missions (and their inherent problems) are actually radically different based on what echelon of force executes them, by the size of force, their location, even by how well they are trained.  Despite the obvious, people still try to deliver “one-size-fits-all” solutions. To properly insure a solution is actually used it is important to become familiar with the pattern of life of the user and their unit.

Pete also pointed out that teams need to “Look for Conflict” between what may have been provided to solve a similar problem and the solution the teams are about to recommend. You needed to ask: Are the circumstances similar? Or are their a myriad of conditions present that will invalidate what was a good solution under different circumstances?

If you can’t see the presentation click here

Mission Model and Value Proposition Canvas
To students, “who are the beneficiaries?” feels fuzzy on day one. And given most of them had no exposure to the DOD or Intel Community it’s not a surprise. The reason we have the teams talk to 10-15 people every week, is that with enough data they can begin to fill in the details. A few of our guests have commented how knowledgeable the teams were in talking about the sponsor organizations and problems.

That said, listening to the team presentations there was a wide difference between teams in how well they understood that the definition of “beneficiaries.”  Many of teams were still listing names of organizations rather than the title and archetype of the people who mattered/cared/decided/users, etc.

Understanding who are the beneficiaries is critical to understanding the rest of the mission model canvas.

When the students have a more nuanced understanding who are the individual beneficiaries is when they can build a detailed Value Proposition Canvas for each beneficiary that makes sense.  (Several teams had Value Proposition Canvas of organizations, some had fewer Proposition Canvas than they had beneficiaries, some Proposition Canvases were so generic it was clear that had insufficient data on individual needs of specific archetypes, etc.)

This is all par for the course and part of the student learning. We now need to sharpen their focus.

An after class action for the teaching team is to read through every team’s week 3 presentation slide-by-slide and give each team a detailed, written, box-by-box critique of the right-side of their Mission Model and Value Proposition Canvas.  We want to help them get this right.

Sponsor Education – a Network Begins to Form
The teaching team, liaisons, mentors and DIUx are all working their networks to get students relevant beneficiaries to talk to. (More about what a wonderful asset DIUx has been in a future post.) Joe and Pete are continuing to work hard on educating the sponsors about their role. (We are collecting all our learning in an Educators Guide so other universities can run the class.)

One emerging unexpected benefit, is that Pete and Joe are continuing to expand the network of innovators in the DOD/IC who are helping our student teams. I’ve had several critique our presentations and offer suggestions on the nuanced parts of the IC mission and acquisition system I didn’t understand.

Live Streaming the Class
The DOD/IC sponsors who gave us these problems were curious about how the teams were learning so rapidly. (Others in their commands and agencies wanted to watch as well.) So this week we began to live-stream the student presentations. And other universities who want to offer this class have begun to have their educators watch the class. (We’ll be offering a train-the-trainer educators class later this year.)

Lessons Learned from Week 3

  • Teams still running at full speed
  • Understanding beneficiaries is critical to understanding the rest of the mission model canvas.
    • Written team-by-team offline critique is needed to keep them on course
  • Support is coming from lots of places in the DOD/IC
    • DIUx and our liasons have been great in connecting the students
Follow

Get every new post delivered to your Inbox.

Join 208,445 other followers