Hacking for Defense @ Stanford – Week 5

We just held our fifth week of the Hacking for Defense class. This week the teams passed the half way mark in the class. They’ve collectively talked to over 550 beneficiaries (users, program managers, stakeholders, etc.) Their focus for this week was to figure out how to get products rapidly deployed into their sponsors organization. Our advanced lecture explained how to get buy-in for your solution by creating an insurgency among your supporters and advocates.

(This post is a continuation of the series. See all the H4D posts here (and also read Pete Newell’s weekly class summaries here.) Because of the embedded presentations this post is best viewed on the website.)


This is not a typical class
If you’ve been reading these weekly blogs, you’ve seen by now this is not a typical class.

The class is a combination of theory and intensive practice. First and foremost, it is experiential and hands-on. The teams live and die by the Lean Startup credo: “There are no facts inside the building so get the hell outside.” That’s why, just halfway through the class, they’ve already talked to 550 beneficiaries (users, program managers, stakeholders, etc.)

The Lean Methodology requires teams to abandon their preconceived notions of how one builds startups and solve problems – The class is designed to break students out of that all too common mindset that they understand customer’s problems, can design a solution and want to get right to work on building it – all without contact with the stakeholders, users, decision makers, etc.

After decades of teaching I have found that getting students to really change these beliefs cannot be done with reading, case studies or in-class simulations – at least not in the short time we have them in the class. If we really want them to understand how to efficiently and rapidly understand and solve customer problems, we needed to immerse them with customers on day one.

And if we want them to understand what life outside the classroom in an early stage venture will look like, then they need to experience chaos, conflicting data, uncertainty and good-enough decision making for 10 confusing weeks.

We start by pushing the teams incredibly hard to set the pace (and wash out any of those who can’t work at this pace.) Teams hit the class running. Before the first class, each team has already spoken to 10 customers, and they are challenged to present their mission model canvases within 20 minutes of walking through the classroom door. Within 5 minutes from the first time a team starts to present, they get hit with “relentlessly direct” critiques.

If you can’t see the presentation click here

But by week 5, (this week) the teams have either embraced the Lean process or we’re not going to get through to them. So at this point in the class we begin to dial down the tone and tenor of the comments, and over the next four weeks become their cheerleaders rather than their taskmasters.

In week 9 we’ll stop and use the week and class for “reflection”. We’ve found that getting the teams off the customer discovery treadmill at this point helps them to look back and reflect on what they’ve really learned, not just about their product/customers but more importantly about the lean processes, themselves, and team work.

Team Presentations: Week 5
This week the teams were primarily trying to answer how products get from demo to deployment in their sponsors agency.

In all team presentations, note that their new learnings each week are highlighted on their Mission Model canvas in red.

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gaps about surface ships from heterogeneous data.

Slides 4-27 is one of the best illustrations (actually an animation) of how all the beneficiaries work and interact. Slide 29-33 is their detailed drill-down on how their solution could get acquired and deployed in the Navy.
Sentinel Mission Model

Slide 34 summarizes their current Mission Model canvas. Notice that each beneficiary has a matching value proposition. These relationships are expanded in detail on slides 35-38.

If you can’t see the presentation click here

NarrativeMind is developing tools that will optimize discovery and investigation of adversary communication trends on social media, allowing  the U.S. Army Cyber Command (ARCYBER) and others to efficiently respond and mitigate threats posed by enemy messaging.

This week, in slide 3, the team further refined where ARCYBER sat in the org chart of who owned the problem within the DOD/IC and the  acquisition process.

They learned about getting rapid funding of R&D and prototypes through a funding mechanism called Other Transactional Authority in slide 4. They further refined their Minimal Viable Product to product/market fit in Slides 7 and 8. Their Mission Model canvas in slide 9 has an updated set of beneficiaries now refined in the Value Proposition canvases in slides 10 – 17.

If you can’t see the presentation click here

Guardian is trying to counter asymmetric threats from commercial drones. This week the team worked to understand a day-in-the-life of a Forward Operating Base (FOB) documenting the roles of the captain, lieutenant and a private on a guard tower (slides 5 and 6). They worked on understanding how they would get a counter drone solution deployed through the Army’s Asymmetric Warfare Group (slides 7-9).

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

The team validated several critical hypotheses about technology and acquisition (slide 2), further refined their Minimum Viable Product (slide 3) and really dug into the path of getting a solution acquired and deployed in the Special Operations Command (SOCOM) in slides 4 – 9.  Their summary of their learning is highlighted on their Mission Model canvas on Slide 10.  Like Fishreel’s analysis, Skynet’s detailed Value Proposition canvases (slides 11 – 16) are also case studies on how to get to a deep understanding of the problems of all the beneficiaries and stakeholders in an organization.

If you can’t see the presentation click here

Capella Space is launching a constellation of synthetic aperture radar (SAR) satellites into space to provide real-time radar imaging.

Capella Space is pivoting towards commercial customers based on consistent feedback from the government. They’re finding that the government doesn’t want to pay for launch and sustainment, but instead would rather be users of commercial data.

One of the previous weeks’ hypotheses was that the Coast Guard would want and need synthetic aperture radar images and data (slide 5.) The team did extensive customer discovery at the Coast Guard 11th district command center and seemed find that the problem was not all that acute.  Depressing – but great learning. They continue to believe that the National Geospatial Agency may be a potential DOD customer (slide 9), but are struggling to find the people to talk to.

If you can’t see the presentation click here

Aqualink started the class working to give Navy divers a system of wearable devices that records data critical to diver health and safety and makes the data actionable through real-time alerts and post-dive analytics. Their customer discovery helped them to understand that there were three beneficiaries (slide 2): the operators (SEAL divers), medical officers and medical researchers.  Slides 4 and slide 5 show their decision process to focus on the more immediate problem – underwater geolocation. Slides 6 and 7 describe the organization of SOCOM and how products get deployed to the divers. Slide 8, their mission model canvas, highlights their new learnings about deployment and beneficiaries in red. Slides 15-17 diagrams their understanding of the product acquisition and deployment process.

If you can’t see the presentation click here

Right of Boom is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission. Now they are developing systems, workflows, and incentives for allied foreign militaries with the goal of improved intelligence fidelity.

This week their customer discovery got them to the current lead in Iraq for the Joint iED Defeat Agency (JIDA) (slide 2). This helped them refine their map of how IED information flows (slide 6).  They mapped how to get a product deployed in JIDA in slide 9. This is leading them to a Minimal Viable Product neither they nor JIDA expected in slide 15.

If you can’t see the presentation click here

Advanced Presentation Supporters and Advocates- Creating an Insurgency
The advanced lecture for this week explained how to create “buy-in” among all the beneficiaries.  Pete Newell described how to use anecdotes, artifacts and story-telling to create an insurgency among the advocates for your solution.

If you can’t see the presentation click here

Team Learning Updates
A few of the teams independently starting writing weekly one-page status reports to their sponsors and mentors. This was a great idea. It keeps the sponsors and mentors informed and makes them feel they’re part of the team.

Hacking for Defense Goes National
This week three Universities announced their intention to offer Hacking For Defense this fall –  The University of California at San Diego (UCSD), the University of Pittsburg, and the University of Rochester/ Rochester Institute for Technology.

As we scale the program, the DOD/IC sponsors have requested they have a single point of contact for the soliciting and validating the problems. So we’ll have a single site acting as the “sponsor challenge clearinghouse” for the schools that will be teaching the course.

We’ll have more to say about scaling the program, funding and the Hacking for Defense educators course in later blog posts.

Lessons Learned

  • The teams are learning what it takes to turn a demo into a deployable solution that gets to the field
  • The teams and sponsors are both learning how to accurately define the problem(s)
  • This learning will save time, money and lives

Hacking for Defense @ Stanford – Week 4

We just held our fourth week of the Hacking for Defense class. This week the teams turned the corner on understanding beneficiaries and finding product/market fit. The 8 teams spoke to 115 beneficiaries (users, program managers, etc.); we sent each team a critique of their mission model canvas; we started streaming the class live to DOD/IC sponsors and other educators; our advanced lecture explained how to go from concept to deployment in the DOD/IC; and we watched as the students got closer to understanding the actual problems their customers have.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)


Beneficiaries equals all the stakeholders
In-between class sessions, we reviewed each team’s mission model canvas and sent them a detailed critique of each of the boxes on the right side of their canvas. The critiques seemed to make a difference in this week’s presentations with a noticeable improvement in teams’ beneficiaries/stakeholder understanding. The teams are beginning to understand that beneficiaries mean “Not the name of an organization but all the stakeholders in an organization (users, program managers, saboteurs, legal, finance, etc.)” and that they can’t really understand customer problems until they can diagram the relationships among all the beneficiaries. Then, and only then, can they move on to developing a detailed value proposition canvas for each of the beneficiaries.

Some of the sponsors commented that the teams had a better grasp of the problem space and a deeper understanding of the beneficiaries and their relationships to each other, than they did.

Team Presentations: Week 4
Great technical teams like often want to use the class as a product incubator when we want them to spend an equal amount of time learning about the rest of the Mission Model canvas.

What we’re trying to prevent is to have teams give the DOD/IC yet another great technology demo. They have plenty of those. What teams need to do is deeply understand all the stakeholders in their sponsor organization (analysts, seniors, finance, legal, etc.) so they can get a great product that solves real problems and can be widely deployed quickly.

Narrative Mind had an amazing week. The sponsor’s brief to the team is to figure out how to understand, disrupt, and counter adversaries’ use of social media. After 46 interviews the team could see that there were conflicting definitions of what problems needed to be solved. They realized that different beneficiaries were each describing a different part of a much bigger picture. Take a look at slide 3, as the team synthesized and then summarized the sum of the hypotheses the beneficiaries have of the problem. This was a big learning moment. Slide 4 was another insight as they mapped out who actually owned the problem across multiple DOD and Intelligence organizations. Finally, their beneficiaries on slide 6 were focused and clear. This team is learning a lot.

If you can’t see the presentation click here 

Right of Boom had an insightful week with 19 customer discovery interviews this week across a broad range of beneficiaries. (See slide 2.) These interviews led them to conclude that their initial hypotheses (slides 3-5) were wrong. In slide 6 they were able to map out the entire IED (Improvised Explosive Device) reporting information flow.

Week_4_H4D_Right_Of_Boom Info flowAnd in slide 7 the team really narrows down their beneficiaries and value proposition. They came to an interesting conclusion about how to measure success in their Mission Achievement box.

If you can’t see the presentation click here

Sentinel initially started by trying to use low-cost sensors to monitor surface ships in a A2/AD environment. The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gap about surface ships.

The team started by diagraming the relationships among their beneficiaries (slide 2). They realized that this is just a start. Now they need to overlay the surface ships’ intelligence information flow shown in slides 16 & 19 on top of this org chart. Slides 3-6 are a good narrative of hypotheses validated, invalidated and refined during the week. Slides 8-11 are an excellent example of a deep understanding of the beneficiaries. Their Minimum Viable Product in slides 12-14 this week shows much more problem insight compared to the prior week (slides 18-21.)

If you can’t see the presentation click here

aquaLink started the week believing they were working to give Navy divers a system of wearable devices that records data critical to diver health and safety, and makes the data actionable through real-time alerts and post-dive analytics.

This was a great but painful week for the team as they experienced a bit of an existential crisis while working to drill down into who their customer truly is. The original problem statement from their sponsor asked for a wearable sensor that would monitor the physiological status of divers. As they proceeded with customer discovery, the team found that the majority of the operators who would wear these sensors were ambivalent about the introduction of a vitals monitoring platform, but were much more excited about solving geolocation problems. On the other hand, the medical professionals and some commanders were more interested in monitoring physiological metrics in order to understand chronic long-term health issues facing divers and optimize short-term performance. Slides 2-6 illustrate aquaLink’s evolving understanding of the range of customer archetypes.

Their key take-away was that they would have to decide which beneficiary to focus on. They decided to focus on the operators and divers within SEAL Delivery Vehicle Team One, along with their immediate chains of command in SDVT-1 and Naval Special Warfare Group 3. These were the beneficiaries who viewed aquaLink’s focus on geolocation as the most valuable. See slides 7, 11 and 12.  The team recognized that it was time for a pivot and aquaLink will spend the rest of the class focusing exclusively on geolocation.

If you can’t see the presentation click here

Skynet is using drones to to provide ground troops with situational awareness – helping prevent battlefield fatalities by pinpointing friendly and enemy positions.

The team made progress understanding the Special Operations Command (SOCOM) acquisitions process in slides 3-5 and mocked-up an MVP. However, they still list organizations as beneficiaries.  We asked that they dive deeper into the each of the stakeholders and create a diagram of how the beneficiaries actually interact.

If you can’t see the presentation click here

Capella Space is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

The team made progress understanding that some beneficiaries want raw SAR imagery and some want analytics. They are starting to understand the beneficiaries in the Coast Guard; however, they are stymied in trying to find the right people to talk to about commercial data acquisition at the National Geospatial Agency. We asked that they dive deeper into each of the stakeholders and diagram how the beneficiaries actually interact.

If you can’t see the presentation click here

Guardian’s problem to solve was to counter asymmetric drone activities.  This week was a big leap forward in truly understanding their problem and beneficiaries. They did a deep dive (slides 5-7) into really understanding what, exactly, is a forward operating base. They refined their options of the problem space (slide 4) and did a great job of truly understanding the workflow in slide 8. Their mission model canvas in slide 9 had a great update on their beneficiaries while the detailed value proposition canvases in slides 10-12 gave great insight about the pains/gains/jobs to be done those beneficiaries had. 

If you can’t see the presentation click here

Advanced Lecture:  Concept to Deployment in the DOD
This week Jackie Space and Lauren Schmidt gave the advanced lecture. Jackie, an exAir Force officer who spent her career managing overhead reconnaissance systems, flies up from LA every week and has now officially joined the teaching team. Lauren is a member of the Defense Innovation Unit Experimental (DIUx) based at Moffett Field in Mountain View and advises our students in the course along with multiple other members of the DIUx.

Slide 5 “purchasing authority” and Slide 6 “key activities” were real eye-openers for the team.

If you can’t see the presentation click here

Team Learnings
A few of the teams are now writing weekly one-page status reports to their sponsors and mentors. Great idea to keep them informed and make them feel they’re part of the team.

It’s been fun to watch as the teams learn from sponsors; a few teams have been broadening their sponsors understanding of the problem. (How do we know this? When the sponsors asked their team, “Can we use your slides to present to our organization?”) That’s a win for everyone.

This week we had one group of students volunteer to go to Iraq or Afghanistan to see the customer problem first-hand. Travel restrictions and other logistical challenges will likely make this trip infeasible, but the team’s genuine interest in getting to the ground level of customer discovery reflects well on their commitment to the principles of the course’s methodology.

Lessons Learned

  • Civilian students with no prior DOD experience can be taught to deeply understand military and intelligence problems and organizations in 4 weeks
  • These students are passionate and committed to solving problems that protect the homeland and keep Americans safe around the world

Hacking for Defense (H4D) @ Stanford – Week 3

We just held our third week of the Hacking for Defense class. This week the 8 teams spoke to 108 beneficiaries (users, program mangers, etc.), we held a Customer Discovery workshop, we started streaming the class live to DOD/IC sponsors and other educators, our advanced lecture was on Product/Market fit for the DOD/IC and we watched as the students solved their customer discovery obstacles and started getting closer to their customers.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

—–

Customer Discovery in the DOD/IC Workshop
We normally hold a Customer Discovery workshop during the evening the first week of the class. But spring break and the “How to Work with the DOD” workshop got in the way. So we inserted an abbreviated version at the front of this week’s class.

When working with the DOD/IC there are some unique obstacles of “getting out of the building and talking to customers”. For example, members of the DOD will not respond to ”cold calls” and those in the Intel community won’t even tell you their names. In addition, most of the sponsors are working on classified problems. So how do teams understand the customer when the customer can’t tell you what they do? The Workshop talked about how to address those and other Discovery issues.

If you can’t see presentation click here

Team Presentations: Week 3
After the Customer Discovery workshop the 8 teams presented what their hypotheses, experiments and what they learned outside the building this week.

Team Right of Boom (previously named Live Tactical Threat Toolkit) is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission. The team originally was developing tech-centric tools for foreign teams to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.  Now they are honing in providing accurate high-volume post-incident IED reporting.

Last week this team was floundering. They had confused getting interviews and building minimal viable products with truly trying to “become the customer.” We strongly suggested that there was no way that could understand the day in the life of an explosive ordnance disposal expert by just listening to them – they needed to stand in their shoes. So to their immense credit the team suited up in full bomb disposal gear and got of the building. They earned our respect (and a name change for the team.)

If you can’t see the Right to Boom video click here (turn up the volume!)

If you can’t see the Right to Boom presentation click here

 

Team Capella Space
  is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

This week the team learned a ton. They mapped out competitive offerings, found that Government funding is not the proper channel for Capella, but did find that the Coast guard is currently in dire need of situational awareness at high resolution and that military customers want access to raw data; commercial customers highly value processed data for actionable insights

If you can’t see the Team Capella Space presentation click here

Team aquaLink is working to give Navy divers a way to monitor their own physiological conditions while underwater (core temperature, maximum dive pressure, blood pressure and pulse.) Knowing all of this would give divers early warning of hypothermia or the bends.

This week they validated that divers will want real-time alerts regarding vitals (and put up with the additional gear/procedures) of issues that threaten mission success. The found that navy medical researchers want data on vitals, the rebreather (air consumption), and the dive computer (dive profile). Their hypotheses going forward are that a heads up display is the ideal form of information transmission during a dive and system should be modular to allow for the integration of evolving technology (geolocation and communication)

If you can’t see the Team aquaLink presentation click here

Team Guardian is working to protect soldiers from cheap, off-the-shelf commercial drones. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons?

Guardians current hypotheses is that they have to provide drone detection, identification and protection against attacks from drones or swarm of drones. And that the user will be a 19 solider not trained to use complex equipment.

If you can’t see the Team Guardian presentation click here

Team Narrative Mind is trying to understand, disrupt, and counter adversaries’ use of social media. Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

The team is coalescing around the idea that the two minimal viable products for their sponsor are, 1) automatically generate an organizational chart of a target terrorist groups over time, and 2) generate a social network map of how terrorist groups interact with each other.

If you can’t see the Team Narrative Mind presentation click here

Team Sentinel initially started by trying to use low cost sensors to monitor surface ships in a A2/AD environment.

The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gap about surface ships in an A2/AD environment via:

  1. Increased number of data streams (i.e. incorporate open source data)
  2. Automated data aggregation (i.e. from disparate sources) and analysis
  3. Enhanced intel through contextualization
  4. Improved UI/UX

If you can’t see the Sentinel presentation click here

Team Skynet is also using drones to to provide ground troops situational awareness. (Almost the inverse of Team Guardian.)

The team invalidated the hypotheses that military/commercial systems exist that could already solve the problem. In addition, they originally believed that soldiers on foot needed a deployable drone system. They discovered that drones are best used with teams with vehicles or for short ranged dismounted operations.

If you can’t see the Team Skynet presentation click here

Advanced Lecture: Product/Market fit in the DOD/IC
The advanced lecture for week 3 was on the unique needs of finding Product/Market fit in the DOD/IC. Pete Newell described why a solutions in the DOD fails and then described “battlefield calculus” – how two identical sounding missions (and their inherent problems) are actually radically different based on what echelon of force executes them, by the size of force, their location, even by how well they are trained.  Despite the obvious, people still try to deliver “one-size-fits-all” solutions. To properly insure a solution is actually used it is important to become familiar with the pattern of life of the user and their unit.

Pete also pointed out that teams need to “Look for Conflict” between what may have been provided to solve a similar problem and the solution the teams are about to recommend. You needed to ask: Are the circumstances similar? Or are their a myriad of conditions present that will invalidate what was a good solution under different circumstances?

If you can’t see the presentation click here

Mission Model and Value Proposition Canvas
To students, “who are the beneficiaries?” feels fuzzy on day one. And given most of them had no exposure to the DOD or Intel Community it’s not a surprise. The reason we have the teams talk to 10-15 people every week, is that with enough data they can begin to fill in the details. A few of our guests have commented how knowledgeable the teams were in talking about the sponsor organizations and problems.

That said, listening to the team presentations there was a wide difference between teams in how well they understood that the definition of “beneficiaries.”  Many of teams were still listing names of organizations rather than the title and archetype of the people who mattered/cared/decided/users, etc.

Understanding who are the beneficiaries is critical to understanding the rest of the mission model canvas.

When the students have a more nuanced understanding who are the individual beneficiaries is when they can build a detailed Value Proposition Canvas for each beneficiary that makes sense.  (Several teams had Value Proposition Canvas of organizations, some had fewer Proposition Canvas than they had beneficiaries, some Proposition Canvases were so generic it was clear that had insufficient data on individual needs of specific archetypes, etc.)

This is all par for the course and part of the student learning. We now need to sharpen their focus.

An after class action for the teaching team is to read through every team’s week 3 presentation slide-by-slide and give each team a detailed, written, box-by-box critique of the right-side of their Mission Model and Value Proposition Canvas.  We want to help them get this right.

Sponsor Education – a Network Begins to Form
The teaching team, liaisons, mentors and DIUx are all working their networks to get students relevant beneficiaries to talk to. (More about what a wonderful asset DIUx has been in a future post.) Joe and Pete are continuing to work hard on educating the sponsors about their role. (We are collecting all our learning in an Educators Guide so other universities can run the class.)

One emerging unexpected benefit, is that Pete and Joe are continuing to expand the network of innovators in the DOD/IC who are helping our student teams. I’ve had several critique our presentations and offer suggestions on the nuanced parts of the IC mission and acquisition system I didn’t understand.

Live Streaming the Class
The DOD/IC sponsors who gave us these problems were curious about how the teams were learning so rapidly. (Others in their commands and agencies wanted to watch as well.) So this week we began to live-stream the student presentations. And other universities who want to offer this class have begun to have their educators watch the class. (We’ll be offering a train-the-trainer educators class later this year.)

Lessons Learned from Week 3

  • Teams still running at full speed
  • Understanding beneficiaries is critical to understanding the rest of the mission model canvas.
    • Written team-by-team offline critique is needed to keep them on course
  • Support is coming from lots of places in the DOD/IC
    • DIUx and our liasons have been great in connecting the students

Hacking for Defense (H4D) @ Stanford – Week 2

We just held our second week of the Hacking for Defense class. This week the 8 teams spoke to 106 beneficiaries (users, program mangers, etc.), we held a DOD/IC 101 workshop, our advanced lecture was on the Value Proposition Canvas, and we watched as the students ran into common customer discovery obstacles and found new ones.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)


DOD/IC 101 – Workshop
We started the week by holding a Monday night workshop – DOD/IC 101. Our goal was to give the students with no military background a tutorial on the challenges facing DoD/IC in the current asymmetric threat environment, how the DOD/IC defines its missions and specifies the products it needs, how it accomplishes these missions and how they get to their ultimate user. This knowledge will help the students understand the overall environment that their Mission Model Canvas is operating in.

We posted the slides here and more important, an annotated narrative for each of the slides here. It’s truly a landmark presentation. Even if you think you know how the DOD works, read the narrative alongside the slides. I learned a lot.

If you can’t see the presentation click here

you can’t see the narrative click here.

Hacking for Defense: Week 2
The second week started with the 8 teams presenting what they learned in their first full week of class.

Capella Space
Team Capella is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

This week the team tested whether other beneficiaries – the Coast Guard and the Oil and Gas industry might be interested in their solution. Great learning.

If you can’t see the presentation click here

 

Live Tactical Threat Toolkit (LTTT)
Team LTTT (Live Tactical Threat Toolkit) is trying to enhance the capacity of  foreign military explosive ordnance disposal (EOD) teams to accomplish their mission. The team is developing tech informed options for these teams to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.

The team did a good job in starting to diagram the customer workflow and intends to gain an appreciation for the ground user challenges in accomplishing these types of missions in this weeks customer discovery efforts.

If you can’t see the presentation click here

Narrative Mind
Team Narrative Mind is trying to understand, disrupt, and counter adversaries’ use of social media. Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

The team did a good job in starting to diagram the customer workflow and their understanding of how to prioritize MVP features.

If you can’t see the presentation click here

Skynet
Team Skynet is also using drones to to provide ground troops situational awareness. (Almost the inverse of Team Guardian.)

Their Mission Model Canvas had a ton of learning, and their MVP engendered a lot of conversation from those who’ve been in combat and were familiar with the challenges of maintaining situational awareness under fire.

If you can’t see the presentation click here

Aqualink
Team aquaLink is working to give Navy divers a way to monitor their own physiological conditions while underwater (core temperature, maximum dive pressure, blood pressure and pulse.) Knowing all of this would give divers early warning of hypothermia or the bends.

In the first week of the class this team was suiting up in full navy diving gear and doing customer discovery by spending an hour in the life of the beneficiary. They did their homework.
Aqualink suiting up

If you can’t see the presentation click here

Guardian
Team Guardian is working to protect soldiers from cheap, off-the-shelf commercial drones. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons? This team is actively working to identify viable responses to these  battlefield inevitabilities.

If you can’t see the presentation click here

Sentinel
Team Sentinel is trying to use low cost sensors to monitor surface ships in a A2/AD environment. The team appreciates that the problem include the sensors as well as the analytics of the sensor data.

Really good summary of hypotheses, experiments, results and action.

If you can’t see the presentation click here

Customer Discovery and the Flipped Classroom – Learnings
After talking to teams in office hours (the teaching team meets every team for 20 minutes every week,) and watching teams present, and then seeing a team send a sponsor an email that read like a bad business school sales pitch, we realized some students had skipped their homework/and or still hadn’t grasped the basics of Customer Discovery.

As a reminder, we run the class as a “flipped classroom” – the lectures – the basics of Customer Discovery and the Mission Model Canvas – are homework watched on Udacity and on Vimeo. It was painfully clear that many of the students hadn’t done their homework. We plan to remedy that in our next week class, warning the students that we will be cold calling on them to show us what they learned.

Some teams did their homework and understood that customer discovery meant “becoming the customer.” For example, the team solving a problem for Navy divers managed to get the Navy to suit them up in full diving regalia. On the other hand, some teams thought that customer discovery simply meant interviewing people and building a minimal viable products. For example, we suggested to the team working on solutions for defusing Improvised Explosive Devices (IEDs) that to truly understand their customer needs might require them to get close to the dirt with some explosive ordnance disposal teams. (Looking ahead we have no doubt that this team will respond aggressively to instructor feedback and suit up in Explosive Ordinance Disposal (EOD) equipment as part of their customer discovery efforts for week 3. Stay tuned.)

Part of the student confusion about customer discovery was the fault of the teaching team. We normally hold a “How to Do Customer Discovery” evening workshop, but we got caught by a tight spring break schedule and we punted this workshop to hold the DOD/IC workshop. In hindsight it was a bad idea – we should have found a way to hold both. We will remedy that by giving an abbreviated workshop first thing next week in the classroom.

All of these were problems we’ve seen before and we’re course correcting quickly to solve them.  But, given the new form of the class we had a few problems we hadn’t encountered.

First, some teams were stymied by the classified nature of the specific data sets they thought they needed to understand the customer problem and build MVPs. In every case, what they lacked was a deep understanding of the customer problem. Which simply required going back to the basics of customer discovery.

Second, a few teams were truly blocked by a few sponsors who were also having a difficult time understanding the role they played in Customer Discovery and required follow up clarification by the teaching team and H4D military liaison officers.

Sponsor Education – Learnings
A few DOD sponsors believed they were not only the gatekeepers to the problem but were the sole source of information for our teams. Given they were supposed to maximize the number of beneficiaries the teams were supposed to talk to, the teaching team jumped on this and rapidly addressed it.

In another case the sponsor so narrowly defined the problem that it was viewed by the team as providing incremental changes to a solution they already have. After discussion the sponsor agreed that the team should focus on the realm of possible and how they would address the problem if there was not a current solution in place and in the process define new plans for how the solutions could be used.

In other cases a few of our sponsors had difficulty generating the leads and contacts within their own ecosystems that were necessary to sustain our teams’ customer discovery beyond the sponsor’s primary contacts. Ultimately teams are required to interview 80-120 beneficiaries, advocates and stakeholders (customers). This is a heavy lift if the sponsor has not thought through who those people are and where they will be found.

Finally, one of our problem sponsors departed their organization and was replaced by an alternate. This created some lag time in reestablishing contact and effectively interacting with the team. Next time we’ll designate a primary and secondary sponsor – the pace of this course requires this.

For us, this was a good learning opportunity to understand the type of sponsor education we need to do in the next class.

Advanced Lecture: Value Proposition Canvas
The advanced lecture for week 2 was on the Value Proposition Canvas – finding product/market fit between Beneficiaries (customers, stakeholders, users) and the Value Proposition (the product/service) in a DOD setting.

Pete Newell started the lecture with a video from his time in the Army’s Rapid Equipping Force.

Pete used the video to take the students through a value proposition canvas and asked the class:

  1. Who are the primary beneficiaries? Who are the other beneficiaries?
  2. What’s the value proposition:
    • To the sergeant?
    • To the mechanics?
    • To the base commander?
    • To contract engineers?
    • To the military vehicle Program Manager?

Pete’ s experienced based vignettes and discussion helped the students appreciate the sometimes competing nature of the interests of a diverse array of beneficiaries.

If you can’t see the presentation click here

Lessons Learned from Week 2

  • Teams are running at full speed
  • Running a flipped classroom requires constant management
    • Problems need to be vetted to insure they can support customer discovery expectations
  • A Customer Discovery Workshop needs to be held
    • Teams need to understand how to work around security issues
  • Sponsors need education and management

Hacking for Defense (H4D) @ Stanford – Week 1

We just had our first Hacking for Defense class and the 8 teams have hit the ground running.

They talked to 86 customers/stakeholders before the class started.

(Because of the embedded presentations this post is best viewed on the website.)


Hacking for Defense is a new class in Stanford’s School Engineering, where students learn about the nation’s security challenges by working with innovators inside the Department of Defense (DoD) and Intelligence Community. The class teaches students the Lean Startup approach to entrepreneurship while they engage in what amounts to national public service.

Hacking for Defense uses the same Lean LaunchPad Methodology adopted by the National Science Foundation and the National Institutes of Health and proven successful in Lean LaunchPad and I-Corps classes with 1,000’s of teams worldwide. Over 70 students applied to this new Stanford class and we selected 32 of them in 8 teams.

One of the surprises was the incredible diversity of the student teams – genders, nationalities, expertise. The class attracted students from all departments and from undergrads to post docs.

Before the class started, the instructors worked with the Department of Defense and the Intelligence Community to identify 20 problems that the class could tackle. Teams then were free to select one of these problems as their focus for the class.

Most discussion about innovation of defense systems acquisition starts with writing a requirements document. Instead, in this class the student teams and their DOD/IC sponsors will work together to discover the real problems in the field and only then articulate the requirements to solve them and deploy the solutions.

Hacking for Defense: Class 1
We started the first class with the obligatory class overview slides. (Most of the students had already seen them during our pre-class information sessions but the class also had team mentors seeing them for the first time.)

If you can’t see the slides click here

Then it was time for each of the 8 teams to tell us what they did before class started.  Their pre-class homework was to talk to 10 beneficiaries before class started. At the first class each team was asked to present a 5-slide summary of what they learned before class started:

  • Slide 1           Title slide
  • Slide 2           Who’s on the team
  • Slide 3           Minimal Viable Product
  • Slide 4:          Customer Discovery
  • Slide 5:          Mission Model Canvas

As the teams presented the teaching team offered a running commentary of suggestions, insights and direction.

Unlike the other Lean Launchpad / I-Corps classes we’ve taught, we noticed that before we even gave the teams feedback on their findings, we were impressed by the initial level of sophistication most teams brought to deconstructing the sponsors problem.

Here are the first week presentations:

Team aquaLink is working on a problem for divers in the Navy who work 60 to 200 feet underwater for 2-4 hours, but currently have no way to monitor their core temperature, maximum dive pressure, blood pressure and pulse. Knowing all of this would give them early warning of hypothermia or the bends. The goal is to provide a wearable sensor system and apps that will allow divers to monitor their own physiological conditions while underwater.

If you can’t see the presentation click here.

Team Guardian is asking how to protect soldiers from cheap, off-the-shelf commercial drones conducting Intelligence, Surveillance and Reconnaissance. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons?

Slides 6 and 7 use the Value Proposition canvas to provide a deeper understanding of product/market fit.

If you can’t see the presentation click here.

Team Skynet is also using drones to to provide ground troops situational awareness. (Almost he inverse of Team Guardian.)

Slides 6 – 8 use the Value Proposition canvas to provide a deeper understanding of product/market fit.

If you can’t see the presentation click here.

Team LTTT (Live Tactical Threat Toolkit) is providing assistance to other countries explosive ordnance disposal (EOD) teams – the soldiers trying to disarm roadside bombs (Improvised Explosive Devices – IEDs). They’re trying to develop tools that would allow foreign explosive experts to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.

If you can’t see the presentation click here.

Team Narrative Mind is trying to determine how to use data mining, machine learning, and data science to understand, disrupt, and counter adversaries’ use of social media (think ISIS). Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

If you can’t see the presentation click here.

Team Capella is launching a constellation of satellites with synthetic aperture radar into space to provide the Navy’s 7th fleet with real-time radar imaging.

If you can’t see the presentation click here.

Pre-Computing the Problem and Solution
As expected, a few teams with great technical assets jumped into building the MVP and were off coding/building hardware. It’s a natural mistake. We’re trying to get students to understand the difference between an MVP and a prototype and the importance of customer discovery (hard when you think you’re so smart you can pre-compute customer problems and derive the solution sitting in your dorm room.)

Mentors/Liaisons/DIUx Support
Besides working with their government sponsors, each team has a dedicated industry mentor. One of the surprises was the outpouring of support from individuals and companies who emailed us from across the country (even a few from outside the U.S.) volunteering to mentor the teams.

Each team is also supported by an active duty military liaison officer drawn from Stanford’s Senior Service College Fellows.

Another source of unexpected support for the teams was from the Secretary of Defense’s DIUx Silicon Valley Innovation Outpost. DIUx has adopted the class and along with the military liaisons translate “military-speak” from the sponsors into English and vice versa.

Advanced Lectures
The Stanford teaching team uses a “flipped classroom” (the lectures are homework watched on Udacity.)  However, for this class some of the parts of the business model canvas, which make sense in a commercial setting, don’t work in the Department of Defense and Intelligence Community. So we are supplementing the video lectures with in-class “advanced” lectures that explain the new Mission Model Canvas. (We’re turning these lectures into animated videos which can serve as homework for the next time we teach this class.)

The first advanced lecture was on Beneficiaries (customers, stakeholders, users, etc.) in the Department of Defense. Slides 4-7 clearly show that solutions in the DOD are always a multi-sided market. Almost every military program has at least four customer segments: Concept Developers, Capability Managers, Program Managers, Users.

If you can’t see the presentation click here.

Each team is keeping a running blog of their customer interactions so we can virtually look over their shoulder as they talk to customers. From the look of the blogs week 2 is going to be equally exciting. Check in next week for an update.

Steve, Pete, Joe & Tom

Lessons Learned from Class 1

  • Talented and diverse students seem eager to solve national defense problems
  • Teams jumped on understanding their sponsors problems – even before the class
  • We’ve put 800+ teams through the NSF I-Corps and another 200 or so through my classes, but this class feels really different. There’s a mission focus and passion to these teams I’ve not seen before

Learning Through Reflection

“Sometimes, you have to look back in order to understand the things that lie ahead.”

We just finished the 6th annual Lean LaunchPad class. This year we made a small but substantive addition to way we teach the class, adding a week for reflection. The results have made the class massively better.


For the last 6 years I’ve taught the Lean LaunchPad class at Stanford and Berkeley. To be honest I built the class out of frustration watching schools teach aspiring entrepreneurs that all they need to know is how to write a business plan or how to sit in an incubator building a product.

If you’ve read any of my previous posts, you know I believe that:

  1. a product is just a part of a startup, but understanding customers, channel, pricing, – the business model – is what turns a product into a business
  2. business plans are fine for large companies where there is an existing market, existing product and existing customers, but they are useless in a startup where most often none of these are known
  3. entrepreneurship is experiential and requires theory and a ton of practice.

Therefore, we developed the 8-week Lean LaunchPad class to teach students how to think about all the parts of building a business, not just the product.  We organized the class as:

  • Team-based
    • Students apply and learn as teams of 4. Eight teams per class
  • A “flipped” classroom
    • Students watch the lectures as homework via our MOOC
  • Every week we teach a new part of the theory of how to commercialize an insight or invention
    • using the business model canvas as the framework
  • Every week we teach the practice of Lean
    • by having the students get out of the classroom and talk to 10-15 customers a week and build a new Minimum Viable Product weekly
    • in order to validate/invalidate their business model hypotheses
    • The teaching team critiques their progress and suggests what they might do next
  • Every week the teams present their results
    • “Here’s what we thought, here’s what we did, here’s what we found, here’s what we are going to do next”

Class flowThe combination of the Business Model Canvas, Customer Development and Agile Engineering is an extremely efficient template for the students to follow. It drives a hyper-accelerated learning process which leads the students to a “information dense, evidence-based” set of conclusions. (Translation: they learn a lot more, in a shorter period of time, than in any other entrepreneurship course we’ve taught or seen.)

Demo Days Versus Lesson Learned Presentations
One thing we always kept in mind – we were teaching students a methodology and a set of skills for the rest of their lives – not running an incubator or accelerator. As a consequence, we couldn’t care less about a “Demo Day” at the end of the class. We don’t want our students focused on fund-raising, we want them to maximize their learning. Secondly, even for fund-raising, you couldn’t invent a less useful format to evaluate a startup’s potential then the Demo Days held by accelerators. Demo Days historically have been exactly what they sound like, “Show me how smart your team is at this instant in time.”  Everything depends on a demo, presentation and speaking style.

We designed our class to do something different. We wanted the teams to tell the story of their journey, sharing with us their “Lessons Learned from our Customers”. They needed to show what they learned and how they learned it after speaking to 100+customers, using the language of class: interview, iterations, pivots, restarts, experiments, minimal viable products, evidence. The focus of their presentations is on how they gathered evidence and how it impacted the understanding of their business models – while they were building their MVP.

Reflection Week
In the past, our teams would call on customers until the last week of the class and then present their Lessons Learned. The good news is that their presentations were dramatically better than those given at demo days – they showed us what they learned over 8 weeks which gave us a clear picture of the velocity and trajectory of the teams. The bad news is since their heads were down working on customer discovery until the very end, they had no time to reflect on the experience.

We realized that we had been so focused in packing content and work into the class, we failed to give the students time to step back and think about what they actually learned.

So this year we made a change. We turned the next to last week of the class into a reflection week.  Our goal—to have the students extract the insights and meaning from the work they had done in the previous seven weeks.

We asked each team to prepare a draft Lessons Learned presentation telling us about their journey and showing us their:

  • Initial hypotheses and Petal diagram
  • Quotes from customers that illustrated learnings and insights
  • Diagrams of key parts of the Canvas –customer flow, channel, get/keep/grow (before and after)
  • Pivot stories
  • Screen shots of the evolution of Minimum Viable Product (MVP)
  • Demo of final MVP

The teaching team reviewed the drafts and provided feedback to the teams and to the class as a whole. We discussed what general patterns and principles they extracted from all the customer interaction they had. On the last day of class, each team shared their Lessons Learned presentations, giving everyone in the class the benefit of what every team has learned.

We used this week to help teams reflect that they accomplished more than they first realized. For the teams who found that their ideas weren’t a scalable business, we let them conclude that while it was great to celebrate the wins, they could also embrace and celebrate their failures as low cost learning.

By the time the final week of the final Lessons Learned presentations rolled around, the students were noticeably more relaxed and happier than teams in past classes. It was clear they had a solid understanding of the magnitude of their journey and the size of their accomplishments – eight teams had spoken to nearly 900 customers, built 50 minimum viable products, and tested tons of hypotheses.

Here are four examples from our 2016 Stanford class

Pair Eyeware
Be sure to look at how they tested their hypotheses on slides 11 and 12, and the before and after value proposition canvases on slide 13 -17. A great competitive Petal diagram is on slide 22

Share and Tell
Great story and setup in slides 3-7. Understanding their market in week 6, slide 31.

Allocate
Notice how they learned about their customer archetypes on slides 12-14. After 80 interviews, a big pivot on slide 16.

Nova Credit
Look at the key hypotheses on slide 2 and their journey in the class on slide 5.

Lessons Learned

  • Dedicating a week for reflections expands what everyone learns
  • Students extract the insights and meaning from the work they did
  • See all the presentations here

The Mission Model Canvas – An Adapted Business Model Canvas for Mission-Driven Organizations

As we prepared for the new Hacking for Defense class at Stanford, we had to stop and ask ourselves: How do we use the Business Model Canvas if the primary goal is not to earn money, but to fulfill a mission? In other words, how can we adapt the Business Model Canvas when the metrics of success for an organization is not revenue?

H4D screen top

Alexander Osterwalder and I think we have the answer – the new Mission Model Canvas.

Here are our collective thoughts.

—-

The Lean Startup is the way most innovators build startups and innovate inside of existing companies. As a formal method, the Lean Startup consists of three parts:

The Business Model Canvas has been a great invention for everyone from startups to large companies. Unlike an org chart, which describes how a company executes to deliver known products to known customers, the Business Model Canvas illustrates the search for the unknowns that most new ventures face. The 9 boxes of the canvas let you visualize all the components needed to turn customer needs/problems into a profitable company.

From Revenue Streams to Mission Achievement
The Business Model Canvas has served all of us well in thinking about building businesses – and therein lies the problem. In a business the aim is to earn more money than you spend. What if you’re a government or a military organization or part of the intelligence community? In these cases you don’t earn money, but you mobilize resources and a budget to solve a particular problem and create value for a set of beneficiaries (customers, support organizations, warfighters, Congress, the country, etc.)

For these organizations, the canvas box labeled Revenue Streams doesn’t make sense.Business Model Canvas no revenue In a mission-driven organization such as the defense and intelligence community, there is no revenue to measure. So the first step in building a canvas for mission-driven organizations is to change the Revenue Stream box in the canvas and come up with a counterpart that would provide a measure of success.

We’re calling this alternative Mission Achievement. Later in this post I’ll explain how we’ll measure and describe Mission Achievement, but first our Mission Model Canvas needs four more tweaks.

  • Customer Segments is changed to Beneficiaries
  • Cost Structure is changed to Mission Cost/Budget
  • Channel is changed to Deployment
  • Customer Relationships is changed to Buy-in/Support

Mission_Model_CanvasThe rest of this blog post explains the how and why of these changes to the canvas.

Customer Segments Change to Beneficiaries
At first glance, when developing a new technology for use in the defense and intelligence community, the customer appears obvious – it’s the ultimate war fighter. They will articulate pains in terms of size, weight, form fit, complexity and durability. But there are other key players involved.  Requirement writers and acquisition folks look at systems integration across the battlefield system, while contracting officers, yet another segment, will count beans, measure the degree of competition and assess the quality of market research involved. The support organizations need to worry about maintainability of code or hardware. Does legal need to sign off for cyber operations?  So yes, war fighters are one customer segment, but others need to be involved before the war fighter can ever see the product.

So the first insight is that in the defense and intelligence community mission models are always multi-sided markets with the goal of not just building a great demo but getting the product adopted and deployed.

Second, in the defense and intelligence communities almost all of the mission models look like that of an OEM supplier – meaning there are multiple layers of customers in the value chain. Your product/service is just part of someone else’s larger system.

So to differentiate “customers” from the standard business model canvas we’ll call all the different customer segments and the layers in the defense and intelligence value chain beneficiaries.

The Value Proposition Canvas
Of all the nine boxes of the canvas, two important parts of the model are the relationship between the Value Proposition (what you’re building) and the beneficiaries. These two components of the business model are so important we give them their own name, Product/Market Fit.osterwalder books

Because of the complexity of multiple beneficiaries and to get more detail about their gains and pains, Osterwalder added an additional canvas called the Value Proposition Canvas.  This functions like a plug-in to the Mission Model Canvas, zooming in to the value proposition to describe the interactions among these beneficiaries, war fighters, etc. and the product/service in more detail. Using the Value Proposition Canvas with the Mission Model Canvas lets you see both the big picture at the mission model level and the detailed picture of each beneficiary at the “product/market fit” level.

Value prop zoom bus modelIn the defense and intelligence community mission models, there will always be multiple beneficiaries.  It’s important that each beneficiary gets its own separate Value Proposition Canvas.

value_proposition_canvas

Distribution Channel changes to Deployment
In the commercial world we ask, “What type of distribution channel (direct sales, app store, system integrator, etc.) do we use to get the product/service from our company to the customer segments?”  For the Department of Defense or Intelligence organizations, we ask instead:

  • “What will it take to deploy the product/service from our current Minimum Viable Product to widespread use among people who need it?” (What architecture components can they innovate on and what can’t they?)
  • “What constitutes a successful deployment? (number of users, units in the field, time to get it into the field, success in the field, etc.)”
  • “How do we turn a Horizon 3 innovation into something that gets adopted by a Horizon 1 organization?”

Customer Relationships changes to Buy-In/Support
In an existing business, Customer Relationships is defined as establishing and maintaining a relationship to support existing customers. In a startup we redefined Customer Relationships to answer the question:  How does a company get, keep and grow customers?

For the defense and intelligence communities, we have modified Customer Relationships to mean, “For each beneficiary (customer segment), how does the team get “Buy-In” from all the beneficiaries?”

Customer discovery helps you understand whose buy-in is needed in order to deploy the product/service (legal, policy, procurement, etc.) and how to get those beneficiaries to buy-in? (Funding? Mandates? User requested? etc.) In addition, the long-term support and maintenance of new projects need to be articulated, understood and bought-into by the support organizations.

At the Pentagon a favorite way to kill something is to coordinate it to death by requiring buy-in from too many people too early. How to determine who are the small group of critical people to get buy-in from – and how to determine who are the next set required to sustain the iterative development of future MVP’s – is one of the arts of entrepreneurship in the defense and intelligence community.

Revenue Streams changes to Mission Achievement
Mission Achievement is the value you are creating for the sum of all of the beneficiaries / the greater good.

It’s important to distinguish between the value for individual beneficiaries (on the Value Proposition Canvas) and overall Mission Achievement. For example, Mission Achievement could be measured in a variety of ways: the number of refugees housed and fed, the number of soldiers saved from roadside bombs, the number of cyberattacks prevented, the increased target surveillance of sensor fusion, etc.  None of these are measured in dollars and cents. Keep in mind, there is only mission achievement if it delivers value to the end beneficiary.

Lessons Learned

  • In the defense and intelligence community the metrics of success are not revenue but mission achievement
    • We’ve modified the Business Model Canvas into a Mission Model Canvas
    • Changed Revenue Streams to Mission Achievement
    • Changed Customer Segments to Beneficiaries
    • Changed Cost Structure to Mission Cost/Budget
    • Changed Channel to Deployment
    • Changed Customer Relationships to Buy-in/Support
  • Organizations without specific revenue goals can now use a version of the Business Model Canvas
Follow

Get every new post delivered to your Inbox.

Join 209,310 other followers

%d bloggers like this: