Steve Blank

We just held our third week of the Hacking for Defense class. This week the 8 teams spoke to 108 beneficiaries (users, program mangers, etc.), we held a Customer Discovery workshop, we started streaming the class live to DOD/IC sponsors and other educators, our advanced lecture was on Product/Market fit for the DOD/IC and we watched as the students solved their customer discovery obstacles and started getting closer to their customers.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

—–

Customer Discovery in the DOD/IC Workshop
We normally hold a Customer Discovery workshop during the evening the first week of the class. But spring break and the “How to Work with the DOD” workshop got in the way. So we inserted an abbreviated version at the front of this week’s class.

When working with the DOD/IC there are some unique obstacles of “getting out of the building and talking to customers”. For example, members of the DOD will not respond to ”cold calls” and those in the Intel community won’t even tell you their names. In addition, most of the sponsors are working on classified problems. So how do teams understand the customer when the customer can’t tell you what they do? The Workshop talked about how to address those and other Discovery issues.

If you can’t see presentation click here

Team Presentations: Week 3
After the Customer Discovery workshop the 8 teams presented what their hypotheses, experiments and what they learned outside the building this week.

Team Right of Boom (previously named Live Tactical Threat Toolkit) is trying to help foreign military explosive ordnance disposal (EOD) teams better accomplish their mission. The team originally was developing tech-centric tools for foreign teams to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.  Now they are honing in providing accurate high-volume post-incident IED reporting.

Last week this team was floundering. They had confused getting interviews and building minimal viable products with truly trying to “become the customer.” We strongly suggested that there was no way that could understand the day in the life of an explosive ordnance disposal expert by just listening to them – they needed to stand in their shoes. So to their immense credit the team suited up in full bomb disposal gear and got of the building. They earned our respect (and a name change for the team.)

If you can’t see the Right to Boom video click here (turn up the volume!)

If you can’t see the Right to Boom presentation click here

Team Capella Space
  is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

This week the team learned a ton. They mapped out competitive offerings, found that Government funding is not the proper channel for Capella, but did find that the Coast guard is currently in dire need of situational awareness at high resolution and that military customers want access to raw data; commercial customers highly value processed data for actionable insights

If you can’t see the Team Capella Space presentation click here

Team aquaLink is working to give Navy divers a way to monitor their own physiological conditions while underwater (core temperature, maximum dive pressure, blood pressure and pulse.) Knowing all of this would give divers early warning of hypothermia or the bends.

This week they validated that divers will want real-time alerts regarding vitals (and put up with the additional gear/procedures) of issues that threaten mission success. The found that navy medical researchers want data on vitals, the rebreather (air consumption), and the dive computer (dive profile). Their hypotheses going forward are that a heads up display is the ideal form of information transmission during a dive and system should be modular to allow for the integration of evolving technology (geolocation and communication)

If you can’t see the Team aquaLink presentation click here


Team Guardian is working to protect soldiers from cheap, off-the-shelf commercial drones. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons?

Guardians current hypotheses is that they have to provide drone detection, identification and protection against attacks from drones or swarm of drones. And that the user will be a 19 solider not trained to use complex equipment.

If you can’t see the Team Guardian presentation click here

Team Narrative Mind is trying to understand, disrupt, and counter adversaries’ use of social media. Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

The team is coalescing around the idea that the two minimal viable products for their sponsor are, 1) automatically generate an organizational chart of a target terrorist groups over time, and 2) generate a social network map of how terrorist groups interact with each other.

If you can’t see the Team Narrative Mind presentation click here

Team Sentinel initially started by trying to use low cost sensors to monitor surface ships in a A2/AD environment.

The team has found that their mission value is really to enable more efficient and informed strategic decisions by filling in intelligence gap about surface ships in an A2/AD environment via:

1. Increased number of data streams (i.e. incorporate open source data)
2. Automated data aggregation (i.e. from disparate sources) and analysis
3. Enhanced intel through contextualization
4. Improved UI/UX

If you can’t see the Sentinel presentation click here

Team Skynet is also using drones to to provide ground troops situational awareness. (Almost the inverse of Team Guardian.)

The team invalidated the hypotheses that military/commercial systems exist that could already solve the problem. In addition, they originally believed that soldiers on foot needed a deployable drone system. They discovered that drones are best used with teams with vehicles or for short ranged dismounted operations.

If you can’t see the Team Skynet presentation click here

Advanced Lecture: Product/Market fit in the DOD/IC
The advanced lecture for week 3 was on the unique needs of finding Product/Market fit in the DOD/IC. Pete Newell described why a solutions in the DOD fails and then described “battlefield calculus” – how two identical sounding missions (and their inherent problems) are actually radically different based on what echelon of force executes them, by the size of force, their location, even by how well they are trained.  Despite the obvious, people still try to deliver “one-size-fits-all” solutions. To properly insure a solution is actually used it is important to become familiar with the pattern of life of the user and their unit.

Pete also pointed out that teams need to “Look for Conflict” between what may have been provided to solve a similar problem and the solution the teams are about to recommend. You needed to ask: Are the circumstances similar? Or are their a myriad of conditions present that will invalidate what was a good solution under different circumstances?

If you can’t see the presentation click here

Mission Model and Value Proposition Canvas
To students, “who are the beneficiaries?” feels fuzzy on day one. And given most of them had no exposure to the DOD or Intel Community it’s not a surprise. The reason we have the teams talk to 10-15 people every week, is that with enough data they can begin to fill in the details. A few of our guests have commented how knowledgeable the teams were in talking about the sponsor organizations and problems.

That said, listening to the team presentations there was a wide difference between teams in how well they understood that the definition of “beneficiaries.”  Many of teams were still listing names of organizations rather than the title and archetype of the people who mattered/cared/decided/users, etc.

Understanding who are the beneficiaries is critical to understanding the rest of the mission model canvas.

When the students have a more nuanced understanding who are the individual beneficiaries is when they can build a detailed Value Proposition Canvas for each beneficiary that makes sense.  (Several teams had Value Proposition Canvas of organizations, some had fewer Proposition Canvas than they had beneficiaries, some Proposition Canvases were so generic it was clear that had insufficient data on individual needs of specific archetypes, etc.)

This is all par for the course and part of the student learning. We now need to sharpen their focus.

An after class action for the teaching team is to read through every team’s week 3 presentation slide-by-slide and give each team a detailed, written, box-by-box critique of the right-side of their Mission Model and Value Proposition Canvas.  We want to help them get this right.

Sponsor Education – a Network Begins to Form
The teaching team, liaisons, mentors and DIUx are all working their networks to get students relevant beneficiaries to talk to. (More about what a wonderful asset DIUx has been in a future post.) Joe and Pete are continuing to work hard on educating the sponsors about their role. (We are collecting all our learning in an Educators Guide so other universities can run the class.)

One emerging unexpected benefit, is that Pete and Joe are continuing to expand the network of innovators in the DOD/IC who are helping our student teams. I’ve had several critique our presentations and offer suggestions on the nuanced parts of the IC mission and acquisition system I didn’t understand.

Live Streaming the Class
The DOD/IC sponsors who gave us these problems were curious about how the teams were learning so rapidly. (Others in their commands and agencies wanted to watch as well.) So this week we began to live-stream the student presentations. And other universities who want to offer this class have begun to have their educators watch the class. (We’ll be offering a train-the-trainer educators class later this year.)

Lessons Learned from Week 3

• Teams still running at full speed
• Understanding beneficiaries is critical to understanding the rest of the mission model canvas.
  • Written team-by-team offline critique is needed to keep them on course
• Support is coming from lots of places in the DOD/IC
  • DIUx and our liasons have been great in connecting the students

Filed under: Hacking For Defense, Teaching | 1 Comment »

We just held our second week of the Hacking for Defense class. This week the 8 teams spoke to 106 beneficiaries (users, program mangers, etc.), we held a DOD/IC 101 workshop, our advanced lecture was on the Value Proposition Canvas, and we watched as the students ran into common customer discovery obstacles and found new ones.

(This post is a continuation of the series. See all the H4D posts here. Because of the embedded presentations this post is best viewed on the website.)

DOD/IC 101 – Workshop
We started the week by holding a Monday night workshop – DOD/IC 101. Our goal was to give the students with no military background a tutorial on the challenges facing DoD/IC in the current asymmetric threat environment, how the DOD/IC defines its missions and specifies the products it needs, how it accomplishes these missions and how they get to their ultimate user. This knowledge will help the students understand the overall environment that their Mission Model Canvas is operating in.

We posted the slides here and more important, an annotated narrative for each of the slides here. It’s truly a landmark presentation. Even if you think you know how the DOD works, read the narrative alongside the slides. I learned a lot.

If you can’t see the presentation click here

you can’t see the narrative click here.

Hacking for Defense: Week 2
The second week started with the 8 teams presenting what they learned in their first full week of class.

Capella Space
Team Capella is launching a constellation of synthetic aperture radar satellites into space to provide real-time radar imaging.

This week the team tested whether other beneficiaries – the Coast Guard and the Oil and Gas industry might be interested in their solution. Great learning.

If you can’t see the presentation click here

Live Tactical Threat Toolkit (LTTT)
Team LTTT (Live Tactical Threat Toolkit) is trying to enhance the capacity of  foreign military explosive ordnance disposal (EOD) teams to accomplish their mission. The team is developing tech informed options for these teams to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.

The team did a good job in starting to diagram the customer workflow and intends to gain an appreciation for the ground user challenges in accomplishing these types of missions in this weeks customer discovery efforts.

If you can’t see the presentation click here

Narrative Mind
Team Narrative Mind is trying to understand, disrupt, and counter adversaries’ use of social media. Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

The team did a good job in starting to diagram the customer workflow and their understanding of how to prioritize MVP features.

If you can’t see the presentation click here

Skynet
Team Skynet is also using drones to to provide ground troops situational awareness. (Almost the inverse of Team Guardian.)

Their Mission Model Canvas had a ton of learning, and their MVP engendered a lot of conversation from those who’ve been in combat and were familiar with the challenges of maintaining situational awareness under fire.

If you can’t see the presentation click here

Aqualink
Team aquaLink is working to give Navy divers a way to monitor their own physiological conditions while underwater (core temperature, maximum dive pressure, blood pressure and pulse.) Knowing all of this would give divers early warning of hypothermia or the bends.

In the first week of the class this team was suiting up in full navy diving gear and doing customer discovery by spending an hour in the life of the beneficiary. They did their homework.

If you can’t see the presentation click here

Guardian
Team Guardian is working to protect soldiers from cheap, off-the-shelf commercial drones. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons? This team is actively working to identify viable responses to these  battlefield inevitabilities.

If you can’t see the presentation click here

Sentinel
Team Sentinel is trying to use low cost sensors to monitor surface ships in a A2/AD environment. The team appreciates that the problem include the sensors as well as the analytics of the sensor data.

Really good summary of hypotheses, experiments, results and action.

If you can’t see the presentation click here

Customer Discovery and the Flipped Classroom – Learnings
After talking to teams in office hours (the teaching team meets every team for 20 minutes every week,) and watching teams present, and then seeing a team send a sponsor an email that read like a bad business school sales pitch, we realized some students had skipped their homework/and or still hadn’t grasped the basics of Customer Discovery.

As a reminder, we run the class as a “flipped classroom” – the lectures – the basics of Customer Discovery and the Mission Model Canvas – are homework watched on Udacity and on Vimeo. It was painfully clear that many of the students hadn’t done their homework. We plan to remedy that in our next week class, warning the students that we will be cold calling on them to show us what they learned.

Some teams did their homework and understood that customer discovery meant “becoming the customer.” For example, the team solving a problem for Navy divers managed to get the Navy to suit them up in full diving regalia. On the other hand, some teams thought that customer discovery simply meant interviewing people and building a minimal viable products. For example, we suggested to the team working on solutions for defusing Improvised Explosive Devices (IEDs) that to truly understand their customer needs might require them to get close to the dirt with some explosive ordnance disposal teams. (Looking ahead we have no doubt that this team will respond aggressively to instructor feedback and suit up in Explosive Ordinance Disposal (EOD) equipment as part of their customer discovery efforts for week 3. Stay tuned.)

Part of the student confusion about customer discovery was the fault of the teaching team. We normally hold a “How to Do Customer Discovery” evening workshop, but we got caught by a tight spring break schedule and we punted this workshop to hold the DOD/IC workshop. In hindsight it was a bad idea – we should have found a way to hold both. We will remedy that by giving an abbreviated workshop first thing next week in the classroom.

All of these were problems we’ve seen before and we’re course correcting quickly to solve them.  But, given the new form of the class we had a few problems we hadn’t encountered.

First, some teams were stymied by the classified nature of the specific data sets they thought they needed to understand the customer problem and build MVPs. In every case, what they lacked was a deep understanding of the customer problem. Which simply required going back to the basics of customer discovery.

Second, a few teams were truly blocked by a few sponsors who were also having a difficult time understanding the role they played in Customer Discovery and required follow up clarification by the teaching team and H4D military liaison officers.

Sponsor Education – Learnings
A few DOD sponsors believed they were not only the gatekeepers to the problem but were the sole source of information for our teams. Given they were supposed to maximize the number of beneficiaries the teams were supposed to talk to, the teaching team jumped on this and rapidly addressed it.

In another case the sponsor so narrowly defined the problem that it was viewed by the team as providing incremental changes to a solution they already have. After discussion the sponsor agreed that the team should focus on the realm of possible and how they would address the problem if there was not a current solution in place and in the process define new plans for how the solutions could be used.

In other cases a few of our sponsors had difficulty generating the leads and contacts within their own ecosystems that were necessary to sustain our teams’ customer discovery beyond the sponsor’s primary contacts. Ultimately teams are required to interview 80-120 beneficiaries, advocates and stakeholders (customers). This is a heavy lift if the sponsor has not thought through who those people are and where they will be found.

Finally, one of our problem sponsors departed their organization and was replaced by an alternate. This created some lag time in reestablishing contact and effectively interacting with the team. Next time we’ll designate a primary and secondary sponsor – the pace of this course requires this.

For us, this was a good learning opportunity to understand the type of sponsor education we need to do in the next class.

Advanced Lecture: Value Proposition Canvas
The advanced lecture for week 2 was on the Value Proposition Canvas – finding product/market fit between Beneficiaries (customers, stakeholders, users) and the Value Proposition (the product/service) in a DOD setting.

Pete Newell started the lecture with a video from his time in the Army’s Rapid Equipping Force.

Pete used the video to take the students through a value proposition canvas and asked the class:

1. Who are the primary beneficiaries? Who are the other beneficiaries?
2. What’s the value proposition:
   • To the sergeant?
   • To the mechanics?
   • To the base commander?
   • To contract engineers?
   • To the military vehicle Program Manager?

Pete’ s experienced based vignettes and discussion helped the students appreciate the sometimes competing nature of the interests of a diverse array of beneficiaries.

If you can’t see the presentation click here

Lessons Learned from Week 2

• Teams are running at full speed
• Running a flipped classroom requires constant management
  • Problems need to be vetted to insure they can support customer discovery expectations
• A Customer Discovery Workshop needs to be held
  • Teams need to understand how to work around security issues
• Sponsors need education and management

Filed under: Hacking For Defense, Teaching | 4 Comments »

We just had our first Hacking for Defense class and the 8 teams have hit the ground running.

They talked to 86 customers/stakeholders before the class started.

(Because of the embedded presentations this post is best viewed on the website.)

Hacking for Defense is a new class in Stanford’s School Engineering, where students learn about the nation’s security challenges by working with innovators inside the Department of Defense (DoD) and Intelligence Community. The class teaches students the Lean Startup approach to entrepreneurship while they engage in what amounts to national public service.

Hacking for Defense uses the same Lean LaunchPad Methodology adopted by the National Science Foundation and the National Institutes of Health and proven successful in Lean LaunchPad and I-Corps classes with 1,000’s of teams worldwide. Over 70 students applied to this new Stanford class and we selected 32 of them in 8 teams.

One of the surprises was the incredible diversity of the student teams – genders, nationalities, expertise. The class attracted students from all departments and from undergrads to post docs.

Before the class started, the instructors worked with the Department of Defense and the Intelligence Community to identify 20 problems that the class could tackle. Teams then were free to select one of these problems as their focus for the class.

Most discussion about innovation of defense systems acquisition starts with writing a requirements document. Instead, in this class the student teams and their DOD/IC sponsors will work together to discover the real problems in the field and only then articulate the requirements to solve them and deploy the solutions.

Hacking for Defense: Class 1
We started the first class with the obligatory class overview slides. (Most of the students had already seen them during our pre-class information sessions but the class also had team mentors seeing them for the first time.)

If you can’t see the slides click here

Then it was time for each of the 8 teams to tell us what they did before class started.  Their pre-class homework was to talk to 10 beneficiaries before class started. At the first class each team was asked to present a 5-slide summary of what they learned before class started:

• Slide 1           Title slide
• Slide 2           Who’s on the team
• Slide 3           Minimal Viable Product
• Slide 4:          Customer Discovery
• Slide 5:          Mission Model Canvas

As the teams presented the teaching team offered a running commentary of suggestions, insights and direction.

Unlike the other Lean Launchpad / I-Corps classes we’ve taught, we noticed that before we even gave the teams feedback on their findings, we were impressed by the initial level of sophistication most teams brought to deconstructing the sponsors problem.

Here are the first week presentations:

Team aquaLink is working on a problem for divers in the Navy who work 60 to 200 feet underwater for 2-4 hours, but currently have no way to monitor their core temperature, maximum dive pressure, blood pressure and pulse. Knowing all of this would give them early warning of hypothermia or the bends. The goal is to provide a wearable sensor system and apps that will allow divers to monitor their own physiological conditions while underwater.

If you can’t see the presentation click here.

Team Guardian is asking how to protect soldiers from cheap, off-the-shelf commercial drones conducting Intelligence, Surveillance and Reconnaissance. What happens when adversaries learn how to weaponize drones with bullets, explosives, or chemical weapons?

Slides 6 and 7 use the Value Proposition canvas to provide a deeper understanding of product/market fit.

If you can’t see the presentation click here.

Team Skynet is also using drones to to provide ground troops situational awareness. (Almost he inverse of Team Guardian.)

Slides 6 – 8 use the Value Proposition canvas to provide a deeper understanding of product/market fit.

If you can’t see the presentation click here.

Team LTTT (Live Tactical Threat Toolkit) is providing assistance to other countries explosive ordnance disposal (EOD) teams – the soldiers trying to disarm roadside bombs (Improvised Explosive Devices – IEDs). They’re trying to develop tools that would allow foreign explosive experts to consult with their American counterparts in real time to disarm IED’s, and to document key information about what they have found.

If you can’t see the presentation click here.

Team Narrative Mind is trying to determine how to use data mining, machine learning, and data science to understand, disrupt, and counter adversaries’ use of social media (think ISIS). Current tools do not provide users with a way to understand the meaning within adversary social media content and there is no automated process to disrupt, counter and shape the narrative.

If you can’t see the presentation click here.

Team Capella is launching a constellation of satellites with synthetic aperture radar into space to provide the Navy’s 7^th fleet with real-time radar imaging.

If you can’t see the presentation click here.

Pre-Computing the Problem and Solution
As expected, a few teams with great technical assets jumped into building the MVP and were off coding/building hardware. It’s a natural mistake. We’re trying to get students to understand the difference between an MVP and a prototype and the importance of customer discovery (hard when you think you’re so smart you can pre-compute customer problems and derive the solution sitting in your dorm room.)

Mentors/Liaisons/DIUx Support
Besides working with their government sponsors, each team has a dedicated industry mentor. One of the surprises was the outpouring of support from individuals and companies who emailed us from across the country (even a few from outside the U.S.) volunteering to mentor the teams.

Each team is also supported by an active duty military liaison officer drawn from Stanford’s Senior Service College Fellows.

Another source of unexpected support for the teams was from the Secretary of Defense’s DIUx Silicon Valley Innovation Outpost. DIUx has adopted the class and along with the military liaisons translate “military-speak” from the sponsors into English and vice versa.

Advanced Lectures
The Stanford teaching team uses a “flipped classroom” (the lectures are homework watched on Udacity.)  However, for this class some of the parts of the business model canvas, which make sense in a commercial setting, don’t work in the Department of Defense and Intelligence Community. So we are supplementing the video lectures with in-class “advanced” lectures that explain the new Mission Model Canvas. (We’re turning these lectures into animated videos which can serve as homework for the next time we teach this class.)

The first advanced lecture was on Beneficiaries (customers, stakeholders, users, etc.) in the Department of Defense. Slides 4-7 clearly show that solutions in the DOD are always a multi-sided market. Almost every military program has at least four customer segments: Concept Developers, Capability Managers, Program Managers, Users.

If you can’t see the presentation click here.

Each team is keeping a running blog of their customer interactions so we can virtually look over their shoulder as they talk to customers. From the look of the blogs week 2 is going to be equally exciting. Check in next week for an update.

Steve, Pete, Joe & Tom

Lessons Learned from Class 1

• Talented and diverse students seem eager to solve national defense problems
• Teams jumped on understanding their sponsors problems – even before the class
• We’ve put 800+ teams through the NSF I-Corps and another 200 or so through my classes, but this class feels really different. There’s a mission focus and passion to these teams I’ve not seen before

Filed under: Hacking For Defense, Teaching | 6 Comments »