Hacking for Defense @ Stanford 2018 – wonder and awe

We just finished our 3rd annual Hacking for Defense class at Stanford. Six teams presented their Lessons Learned presentations.

Watching them I was left with wonder and awe about what they accomplished in 10 weeks.

  • Six teams spoke to over 600 beneficiaries, stakeholders, requirements writers, program managers, warfighters, legal, security, customers, etc.
  • By the end the class all of the teams realized that the problem as given by the sponsor had morphed into something bigger, deeper and much more interesting.

Each of the six teams presented a 2-minute video to provide context about their problem and then gave an 8-minute presentation of their Lessons Learned over the 10-weeks. Each of their slide presentations follow their customer discovery journey. All the teams used the Mission Model Canvas, Customer Development and Agile Engineering to build Minimal Viable Products, but all of their journeys were unique.

The teams presented in front of several hundred people in person and online.

Team: TrackID

If you can’t see the TrackID video click here

If you can’t see the TrackID slides click here

Team: Polaris

If you can’t see the Polaris video click here
If you can’t see the Polaris slides click here

Team: Acquiforce

If you can’t see the Acquiforce video click here

If you can’t see the Acquiforce slides click here

Team: Intelgrids

If you can’t see the Intelgrids video click here
If you can’t see the Intelgrids slides click here

Team: See++

If you can’t see the See++ video click here
If you can’t see the See++ slides click here

Team: Theia

If you can’t see the Theia video click here
If you can’t see the Theia slides click here
Video of the teams live presentation are here.  Worth your time to watch.

The Class
Our mantra to the students was that we wanted them to learn about “Deployment not Demos.” Our observation is that the DOD has more technology demos than they need, but often lack deep problem understanding.  Our goal was to have the students first deeply understand their sponsors problem – before they started building solutions. As you can imagine with a roomful of technologists this was tough. Further we wanted the students to understand all parts of the mission model canvas, not just the beneficiaries and the value proposition. We wanted them to learn what it takes to get their product/service deployed to the field, not give yet another demo to a general. This meant that the minimal viable products the students built were focused on maximizing their learning of what to build, not just building prototypes.

(Our sponsors did remind us, that at times getting a solution deployed meant that someone did have to see a demo!)

Note: The Hacking for Defense class was designed as “fundamental research” to be shared broadly and the results are not subject to restriction for proprietary or national security reasons. In the 10 weeks the students have, Hacking for Defense hardware and software prototypes don’t advance beyond a Technology Readiness Level 4 and remain outside the scope of US export control regulations and restrictions on foreign national participation.

Goals for the Hacking for Defense Class
Our primary goal was to teach students Lean Innovation while they engaged in a national public service. Today if college students want to give back to their country they think of Teach for America, the Peace Corps, or Americorps or perhaps the US Digital Service or the GSA’s 18F. Few consider opportunities to make the world safer with the Department of Defense, Intelligence Community or other government agencies.

Next, we wanted the students to learn about the nation’s threats and security challenges while working with innovators inside the DoD and Intelligence Community. While doing so, also teach our sponsors (the innovators inside the Department of Defense (DOD) and Intelligence Community (IC)) that there is a methodology that can help them understand and better respond to rapidly evolving asymmetric threats. That if we could get teams to rapidly discover the real problems in the field using Lean methods, and only then articulate the requirements to solve them, could defense acquisition programs operate at speed and urgency and deliver timely and needed solutions.

Finally, we wanted to familiarize students about the military as a profession, its expertise, and its proper role in society. And conversely show our sponsors in the Department of Defense and Intelligence community that civilian students can make a meaningful contribution to problem understanding and rapid prototyping of solutions to real-world problems.

Origins of the class
Hacking for Defense has its origins in the Lean LaunchPad class I first taught at Stanford in 2011. It was adopted by the National Science Foundation in 2012 to train Principal Investigators who wanted to get a federal grant for commercializing their science (an SBIR grant.) The NSF observed, “The class is the scientific method for entrepreneurship. Scientists understand hypothesis testing” and relabeled the class as the NSF I-Corps (Innovation Corps). The class is now taught in 81 universities and has trained over 1500 science teams. It was adopted by the National Institutes of Health as I-Corps at NIH in 2014 and at the National Security Agency in 2015.

In 2016, brainstorming with Pete Newell of BMNT and Joe Felter at Stanford we observed that students in our research universities had little connection to the problems their government as well as the larger issues civil society was grappling with. Wondering how we could get students engaged, we realized the same Lean LaunchPad/I-Corps class would provide a framework to do so. Both Hacking for Defense and Hacking for Diplomacy with the State Department were born. Hacking for Energy at Columbia, Hacking for Impact (Non-Profits) at Berkeley and Hacking for Conservation and Development at Duke quickly followed.

 

The Innovation Insurgency Spreads
Hacking for Defense is now offered at eleven universities in addition to Stanford – Georgetown, University of Pittsburgh, Boise State, UC San Diego, James Madison University, University of Southern Mississippi, University of Southern California and Columbia University. Over the next year it will expand to 22 universities. Hacking for Defense.org a non-profit, was established to train educators and to provide a single point of contact for connecting the DOD/IC sponsor problems to these universities.

We’ve been surprised was how applicable the “Hacking for X…” methodology is for other problems. It’s equally applicable to solving public safety, energy, policy, community and social issues internationally and within our own communities. In the next year we’ll see three new variants of the class:

  • Hacking for the Environment
  • Hacking for Oceans
  • Hacking for Cities

It Takes a Village
While I authored this blog post, this classes is a team project. The teaching team consisted of:

  • Pete Newell is a retired Army Colonel currently a Senior Visiting Research Fellow at the National Defense University’s Center for Technology and National Security Policy and CEO of BMNT.
  • Steve Weinstein a 30-year veteran of Silicon Valley technology companies and Hollywood media companies.  Steve is CEO of MovieLabs the joint R&D lab of all the major motion picture studios.
  • Jeff Decker is a social science researcher at Stanford. Jeff served in the U.S. Army as a special operations light infantry squad leader in Iraq and Afghanistan.

Two of our teaching assistants were prior students: Samuel Jackson our lead TA, and Will Papper and Annie Shiel and Paricha Duangtaweesub also assisted.

Special thanks to our course advisors – Tom Byers, Professor of Engineering and Faculty Director, STVP, Arun Majumdar and Sally Benson Co-directors of the Stanford Precourt Energy Institute, and John Mitchell, Stanford Provost of Teaching and Learning.

A special thanks to Rich Carlin and the Office of Naval Research for supporting the program at Stanford and across the country.

We were lucky to get a team of mentors (VC’s and entrepreneurs) who selflessly volunteered their time to help coach the teams. Thanks to Tom Bedecarre, Kevin Ray, Daniel Bardenstein, Rafi Holtzman, Craig Seidel, Michael Chai, Lisa Wallace and Dave Gabler.

We were privileged to have the support of an extraordinary all volunteer team of professional senior military officers representing all branches of service attending fellowship programs at Stanford’s Hoover Institution, and Center for International Security and Cooperation (CISAC) and Asia Pacific Research Center (APARC) at the Freeman Spogli Institute (FSI). These included: Colonel Bradley Boyd, Lieutenant Colonel James “Gumbo” Coughlin, Lieutenant Colonel Marcus Ferrara, Lieutenant Colonel Jer “Jay” Garcia, Lieutenant Commander Nick Hill, Commander Michael Nordeen, Commander Rebecca Ore, Commander Michael Schoonover, Colonel Jason “Shrek” Terry and Todd Forsman.

And of course a big shout-out to our sponsors. At SOCOM, Matt Leland and Angel Zajkowski, at MITRE, Suresh Damodaran, at NAVFAC, Ben Wilcox, at the 9th ISR, Ian Eishen, at AFRL, Jeff Palumbo and Mike Rottmayer at the Defense Acquistion University, Shirley Franko and at ERDC Thomas Bozada.

Thanks!

The Innovation Stack: How to make innovation programs deliver more than coffee cups

Is your organization full of Hackathons, Shark Tanks, Incubators and other innovation programs, but none have changed the trajectory of your company/agency?

Over the last few years Pete Newell and I have helped build innovation programs inside large companies, across the U.S. federal science agencies and in the Department of Defense and Intelligence Community. But it is only recently that we realized why some programs succeed and others are failing.

After doing deep dives in multiple organizations we now understand why individual innovators are frustrated, and why entrepreneurial success requires heroics. We also can explain why innovation activities have generated innovation theater, but few deliverables. And we can explain why innovation in large organizations looks nothing like startups. Most importantly we now have a better idea of how to build innovation programs that will deliver products and services, not just demos.

It starts by understanding the “Innovation Stack” – the hierarchy of innovation efforts that have emerged in large organizations. The stack consists of: Individual Innovation, Innovation Tools and Activities, Team-based Innovation and Operational Innovation.

Individual Innovation
The pursuit of innovation inside large companies/agencies is not a 21st-century invention. Ever since companies existed, there have been passionate individuals who saw that something new, unplanned and unscheduled was possible. And pushing against the status quo of existing process, procedure and plan, they went about building a demo/prototype, and through heroic efforts succeeded in getting a new innovation over the goal line – by shipping/deploying a new innovation.

We describe their efforts as “heroic” because all the established procedures and processes in a large company are primarily designed to execute and support the current business model. From the point of view of someone managing an engineering, manufacturing or operations organization, new, unplanned and unscheduled innovations are a distraction and a drag on existing resources. (The best description I’ve heard is that, “Unfettered innovation is a denial of service attack on core capabilities.”) That’s because until now, we hadn’t levied any requirements, rigor or evidence on the innovator to understand what it would take to integrate, scale and deploy products/services.

Finally, most corporate/agency innovation processes funnel “innovations” into “demo days” or “shark tanks” where they face an approval/funding committee that decides which innovation ideas are worth pursuing. However, without any measurable milestones to show evidence of the evolution of what the team has learned about validity of the problem, customer needs, pivots, etc., the best presenter and flashiest demo usually win.

In some companies and government agencies, innovators even have informal groups, i.e. an Innovators Alliance, where they can exchange best practices and workarounds to the system. (Think of this as the innovator’s support group.) But these innovation activities are ad hoc, and the innovators lack authority, resources and formal process to make innovation programs an integral part of their departments or agencies.

Innovators vs. Entrepreneurs
There are two types of people who engage in large company/agency innovation: Innovators – those who invent new technology, product, service or processes; and Entrepreneurs – those who’ve figured out how to get innovation adopted and delivered through the existing company/agency procedures and processes. Although some individuals operate as both innovator and entrepreneur, any successful innovation program requires an individual or a team with at least these two skill sets. (More detail can be found here.)

Innovation Tools and Activities
Over the last decade, innovators have realized that they needed tools and activities different from traditional project management tools used for new versions of existing products/customers.They have passionately embraced innovation tools and activities that for the first time help individual innovators figure out what to build, who to build it for and how to create effective prototypes and demos.

Some examples of innovation tools are Customer Development, Design Thinking, User-Centric Design, Business Model Canvas, Storytelling, etc. Companies/agencies have also co-opted innovation activities developed for startups such as Hackathons, Incubators, internal Kickstarters, as well as Open Innovation programs and Maker Spaces that give individual innovators a physical space and dedicated time to build prototypes and demos. In addition, companies and agencies have set up Innovation Outposts (most often located in Silicon Valley) to be closer to relevant technology and then to invest, partner or buy.

These activities make sense in a startup ecosystem (where 100% of the company is focused on innovation,) however they generate disappointing results inside companies/agencies (when 98% of the organization is focused on executing the existing business/mission model.) While these tools and activities educated innovators and generated demos and prototypes, they lacked an end-to-end process that focused on delivery/deployment. So it should be no surprise that very few contributed to the company’s top or bottom line (or an agency’s mission).

One of the ironies of the tools/activities groups is rather than talking about the results of using the tools – i.e. the ability to rapidly deliver new products/services that are wanted and needed – their passion has them evangelizing the features of the tools and activities. This means that senior leadership has pigeonholed most of these groups as extensions of corporate training departments and skeptics view this as the “latest fad.”

Team-based Innovation
Rather than just teaching innovators how to use new tools or having them build demos, we recognized that there was a need for a process that taught all the components of a business/mission model (who are the customers, what product/service solves their problem, how do we get it to them, support it, etc.) The next step in entrepreneurial education was to teach teams a formal innovation process for how to gather evidence that lets them test if their idea is feasible, desirable and viable. Examples of team-based innovation programs are the National Science Foundation Innovation Corps (I-Corps @ NSF), for the Intelligence Community I‑Corps@ NSA, and for the Department of Defense, Hacking for Defense (H4D).

In contrast to single-purpose activities like Incubators, Hackathons, Kickstarters, etc., these curricula teach what it takes to turn an idea into a deliverable product/service by using the scientific method of hypothesis testing and experimentation outside the building. This process emphasizes rapid learning cycles with speed, urgency, accepting failure as learning, and innovation metrics.

Teams talk to 100+ beneficiaries and stakeholders while building minimal viable products to maximize learning and discovery. They leave the program with a deep understanding of all the obstacles and resources needed to deliver/deploy a product.

The good news – I-Corps, Hacking for Defense and other innovation programs that focus on training single teams have raised the innovation bar. These programs have taught thousands of teams of federally funded scientists as well as innovators in corporations, the Department of Defense and intelligence community. However, over time we’ve seen teams that completed these programs run into scaling challenges. Even with great evidence-based minimal viable products (prototypes), teams struggled to get these innovations deployed at scale and in the field. Or a team that achieved product-market fit building a non-standard architecture could find no way to maintain it at scale within the parent organization.

Upon reflection we identified two root causes. The first is a lack of connection between innovation teams and their parent organization. Teams form/and are taught outside of their parent organization because innovation is disconnected from other activities. This meant that when teams went back to their home organization, they found that execution of existing priorities took precedence. They returned speaking a foreign language (What’s a pivot? Minimum viable what?) to their colleagues and bosses who are rewarded on execution-based metrics. Further, as budgets are planned out years in advance, their organization had no slack for “good ideas.” As a result, there was no way to finish and deploy whatever innovative prototypes the innovators had developed – even ones that have been validated.

The second root cause emerged because neither the innovator’s teams nor their organizations had the mandate, budget or people to build an end-to-end innovation pipeline process, one that started with innovation sourcing funnel (both internal and external sources) all the way to integrating their prototypes into mainstream engineering production. (see below and this HBR article on the innovation pipeline.)

Operational Innovation
As organizations have moved from – individual innovators working alone, to adopting innovation tools and activities, to teaching teams about evidence-based innovation – our most important realization has been this: Having skills/tools and activities are critical building blocks but by themselves are insufficient to build a program that delivers results that matter to leadership.  It’s only when senior leaders see how an innovation process can deliver stuff that matters – at speed—that they take action to change the processes and procedures that get in the way.

We believe that the next big step is to get teams and leaders to think about the innovation process from end-to-end – that is to visualize the entire flow of how and from where an idea is generated (the source) all the way to deployment (how it gets into users’ hands). So, we’ve drawn a canonical innovation pipeline. (The HBR article here describes it in detail.) For context, in the figure below, the I-Corps program described earlier is the box labeled “Solution Exploration/Hypotheses Testing.” We’ve surrounded that process with all the parts necessary to build and deliver products and services at speed and at scale.

Second, we’ve realized that while individual initiatives won “awards,” and Incubators and Hackathons got coffee cups and posters, senior leadership sat up and took notice when operating groups transformed how they work in the service of a critical product or mission. When teams in operating groups adopted the innovation pipeline, it made an immediate impact on delivering products/services at speed.

An operating group can be a corporate profit and loss center or anything that affects revenue, profit, users, market share, etc. In a government agency it can be something that allows a group to execute mission more effectively or in a new disruptive way. Operating groups have visibility, credibility and most importantly direct relevance to mission.

Where are these groups? In every large company or agency there are groups solving operational problems that realize “they can’t go on like this” and/or “we need to do a lot more stuff” and/or “something changed, and we rapidly need to find new ways to do business.” These groups are ready to try something new. Most importantly we learned that “the something new” is emphatically not more tools or activities (design thinking, user-centric design, storytelling, hackathons, incubators, etc.) Because these groups want an end-to-end solution, the innovation pipeline resonates with the “do’ers” who lead these groups.

(One example of moving up the Innovation Stack is that the NSA I-Corps team has recently shifted their focus from working with individual teams to helping organizations deploy the methodology at scale.  In true lean startup fashion, they are actively testing a number of approaches with a variety of internal organizations ranging in size from 40 to 1000+ people.)

However, without a mandate for actually delivering innovation from senior leadership, scaling innovation across the company/agency means finding one group at a time – until you reach a tipping point of recognition. That’s when leadership starts to pay attention. Our experience to date is that 25- to 150-person groups run by internal entrepreneurs with budget and authority to solve critical problems are the right place to start to implement this. Finding these people in large companies/agencies is a repeatable process. It requires patient and persistent customer discovery inside your company/agency to find these groups and deeply understand their pains/gains and jobs to be done.

Lessons Learned

  • Companies/agencies have adapted and adopted startup innovation tools
    • Lean, Design Thinking, User-centric Design, Business Model Canvas, etc.
  • As well as startup activities and team-based innovation 
    • Hackathons, Incubators, Kickstarters, I-Corps, FastWorks, etc.
  • Because they are disconnected from the mainstream business/mission model very few have been able to scale past a demo/prototype
  • Use the Innovation Stack and start working directly with operating groups
    • Find those who realize “they can’t go on like this” and/or “we need to do a lot more stuff” and/or “something changed, and we rapidly need to find new ways to do business”
  • You’ll deliver stuff that matters instead of coffee cups
%d bloggers like this: