The Red Queen Problem – Innovation in the DoD and Intelligence Community

“…it takes all the running you can do to keep in the same place. ”
The Red Queen Alice in Wonderland

Innovation, disruption, accelerators, have all become urgent buzzwords in the Department of Defense and Intelligence community. They are a reaction to the “red queen problem” but aren’t actually solving the problem. Here’s why.


In the 20th century our nation faced a single adversary – the Soviet Union. During the Cold War the threat from the Soviets was quantifiable and often predictable. We could specify requirements, budget and acquire weapons based on a known foe. We could design warfighting tactics based on knowing the tactics of our opponent. Our defense department and intelligence community owned proprietary advanced tools and technology. We and our contractors had the best technology domain experts. We could design and manufacture the best systems. We used these tools to keep pace with the Soviet threats and eventually used silicon, semiconductors and stealth to create an offset strategy to leapfrog their military.

That approach doesn’t work anymore. In the 21st century you need a scorecard to keep track of the threats: Russia, China, North Korea, Iran, ISIS in Yemen/Libya/Philippines, Taliban, Al-Qaeda, hackers for hire, etc. Some are strategic peers, some are near peers in specific areas, some are threats as non-state disrupters operating with no rules.

In addition to the proliferation of threats, most of the tools and technologies that were uniquely held by the DoD/IC or only within the reach of large nation states are now commercially available (Cyber, GPS, semiconductors, analytics, centrifuges, drones, genetic engineering, agile and lean methodologies, ubiquitous Internet, crypto and smartphones, etc.). In most industries, manufacturing is no longer a core competence of the U.S.

U.S. agencies that historically owned technology superiority and fielded cutting-edge technologies now find that off-the-shelf solutions may be more advanced than the solutions they are working on, or that adversaries can rapidly create asymmetric responses using these readily available technologies.

The result is that our systems, organizations, headcount and budget – designed for 20th century weapons procurements and warfighting tactics on a predictable basis – can’t scale to meet all these simultaneous and unpredictable challenges. Today, our DoD and national security agencies are running as hard as they can just to stay in place, but our adversaries are continually innovating faster than our traditional systems can respond. They have gotten inside our OODA loop (Observe, Orient, Decide and Act).

We believe that continuous disruption can only be met with a commitment to continuous innovation.

Pete Newell and I have spent a lot of time bringing continuous innovation to government organizations. Newell ran the U.S. Army’s Rapid Equipping Force on the battlefields of Iraq and Afghanistan finding and deploying technology solutions against agile insurgents. He’s spent the last four years in Silicon Valley out of uniform continuing that work. I’ve spent the last six years teaching our country’s scientists how to rapidly turn scientific breakthroughs into deliverable products by creating the curriculum for the National Science Foundation Innovation Corps – now taught in 53 universities. Together Pete, Joe Felter and I created Hacking for Defense, a nationwide program to teach university students how use Lean methodologies to solve defense and national security problems.

The solution to continuous disruption requires new ways to think about, organize, and build and deploy national security people, organizations and solutions.

Here are our thoughts about how to confront the Red Queen trap and adapt a government agency to infuse continuous innovation in its culture and practices.

Problem 1: Regardless of a high-level understanding that business as usual can’t go on, all agencies are given “guidance and metrics (what they are supposed to do (their “mission”) and how they are supposed to measure success). To no one’s surprise the guidance is “business as usual but more of it.” And to fulfill that guidance agencies create structure (divisions, directorates, etc.) designed to execute repeatable processes and procedures to deliver solutions that meet the requirements of the overall guidance.

Inevitably, while all of our defense and national security agencies will tell you that innovation is one of their pillars, innovation actually is an ill-defined and amorphous aspirational goal, while the people, budget and organization continue to flow to execution of mission (as per guidance.)

There is no guidance or acknowledgement that in our national security agencies, even as we execute our current mission, our capabilities decline every year due to security breaches, technology timing out, tradecraft obsolescence, etc. And there is no explicit requirement for creation of new capabilities that give us the advantage.

Solution 1: Extend agency guidance to include the requirements to create a continuous innovation process that a) resupplies the continual attrition of capabilities and b) creates new capabilities that gives us a mission advantage. The result will be agency leadership creating new organizational structures that make innovation a continual process rather than an ad hoc series of heroic efforts.

Problem 2: The word “Innovation” actually describes three very different types of activities.

Solution 2: Use the McKinsey Three Horizons Model to differentiate among the three types. Horizon 1 ideas provide continuous innovation to a company’s existing mission model and core capabilities. Horizon 2 ideas extend a company’s existing mission model and core capabilities to new stakeholders, customers, or targets. Horizon 3 is the creation of new capabilities to take advantage of or respond to disruptive technologies/opportunities or to counter disruption.

We’d add a new category, Horizon 0, which kills ideas that are not viable or feasible (something that Silicon Valley is tremendously efficient at doing).

These Horizons also apply to government agencies and other large organizations. Agencies and commands need to support all three horizons.

Problem 3: Risk equals failure and failure is to be avoided as it indicates a lack of competence.

Solution 3: The three-horizon model allows everyone to understand that failure in a Horizon 1/existing mission activity is different than failure in a Horizon 3 “never been done before” activity. We want to take risks in Horizon 3. If we aren’t failing with some efforts, we aren’t trying hard enough. An innovation process embraces and understands the different types of failure and risk.

Problem 4: Innovators tend to create activities rather than deployable solutions that can be used on the battlefield or by the mission. Accelerators, hubs, cafes, open-sourcing, crowd-souring, maker spaces, Chief Innovation Officers, etc. are all great but they tend to create innovation theater – lots of motion but no action. Great demos are shown and there are lots of coffee cups and posters, but if you look at the deliverables for the mission over a period of years the result is disappointing. Most of the executors and operators have seen little or no value from any of these activities. While the activities individually may produce things of value, they aren’t valued within the communities they serve because they aren’t connected to a complete pipeline that harnesses that value and turns it into a deliverable on the battlefield where it matters.

Solution 4: What we have been missing is an innovation pipeline focused on deployment not demos.

The Lean Innovation process is a self-regulating, evidence-based innovation pipeline. It is a process that operates with speed and urgency, where innovators and stakeholders curate and prioritize their own problems/Challenges/ideas/technology. It is evidence based, data driven, accountable, disciplined, rapid and mission- and deployment-focused.

The process recognizes that Innovation isn’t a single activity (an incubator, a class, etc.) it is a process from start to deployment.
The canonical innovation pipeline:

As you see in the diagram, there are 6 steps to the innovation pipeline: sourcing, challenge/curation, prioritization, solution exploration and hypothesis testing, incubation and integration.

Innovation sourcing: a list of problems/challenges, ideas, and technologies that might be worth investing in. These can come from hackathons, research groups, needs from operators in the field, etc.

Challenge/Curation: innovators get out of their own offices and talk to colleagues and customers with the goal of finding other places in the DoD where a problem or challenge might exist in a slightly different form, to identify related internal projects already in existence, and to find commercially available solutions to problems. It also seeks to identify legal issues, security issues, and support issues.

This process also helps identify who the customers for possible solutions would be, who the internal stakeholders would be, and even what initial minimum viable products might look like.

This phase also includes building initial minimal viable products (MVPs.) Some ideas drop out when the team recognizes that they may be technically, financially, or legally unfeasible or they may discover that other groups have already built a similar product.

Prioritization: Once a list of innovation ideas has been refined by curation, it needs to be prioritized using the McKinsey Three Horizons Model.

Once projects have been classified, the team prioritizes them, starting by asking: is this project worth pursing for another few months full time? This prioritization is not done by a committee of executives but by the innovation teams themselves.

Solution exploration and hypotheses testing: The ideas that pass through the prioritization filter enter an incubation process like Hacking for Defense/I-Corps, the system adopted by all U.S. government federal research agencies to turn ideas into products.

This six- to ten-week process delivers evidence for defensible, data-based decisions. For each idea, the innovation team fills out a mission model canvas. Everything on that canvas is a hypothesis. This not only includes the obvious – is there solution/mission fit? — but the other “gotchas” that innovators always seem to forget. The framework has the team talking not just to potential customers but also with people responsible for legal, support, contracting, policy, and finance. It also requires that they think through compatibility, scalability and deployment long before this gets presented to engineering. There is now another major milestone for the team: to show compelling evidence that this project deserves to be a new mainstream capability. Alternatively, the team might decide that it should be spun into its own organization or that it should be killed.

Incubation: Once hypothesis testing is complete, many projects will still need a period of incubation as the teams championing the projects gather additional data about the application, further build the minimum viable product (MVP), and get used to working together. Incubation requires dedicated leadership oversight from the horizon 1 organization to insure the fledgling project does not die of malnutrition (a lack of access to resources) or become an orphan (continue to work with no parent to guide them).

Integration and refactoring: At this point, if the innovation is Horizon 1 or 2, its time to integrate it into the existing organization. (Horizon 3 innovations are more likely set up as their own entities or at least divisions.) Trying to integrate new, unbudgeted, and unscheduled innovation projects into an engineering organization that has line item budgets for people and resources results in chaos and frustration. In addition, innovation projects carry both technical and organizational debt. This creates an impedance mismatch between the organizations that can be easily be resolved with a small dedicated refactoring team. Innovation then becomes a continuous cycle rather than a bottleneck.

Problem 5: The question being asked across the Department of Defense and national security community is, “Can we innovate like startups in Silicon Valley” and insert speed, urgency and agility into our work?

Solution 5: The reality is that the DoD/IC is not Silicon Valley. In fact, it’s much more like a large company with existing customers, existing products and the organizations built to support and service them. And much like large companies they are being disrupted by forces outside their control.

But what’s unique is, that unlike a large company that doesn’t know how to move rapidly, on the battlefields of Iraq and Afghanistan our combatant commands and national security community were more agile, creative and Lean than any startup. They wrote the book on how to collaborate (read Team of Teams) or adopt new technologies (see the Rapid Equipping Force.) The problem isn’t that these agencies and commands don’t know how to be innovative. The problem is they don’t know how to be innovative in peacetime when innovation succumbs to the daily demands of execution. Part of the reason is that large agencies are run by leaders who tend to be excellent Horizon 1 managers of existing people, process and resources but have no experience in building and leading Horizon 3 organizations.

The solution is to understand that an innovation pipeline requires different people, processes, procedures, and metrics, then execution.

Problem 6: How to get started? How to get leadership behind continuous innovation?

Solution 6: To leadership, incubators, cafes, accelerators and hackathons appear to be just background noise unrelated to their guidance and mission. Part of the problem lies with the innovators themselves. Lots of innovation activities celebrate the creation of demos, funding, new makerspaces, etc. but there is little accountability for the actual rapid deployment of useful tools. Once we can convince and demonstrate to leadership that continuous innovation can solve the Red Queen problem, we’ll have their attention and support.

We know how to do this. Our country requires it.
Let’s get started.

Lessons Learned

  • Organizations must constantly adapt and evolve, to survive when pitted against ever-evolving opposition in an ever-changing environment
  • Government agencies need to both innovate and execute
  • In peacetime innovation succumbs to the demands of execution
  • We need explicit guidance for innovation to agencies and their leadership requiring an innovation organization and process, that operates in parallel with the execution of current mission
  • We need an innovation pipeline that delivers rapid results, not separate, disconnected innovation activities

Office of Naval Research (ONR) Goes Lean

The Office of Naval Research (ONR) has been one of the largest supporters of innovation in the U.S. Now they are starting to use the Lean Innovation process (see here and here) to turn ideas into solutions. The result will be defense innovation with speed and urgency.

—-

Here’s how the Office of Naval Research (ONR) was started. In World War II the U.S. set up the Office of Scientific Research and Development (OSRD) to use thousands of civilian scientists in universities to build advanced technology weapons (radar, rockets, sonar, electronic warfare, nuclear weapons.) After the war, the U.S. Navy adopted the OSRD model and set up the Office of Naval Research – ONR. Since 1946 ONR has funded basic and applied science, as well as advanced technology development, in universities across the U.S. (Stanford’s first grants for their microwave and electronic lab came from ONR in 1946.)

Rich Carlin heads up ONR’s Sea Warfare and Weapons Department. He’s responsible for science and programs for surface ships, submarines, and undersea weapons with an annual budget of over $300 million per year.

Rich realized that while the Department of Defense DoD spends a lot of money and has lots of requirements and acquisition processes, they don’t work well with a rapid innovation ecosystem. He wanted to build an innovation pipeline that would allow the Navy to:

  • Create “dual-use” products (build solutions that could be used for the military but also sold commercially, and attract venture capital investments.) “Dual-use” products reduce the cost for defense adoption of products.
  • Test if the Lean Innovation process actually accelerates technology adoption and an innovation ecosystem.
  • Use best practices in contracting that accelerate awards and provide flexibility and speed in technology maturation and adoption.

Today ONR has taken the Lean Innovation process, adapted it for their agency, and is running pilots for defense innovation teams.

Lean Innovation is a Process
The Lean Innovation process is a self-regulating, evidence-based innovation pipeline. It is a process that operates with speed and urgency. Innovators and stakeholders curate and prioritize their own problems/Challenges/ideas/technology.

The process recognizes that Innovation isn’t a single activity (an incubator, a class, etc.). It is a process from start to deployment.

The ONR pipeline has all the steps of the canonical innovation pipeline:

Innovation sourcing: a list of problems/challenges, ideas, and technologies that might be worth investing in.

Problem/Challenge Curation: innovators get out of their own offices and talk to colleagues and customers with the goal of finding other places in the DoD where a problem or challenge might exist in a slightly different form, identifying related internal projects already in existence, and finding commercially available solutions to problems. They also seek to identify legal issues, security issues, and support issues.

This process also helps identify who the customers for possible solutions would be, who the internal stakeholders would be, and even what initial minimum viable products (MVPs) might look like.

This phase also includes building initial MVPs. Some ideas drop out when the team recognizes that they may be technically, financially, or legally unfeasible or they may discover that other groups have already built a similar product.

Prioritization: Once a list of innovation ideas has been refined by curation, it needs to be prioritized using the McKinsey Three Horizons Model. Horizon 1 ideas provide continuous innovation to a company’s existing business model and core capabilities. Horizon 2 ideas extend a company’s existing business model and core capabilities to new customers, markets or targets. Horizon 3 is the creation of new capabilities to take advantage of or respond to disruptive opportunities or disruption. We added a new category, Horizon 0, which refers to graveyard ideas that are not viable or feasible.

Once projects have been classified, the team prioritizes them, starting by asking: is this project worth pursing for another few months full time? This prioritization is not done by a committee of executives but by the innovation teams themselves.

Solution exploration and hypotheses testing: The ideas that pass through the prioritization filter enter an incubation process like Hacking for Defense/I-Corps, the system adopted by all U.S. government federal research agencies to turn ideas into products.

This six- to ten-week process delivers evidence for defensible, data-based decisions. For each idea, the innovation team fills out a mission model canvas. Everything on that canvas is a hypothesis. This not only includes the obvious -is there solution/mission fit? — but the other “gotchas” that innovators always seem to forget. The framework has the team talking not just to potential customers but also with regulators, and people responsible for legal, contracting, policy, and finance support.  It also requires that they think through compatibility, scalability and deployment long before this gets presented to engineering. There is now another major milestone for the team: to show compelling evidence that this project deserves to be a new mainstream capability. Alternatively, the team might decide that it should be spun into its own organization or that it should be killed.

Incubation: Once hypothesis testing is complete, many projects will still need a period of incubation as the teams championing the projects gather additional data about the application, further build the MVP, and get used to working together. Incubation requires dedicated leadership oversight from the horizon 1 organization to insure the fledgling project does not die of malnutrition (a lack of access to resources) or become an orphan (no parent to guide them).

Lean Innovation Inside the Office of Naval Research (ONR)
To come up with their version of the innovation pipeline ONR mapped four unique elements.

First, ONR is using Hacking for Defense classes to curate “Problem Statements” (ONR calls them Challenge/Opportunity Statements) to find solution/mission fit and commercial success.

Second, they’re using existing defense funding to prove out these solutions depending on the level of technical maturity. (There are three existing sources for funding defense innovation: COTS/GOTS validation (testing whether off-the-shelf  products can be used); Concept Validation and Technology Advancement; and SBIR/STTR funds – there’s over >$1B per year in the DoD SBIR program alone.)

Third, they are going to use Pete Newell’s company, BMNT and other business accelerators to apply Lean Launchpad Methodologies to build the business case for resulting prototypes and products and to attract private investments.

Fourth, they are going to use grants, purchase orders and Other Transaction Agreements (OTAs) to attract startups and nontraditional defense contractors, speed the award process, and provide startups the flexibility to pivot their business model and prototype/product solution when necessary.

BMNT and Hacking for Defense serve as the essential crosslink for tying together the assets already available in DoD to implement the Lean Innovation process for defense innovation.

Lessons Learned

  • The Office of Naval Research has been funding innovation in universities for 70+ years
  • They are piloting the Lean Innovation Process to move defense innovation forward with speed and urgency

Removing the Roadblocks to Corporate Innovation – When Theory Meets Practice

Innovation theory and innovation in practice are radically different. Here are some simple tools to get your company’s innovation pipeline through the obstacles it will encounter.

Pete Newell and I’ve been working with Karl, the Chief Innovation Officer of a large diversified multi-billion-dollar company I’ll call Spacely Industries. Over the last 15 months his staff got innovation teams operating with speed and urgency. The innovation pipeline had been rationalized. His groups whole-heartedly adopted and adapted Lean. His innovators and stakeholders curated and prioritized their problems/idea/technology before handing them off.

Karl’s innovation pipeline had hundreds of employees going through weekend hackathons. 14 different innovation teams were going through a 3-month I-Corps/Lean LaunchPad program to validate product/market fit. His organization had a corporate incubator for disruptive (horizon 3) experiments and provided innovation support for his company’s operating divisions for process and business model innovations (Horizon 1 and 2.)  Karl’s innovation process looked like this:

(Read our last post for the details.)

Given all this activity we were surprised that in our last call with Karl he started with “I think they’ve beaten me down.” He listed the internal obstacles his teams continued to face, “We’ve created innovation teams in both the business units and in corporate. Our CEO is behind the program. The division general managers have verbally given us their support. But we’re 15 months into the program and the teams still run into continual roadblocks and immovable obstacles in every part of the company. Finance, HR, Legal, Security, Policy, Branding, you name it, everyone in a division or corporate staff has an excuse for why we can’t do something, and everyone has the power to say no and no urgency to make a change.”

The reality is that Karl’s innovation pipeline looked like this:

And worse, every time Karl negotiated to remove a roadblock for one team, he was back negotiating 6 months later to remove the very same one for a different team.

Karl was frustrated, “How do we get all these organizations to help us move forward with innovation? My CEO has been reading all the Lean Innovation books and wants to fix this and is ready to bring in a big consulting firm to redo all our business processes.”

Uh oh.

Recognizing the Roadblocks
Most of the impediments the innovation teams faced were pretty tactical: for example, an HR policy that said the innovative groups could only recruit employees by seniority. Or a branding group that refused to allow any form of the Spacely name to appear on any minimal viable product or web site. Or legal, who said minimal viable products opened the company to lawsuits. Or sales, who shut out innovation groups from doing customer discovery with any existing, or even potential customer. Or finance, who insisted on measuring the success of new ventures on their first year’s revenue and gross margin. Or purchasing who insisting that the teams use their existing vendors that took weeks to deliver simple purchases that Amazon would deliver overnight.

Listening to this we realized that while the company had playbooks for execution, what was missing were specific processes for innovation. We agreed that the goal was not to change any of the existing execution processes, procedures, incentives, metrics but rather to work with all the organizations to write new ones for innovation projects.

And the specific innovation policies would grow one at a time as needed collaboratively from the bottom of the organization, not top down by some executive mandate or consulting firm.

Read this section again. The creation of the innovation policies incrementally by the innovation groups collaborating with the service organizations is a very big idea.

We pointed out to Karl, that if he was successful, innovation and execution policies, processes, procedures, incentives, metrics would then co-exist side-by-side. In their day-to-day activities, the support organizations would simply ask, “Are we supporting an execution process (hopefully 90% of the time) or are we supporting an innovation process?” and apply the appropriate policy.

We offered that a top-down revamp of every business process should be a last resort. We suggested that Karl consider trying a 6-month experimental “Get to Yes” program. (It was critical that his CEO is a supporter.)

Get to Yes
Step one was that Karl needed a single memo from his CEO to his direct reports. We suggested that Karl ask for something like this:


Step two was that Karl needed a memo that went out to all department heads. The one from Finance looked like this:

The “Get to Yes” Collaboration
Every time an innovation team needed a new policy, procedure, etc. from an existing organization (legal, finance, sales, HR, branding, etc.), they worked with their designated point of contact (Finance, HR, Legal, etc.) Working together the two groups used the single-page standard Spacely “Get to Yes” form. It asked what problem the team was trying to solve that required a policy change, how it wanted the new policy to read, the impact the new policy would have on other policies and organizations, and most importantly the risks to the core existing business.

The “Get to Yes” request form looked like this:

In practice, once the finance, legal, and other organizations had gotten top-down guidance that innovation needed solutions to move forward, in 70% of the instances the approvals could be worked out at the lowest part of both organizations.

For the times that the innovation team and point of contact couldn’t agree, the issue got escalated to the appeals board of the appropriate department (legal, hr, security, finance, etc.) They had one week to ask questions, gather information, meet with the innovation team and evaluate the costs and risks of the proposed process. They could either:

  1. approve and adopt
  2. suggest modifications that the team agreed with
  3. deny the request.

The escalation form looked like this:

Although it was just a one-page form, the entire concept was radical:

  • The innovation team would be proposing the new process, procedure, metric, etc. – not waiting for ones to be written.
  • There was a hard 1-week deadline for the execution team to respond.
  • Yes, was the default answer, a No required detailed explanation.

For the few issues that the innovation program thought needed the executive staff attention, one further appeal was possible to the Chief Innovation Officer.

The big idea is that Spacely was going to create innovation by design, not by exception, and they were going to do it by co-opting the existing execution machinery.

The key to making this work was: 1) a simple top-down request from the CEO to their direct reports and from their direct reports down their organization, 2) a collaborative solution seeking partnership between innovation and execution teams, 3) an appeals and escalation process that would be acted on within the next week. And it’s there that the execution department had to make its case of why this request should not be approved. (If there was still no agreement, it became an issue for the executive staff.)

The time for a process resolution in a billion-dollar corporation – two weeks. At Spacely innovation was starting to move at the speed of a startup.

(BTW, when we showed this process to a government agency their first response was, “Oh that will work in a company, but in a government agency we’re bound by laws and regulations that mandate specifically what we are allowed to do.”  So, we modified the Get to Yes program to ensure that a “No” had to specifically point to the law or regulation, not agency legend, then confirm there was no potential exception to the regulation or law that could be applied.)

Lessons Learned

  • Explicit top down support for innovation only takes a single-page memo
    • If you can’t get leadership support, internal innovation won’t happen
  • Innovation process and procedures developed by collaboration, not by exception
  • Execution organizations now manage both execution and innovation
  • Innovation policy, process and procedures get written as needed, one at a time
  • Over time a set of innovation processes are created from the bottom up
  • Bias for immediate action not perpetual delay

How companies strangle innovation – and how you can get it right

 

A shorter version of this post first appeared on the HBR blog

I just watched a very smart company try to manage innovation by hiring a global consulting firm to offload engineering from “distractions.” They accomplished their goal, but at a huge, unanticipated cost: the processes and committees they designed ended up strangling innovation.

There’s a much better way.


An existing company or government organization is primarily organized for day-to-day execution of its current business processes or mission. From the point of view of the executors, having too many innovation ideas gets in the way of execution.

The Tidal Wave of Unfiltered Ideas
Pete Newell and I were working with a company that was getting its butt kicked from near-peer competitors as well as from a wave of well-funded insurgent startups. This was a very large and established tech company; its engineering organization developed the core day-to-day capabilities of the organization. Engineering continually felt overwhelmed. They were trying to keep up with providing the core services necessary to run the current business and at the same time deal with a flood of well-meaning but uncoordinated ideas about new features, technologies and innovations coming at them from all directions. It didn’t help that “innovation” was the new hot-button buzzword from senior leadership, and incubators were sprouting in every division of their company, it just made their job more unmanageable.

One of the senior engineering directors I greatly admire (who at one time or another had managed their largest technology groups) described the problem in pretty graphic terms:

The volume of ideas creates a denial of service attack against capability developers, furthers technical debt, and further encumbers the dollars that should be applied towards better innovation.”

Essentially, the engineering organization was saying that innovation without a filter was as bad as no innovation at all. So, in response the company had hired a global consulting firm to help solve the problem. After a year of analysis and millions of dollars in consulting fees, the result was a set of formal processes and committees to help create a rational innovation pipeline. They would narrow down the proposed ideas and choose which ones to fund and staff.

Build the Wall
I took one look at the process they came up with and could have sworn that it was invented by the company’s competitors to throttle innovation.

The new innovation process had lots of paperwork – committees, application forms and presentations, and pitches. People with ideas, technology or problems pitched in front of the evaluation committee. It seemed to make sense to have have all the parties represented at the committee, so lots of people attended – program managers who controlled the budget, the developers responsible for maintaining and enhancing the current product and building new ones, and representatives from the operating divisions who needed and would use these products. Someone with an idea would fill out the paperwork justifying the need for this innovation, it would go to the needs committee, and then to an overall needs assessment board to see if the idea was worth assigning people and budget to. And oh, since the innovation wasn’t in this year’s budget, it would only get started in the next year.

Seriously.

As you can guess in the nine months this process has been in place the company has approved no new innovation initiatives. But new unbudgeted and unplanned threats kept emerging at a speed their organization couldn’t respond to.

At least it succeeded in not distracting the developers.

This was done by smart, well-intentioned adults thinking they were doing the right thing for their company and consultants who thought this was great innovation advice.

What went wrong here? Three common mistakes.

First, this company (and most others) viewed innovation as unconstrained activities with no discipline. In reality for innovation to contribute to a company or government agency, it needs to be designed a process from start to deployment.

Second, the company had not factored in that their technology advantage attrited every year, and new threats would appear faster than their current systems could handle. Ironically, by standing still, they were falling behind.

Third, this company had no formal innovation pipeline process before proposals went to the committee. Approvals tended to be based on who had the best demo and/or slides or lobbied the hardest. There was no burden on those who proposed a new idea or technology to talk to customers, build minimal viable products, test hypotheses or understand the barriers to deployment. The company had a series of uncoordinated tools and methodologies as activities, but nothing to generate evidence to refine the ideas, technology or problems as an integrated innovation process (though they did have a great incubator with wonderful coffee cups). There were no requirements for the innovator. Instead the process dumped all of these “innovations” onto well-intentioned, smart people sitting in a committee who thought they could precompute whether these innovation ideas were worth pursuing.

An Innovation Process and Pipeline
What the company needed was a self-regulating, evidence-based innovation pipeline. Instead of having a committee vet ideas, they needed a process that operated with speed and urgency, and innovators and stakeholders who curated and prioritized their own problems/idea/technology.

All of this would occur before any new idea, tech or problem hit engineering. This way, the innovations that reached engineering would already have substantial evidence – about validated customer needs, processes, legal, security and integration issues identified — and most importantly, minimal viable products and working prototypes already tested. A canonical Lean Innovation process inside a company or government agency would look something like this:

Curation
As the head of the U.S. Army’s Rapid Equipping Force, Pete Newell built a battle-tested process to get technology solutions deployed rapidly. This process, called Curation, gets innovators to work through a formal process of getting out of their offices and understanding:

Internal and External Survey

  • Other places the problem might exist in a slightly different form
    • Internal projects already in existence
    • Commercially available solutions
  • Legal issues
  • Security issues
  • Support issues

Use Cases/Concept of Operations

  • Who are the customers? Stakeholders? Other players?
  • How did they interact? Pains/Gains/Jobs to be done?
  • How does the proposed solution work from the viewpoint of the users?
  • What would the initial minimal viable products (MVPs) – incremental and iterative solutions – look like?

In the meantime, the innovators would begin to build initial minimal viable products (MVPs) – incremental and iterative tests of key hypotheses. Some ideas will drop out when the team itself recognizes that they may be technically, financially or legally unfeasible or they may discover that other groups have already built a similar product.

Prioritization
One of the quickest ways to sort innovation ideas is to use the McKinsey Three Horizons Model. Horizon 1 ideas provide continuous innovation to a company’s existing business model and core capabilities. Horizon 2 ideas extend a company’s existing business model and core capabilities to new customers, markets or targets. Horizon 3 is the creation of new capabilities to take advantage of or respond to disruptive opportunities or disruption. And we added a new category, Horizon 0, which defers or graveyards ideas that are not viable or feasible.

At the end of this prioritization step, the teams meet another milestone: is this project worth pursing for another few months full time? A key concept of prioritization across all horizons is that this ranking is not done by a remote committee, but by the innovation teams themselves as an early step in their discovery process.

Solution Exploration/Hypotheses Testing
The ideas that pass through the prioritization filter enter an I-Corps incubation process. I-Corps was adopted by all U.S. government federal research agencies to turn ideas into products. Over a 1,000 teams of our country’s best scientists have gone through the program taught in over 50 universities. (Segments of the U.S. Department of Defense and Intelligence community have also adopted this model as the Hacking for Defense process.)

This six- to ten-week process delivers evidence for defensible, data-based decisions. It tests the initial idea against all the hypotheses in a business model (or for the government, the mission model) canvas. This not only includes the obvious — is there product/market (solution/mission) fit? — but the other “gotchas” that innovators always seem to forget. The framework has the team talking not just to potential customers but also with regulators, and people responsible for legal, policy, finance, support. It also requires that they think through compatibility, scalability and deployment long before this gets presented to engineering. There is now another major milestone for the team: to show compelling evidence that this project deserves to be a new mainstream capability and inserted into engineering. Or does it create a new capability that could be spun into its own organization? Or does the team think it should be killed?

Incubation
Once hypothesis testing is complete, many projects will still need a period of incubation as the teams championing the projects need to gather additional data about the application as well as may need to mature as a team before they are ready to integrate with a horizon 1 engineering organization or product division. Incubation requires dedicated leadership oversight from the horizon 1 organization to insure the fledgling project does not die of malnutrition (a lack of access to resources) or become an orphan (no parent to guide them).

Integration/Refactoring
Trying to integrate new, unbudgeted and unscheduled Horizon 1 and 2 innovation projects into an engineering organization that has line item budgets for people and resources results in chaos and frustration. In addition, innovation projects not only carry technical debt, but also organizational debt.

Technical debt describes what happens when software or hardware is built quickly to validate hypotheses and find early customers. This quick and dirty development results in software that can become unwieldy, difficult to maintain and incapable of scaling. Organizational debt is all the people/culture compromises made to “just get it done” in the early stages of an innovation project. You clean up technical debt through refactoring, by going into the existing code and restructuring it to make the code stable and understandable. You fix organizational debt by refactoring the team, realizing that most of the team who built and validated a prototype may not be the right team to take it to scale but is more valuable starting the next innovation initiative.

Often when an innovation pipeline runs head-on into a process-driven execution organization, chaos and finger-pointing ensues and adoption of new projects stall. To solve this problem we acknowledge that innovation projects will need to refactor both technical and organizational debt to become a mainstream product/service. To do so, the innovation pipeline has engineering set up a small refactoring organization to move these validated prototypes into production. In addition, to solve the problem that innovation is always unscheduled and unbudgeted, this group has a dedicated annual budget.

Disruptive Products
Some products and services going through the pipeline create new capabilities or open new markets. These Horizon 3 disruptive innovations need to separate from the existing development organizations and be allowed to grow and develop in physically separate spaces. They need the support and oversight of the CEO.

Fast forward a year, and slowly, like turning a supertanker, the innovation pipeline we proposed is taking shape. The company has adopted Lean language and process: curation, prioritization, three horizons, I-Corps – business/mission model canvas, customer development and agile engineering.

Lessons Learned

  • Every large company and government agency is dealing with disruption
  • Most have concluded that “business as usual” can’t go on
  • Yet while the top of the organization gets it, and the innovators on the bottom get it, there has been no relief for the engineering groups trying to keep the lights on
  • Innovation isn’t a single activity; it is a process from start to deployment
  • In “execution engines,” committees and broad stakeholder involvement make sense because experience, knowledge, and data from the past allow better decision-making
  • In “innovation engines” there isn’t the data to decide between competing ideas/projects (since nobody’s been in the future), so the teams need to gather facts outside their cubicle or building quickly
  • A self-regulating, evidence-based Lean Innovation process will deliver continuous innovation and disruptive breakthroughs with speed and urgency

Working Outside the Tech Bubble

Annual note to self – most of the world exists outside the tech bubble.

—–

We have a summer home in New England in a semi-rural area, just ~10,000 people in town, with a potato farm across the street. Drive down the road and you can see the tall stalks of corn waving on other farms. Most people aren’t in tech or law or teaching in universities; they fall solidly in what is called working-class. They work as electricians, carpenters, plumbers, in hospitals, restaurants, as clerks, office managers, farmers, etc. They have solid middle-class values of work, family, education and country – work hard, own a home, have a secure job, and save for their kids’ college and their retirement.

This summer I was sitting in the Delekta Pharmacy in the nearby town of Warren having a Coffee Cabinet (a coffee milkshake).  It’s one of the last drugstores with a real soda fountain. The summer tourists mostly come through on the weekend but during the week the locals come by to gab with the guy behind the counter. There are four small wooden booths along the wall in front of the fountain, and as I drank my Cabinet I got to overhear townie conversations from the other three booths.

Unlike every cafe I sit in the valley or San Francisco, their conversations were not about tech.

While they own tech, smartphones and computers, most can’t tell you who the ex-CEO of Uber is, or the details of the diversity blowup at Google. More important issues dominate their daily lives.

I was listening to one guy talk about how much his mortgage and kid’s college expenses were increasing while he hadn’t gotten a raise in three years and was worried about paying the bills. A woman talked about her husband, and how after 21 years as an electrician in the local hospital, he had just been laid-off. Others chimed in with their stories, best summarized by a feeling of economic anxiety. Of being squeezed with no real exit.

It was a long time ago, but I knew the feeling well.

I grew up in New York in a single-parent household that teetered on the bottom end of what today we’d call working class. My parents were immigrants and when they were divorced my mother supported us on the $125 a week she made as a bookkeeper. The bills got paid, and we had food in the house, but there was nothing extra left. No vacations. New clothes were bought once a year before school.

Years later when I got out of the Air Force, I installed broadband process control systems in automobile assembly plants and steel mills across the industrial heart of the Midwest. I got to see the peak of America’s manufacturing prowess in the 1970s, when we actually made things – before we shipped the factories and jobs overseas. I hung out with the guys who worked there, went bowling and shooting with them, complained about the same things, wives, girlfriends, jobs and bosses, and shared their same concerns.

Listening to these conversations in the Pharmacy, and the other stories I have heard as I explored the small towns here, reminded me that people I grew up with, served with and others I worked with, still live in this world. In fact, more than half of Americans fall into the working class. And the conversations I was listening to were a real-life narrative of the “middle-class squeeze.” While the economy has continued to grow, in the name of corporate efficiency and profitability we’ve closed the shipyards and factories and moved those jobs overseas. The bulk of those gains have ended up in the pockets of the very affluent. Income inequality stares you in the face here. The level of despair is high. The small city next to us has been hard hit in the opioid crisis: 63 people died last year.

My annual trek out here reminds me that that I live in a Silicon Valley bubble—and that a good part of the country is not reading what we read, caring about what we care about or thinking about what we think about. They have a lot more immediate concerns.

It’s good to spend time outside the bubble –  but I get to go back. My neighbors here, people in that pharmacy and the many others like them can’t. In the U.S. people used to move to where the jobs are. But today, Americans are less mobile. Some are rooted, embedded in their communities; and some are trapped — because housing is unaffordable where the better paying jobs are. And the jobs that are high paying are not the jobs they built their lives on. Likely their circumstances won’t have changed much by the time I return next year.

I don’t know how the people I listened to and talked to voted, but it’s easy to see why they might feel as if no one in Washington is living their lives.  And that the tech world is just as distant as Hollywood or Wall Street.

There isn’t an app to fix this.

You adapt, or you adopt, or you die

I was in Boston and stopped by the Harvard Business Review for their IdeaCast podcast. I shared my current thinking about innovation in companies and government agencies. The interviewer, Curt Nickisch was great and managed to get me to summarize several years of learning in one podcast.  He even got me to tell my Steve Jobs interview story.

It’s worth a listen.

Listen to the entire interview here:

Or listen to just parts of the interview:
3:29 Entrepreneurs make their own luck
4:35 The difference between an idea and an entrepreneur
5:18 Why entrepreneurship thrived in Silicon Valley
7:10 The pay-it-forward culture
7:53 Failure as part of the process
9:32 When I was more wrong than anyone on earth
11:20 Steve Jobs on Customer Development
12:38 The first time I did customer discovery
14:49 Engineers built products for themselves and the “next bench”
15:27 Why MBA’s avoided Silicon Valley
16:01 20th century investors were not entrepreneurs
16:45 Startups are not smaller versions of large companies
18:18 We needed a management stack for innovation
19:17 HBR and the Lean Startup and corporations
20:09 Why Lean fails in corporations
20:45 Startups can do anything, companies can only do what’s legal
22:0o The team
22:23 Rewards
23:00 What will drive continuous corporate innovation?
24:01 Innovation Theater
24:39 Innovation at speed
25:27 You adapt, or you adopt, or you die

Why Corporate Innovation is Harder Now

I was at Stanford in the Graduate School of Business and was interviewed by Peter Gardner of StartGrid for his On The Road podcast. I shared my current thinking about innovation in companies and government agencies.

It’s worth a listen.

BTW, it seems every podcast has a trick last question. This one was, “if I was on a road trip what’s the destination and what’s playing on the radio?”

Listen to the entire interview here:

Or just parts of the interview:

01:59     What drew me to entrepreneurship
03:43     What motivates me about innovation today
05:25     Entrepreneurship in the 20th century
06:30     The difference between large companies and startups
07:13     The Lean Startup Lightbulb moment
07:44     An MBA meant Master of Business Administration
08:18     Agile Development and Lean – Eric Ries
09:05     Business Model Canvas – Alexander Osterwalder
12:32     Innovation in Large Companies in the 20th Century
13:05     Startup Capital at Scale threatens large Companies
13:58     Startups operate with alacrity, agility and at times a death wish
15:06     Companies can only do things that are legal, while startups can do anything
16:00     Corporate defcon level – the wartime footing level
17:41     Innovation Theater
18:23     Did you move the top or bottom line?
19:50     Two types of 21st century corporations
20:55     Hedge funds and dual-class stock
22:28     Innovation pipeline not silos
24:25     Innovation Outposts
26:32     Why Innovators Leave Companies
27:30     We can’t afford to have our government go out of business
28:54    Why I turn on the Beach Boys

%d bloggers like this: