Software Once Led Us to the Precipice of Nuclear War. What Will AI Do?

A version of this story previously appeared in Defense One.

The story of RYAN and Able Archer is an oft-told lesson of a U.S. intelligence failure, miscalculation, and two superpowers unaware they were on the brink of an accidental nuclear war — all because the Soviet Union relied on a software program to make predictions that were based on false assumptions.

As more of our weapons systems and analytical and predictive systems become enabled by AI and Machine Learning, the lessons of RYAN and Able Archer is a cautionary tale for the DoD.


In 1983, the world’s superpowers drew near to accidental nuclear war, largely because the Soviet Union relied on software to make predictions that were based on false assumptions. Today, as the Pentagon moves to infuse artificial-intelligence tools into just about every aspect of its workings, it’s worth remembering the lessons of RYAN and Able Archer.

Two years earlier, the Soviet Union had deployed a software program dubbed RYAN, for Raketno Yadernoye Napadenie, or sudden nuclear missile attack. Massive for its time, RYAN sought to compute the relative power of the two superpowers by modeling 40,000 military, political, and economic factors, including 292 “indicators” reported from agents (spies) abroad. It was run by the KGB, which employed more than 200 people just to input the data.

The Soviets built RYAN to warn them when their country’s relative strength had declined to a point that the U.S. might launch a preemptive first strike on the Soviet Union. Leaders decided that if Soviet power was at least 70 percent of that of the United States the balance of power was stable. As the months went by, this number plummeted. By 1983, RYAN reported that Soviet power had declined to just 45 percent of that of the United States.

This amplified Soviet leaders’ paranoia. After 25 years of back-and-forth in the nuclear arms race, the Peacekeeper ICBM and the Trident SLBM were tipping the balance in favor of the United States. Responding to the Soviet introduction of SS-20 intermediate-range ballistic missiles to Eastern Europe in 1983, the U.S. deployed Pershing II and Ground Launched Cruise Missiles missiles to Western Europe, which reduced warning time of attack on Moscow to less than eight minutes. And in March 1983, President Reagan announced the Strategic Defense Initiative – “Star Wars” – to intercept Soviet ICBMs, then piled on just weeks later by publicly labeling the Soviet Union “the Evil Empire.” And to cap off a very bad year in the Cold War, in September 1983 the Soviets accidentally shot down a civilian 747 airliner—KAL 007—killing all 269 aboard.

By 1983, Soviet political and military leaders truly believed a nuclear war was coming. The RYAN program took on even greater importance. To feed RYAN, the KGB made its top priority to collect indicators of anything that might precede a potential surprise nuclear missile attack. They were looking for direct indicators—had the U.S. Continuity of Government program (doomsday planes) been activated? Had the U.S. given advance warning to launch our strategic nuclear forces? They also collected secondary indicators. Their agents inside the U.S. and allied countries watched for heightened activities in and around Washington offices (White House, Pentagon, State Dept, CIA, etc.), including the White House parking lot, places of evacuation and shelter, the level of blood held in blood banks, observation of where nuclear weapons were stored, etc. Some of the indicators were based on a mirror-image of how the Warsaw Pact would prepare for war. Soviet case officers were instructed to look for deviations in the behavior of people in possession of classified information suddenly moving into specially equipped secure accommodations.

While most of the KGB station chiefs and case officers thought Moscow was being paranoid, they dutifully reported what they thought their leaders wanted to hear.

By November 1983, Soviet military and political leaders had convinced themselves that a nuclear first strike from the United States was probable. The RYAN program told them that the odds favored the U.S., and the war indicators in Moscow were flashing red.

That month, NATO ran a highly realistic set of wargames in Europe called Able Archer 83. These included an airlift of 19,000 U.S. soldiers in 170 aircraft under radio silence to Europe, the shifting of commands from Permanent War Headquarters to the Alternate War Headquarters, and practicing nuclear weapons release procedures.

In reaction, the Chief of the Soviet Air Forces ordered all units of the Soviet 4th Air Army on alert which included preparations for immediate use of nuclear weapons. It appears that at least some Soviet forces were preparing to preempt or counterattack a NATO strike launched under cover of Able Archer.

Luckily, no one overreacted. The Able Archer 83 exercise passed.

For years, the U.S had no idea that the Soviet Union had believed the exercise was a cover to launch a nuclear first strike. The Berlin Wall had fallen by the time information from a defector and an end-of-tour letter from the U.S. general responsible for Air Force Intelligence in Europe prompted presidential intelligence board to revisit what the Soviets had thought. In hindsight, RYAN and Able Archer took the Cold War to the brink of Armageddon.

Even when RYAN was reporting that the U.S. had a decisive military advantage, what made the Soviets believe that we would launch a first-nuclear strike? No one knows. However, given Nazi Germany’s surprise attack on the Soviet Union in WWII, resulting in 25 million dead and the extreme devastation inflicted on their country, the Soviet Union had reason to be paranoid. Some have suggested that the Soviets had interpreted President Carter’s 1980 Presidential Directive 59 Nuclear Weapon Employment Policy as preparation for a nuclear first strike. Perhaps the Soviet Union ascribed their own plans for a first strike on the U.S. to their Cold War enemy. Or perhaps the U.S. actually did have a first-strike option in one of our operational plans that the Soviets discovered via espionage.

Why were the Soviets convinced that a war would start with a war game? Several months after Able Archer, the Soviet Minister of Defense publicly acknowledged his country’s inability to tell a big NATO exercise from an actual attack: “It was difficult to catch the difference between working out training questions and actual preparation of large-scale aggression.” It’s quite likely that the Soviets’ own plans for launching a war in Europe would have been as part of a war game.

Certainly the Soviets, believing the signals of the RYAN alert system, were primed to assume a U.S. attack. In attempting to automate military policy and potential actions, the Soviets had amplified their existing paranoia. (A movie called War Games came out that year with some of the same themes.)

A Cautionary Tale for Automating Policy and Prediction
Forty years ago RYAN attempted to automate military policy and potential actions. But in the end, RYAN failed in actually predicting U.S. intent. Instead, RYAN reinforced existing fears, and accidently created its own paranoia.

While the intelligence lessons of RYAN and Able Archer have been rehashed for decades, as our own AI initiatives scale no one is asking what lessons RYAN/Able Archer should have taught us about building predictive models and what happens when our adversaries rely on them.

Which leads to the question: What could happen when we start using Artificial Intelligence and Machine Learning to shape policy?

  • What could happen when we start using artificial intelligence and machine learning to shape policy?
  • Will AI/ML actually predict human intent?
  • What happens when the machines start seeing patterns that aren’t there?
  • How do we ensure that unintentional bias doesn’t creep into the model?
  • How much will we depend on an AI that can’t explain how it reached its decision?
  • How do we deconflict and deescalate machine-driven conclusions? Where and when should the humans be in the loop?
  • How do we ensure foreign actors can’t pollute the datasets and sensors used to drive the model and/or steal the model and look for its vulnerabilities?
  • How do we ensure that those with a specific agenda (i.e. Andropov, chairman of the KGB) don’t bias the data?
  • How do we ensure we aren’t using a software program that misleads our own leaders?

The somewhat-comforting news is that others have been thinking about these problems for a while. In 2020, the Defense Department formally adopted five AI ethical principles recommended by the Defense Innovation Board for the development of artificial intelligence capabilities: AI projects need to be Responsible, Equitable, Traceable, Reliable and Governable. The Joint Artificial Intelligence Center appointed a head of ethics policy to translate these principles into practice. Under JAIC’s 2.0 mission, they are no longer the sole developer of AI projects, but instead providing services and common software platforms. Now it’s up to the JAIC ethics front office to ensure that the hundreds of mission areas and contractors across the DoD adhere to these standards.

Here’s hoping they all remember the lessons of RYAN.

Lessons Learned

  • RYAN amplified the paranoia the Soviet leadership already had
  • The assumptions and beliefs of people who create the software shape the outcomes
  • Using data to model an adversary’s potential actions is limited by your ability to model its leaderships intent
  • Your planning and world view are almost guaranteed not to be the same as those of your adversary
  • Having an overwhelming military advantage may force an adversary into a corner. They may act in ways that seem irrational
  • Responsible, Equitable, Traceable, Reliable and Governable are great aspirational goals

Hacking for Allies

During the Cold War U.S. diplomatic and military alliances existed to defend freedom around the world. Today, these alliances are being reshaped to respond to Russian threats to the Baltics and Eastern Europe and to China’s economic, military, and technological influence worldwide.

Hacking for Allies
The U.S. Department of Defense works with our allies to expand their industrial base. We benefit because it helps the North Atlantic Treaty Organization (NATO) standardize on equipment and our allies’ industrial capacity, capability and workforce can complement those of the United States. Allied countries benefit under the Global Capabilities Program which offers allies opportunities to partner on research and development, with the goal to build prototypes and eventually co-produce systems.

The goal of Hacking for Allies, (which will launch a second cohort next week,) is to connect dual-use startups (those that sell to companies and government agencies) in allied nations to the U.S. defense ecosystem.

Startup ecosystems in many of the smaller NATO countries don’t enjoy the long-established expertise or funding opportunities we have in Silicon Valley or other innovation clusters. For example, today it takes 7 to 10 years for a company in Norway to sell into the U.S. defense market. To shorten that time, we wanted to teach them the best practices of Hacking for Defense/Lean Startup/I-Corps (customer discovery, MVPs, pivots, business model canvas, etc.) And give them a roadmap for how to play in the U.S. defense market.

Hacking for Allies – Norway Edition
Norway is a founding member of the NATO and they are NATO’s bulwark against Russian incursion in the strategically critical “High North” region. Norway has experienced Russian simulated air attacks on Norwegian targets and jamming of GPS signals that threaten civilian aviation. Last fall, Russia conducted a cyberattack on the Norwegian parliament.

The Norwegian Defence Research Establishment, Innovation Norway, and H4XLabs (BMNT’s early stage tech accelerator) just ran the first Hacking for Allies cohort of Norwegian companies. The teams were guided using the “player-coach” approach. The program conducted weekly deep dives with each company working through their challenges. It combined this with sourcing outside experts for all cohort topics of interest. These topics included: Raising funds as a European company, what it takes to work with the DoD, customer discovery for adjacent markets, and more.

85 startups applied for this first cohort. They were down-selected to a few promising teams. Some of the teams included:

  • Alva Industries – making 3D printed electric motor stators 20% more efficient. That means more battery life and/or power for unmanned aerial vehicles.

  • Excitus: a medical device to clear blocked airways in the battlefield. Their device replaces existing suction pumps with the equivalent of a handheld vacuum cleaner with a sterile disposable cup.

  • Fieldmade: An additive manufacturing microfactory with a library of certified printable 3D parts, which radically reduces parts inventory.

  • Ubiq Aerospace: started as de-icing for drones but potentially pivoting to sensor data fusion.

The teams launched out of the program talked to tons of people in the U.S. they never would have connected with (“it would have taken us years to make these connections”), made pivots, built new product suites and capabilities around their core services – all of which made them attractive to wider markets — and raised additional funding.

Now a new cohort of the program is getting under way. Innovation offices from NATO countries and other allies who want to teach their dual-use startups how to work with the U.S. government should attend the Hacking for Allies webinar February 23rd at 8 am Pacific, 11 am Eastern.

Register here.

When National Security Falls Between the Cracks

A version of this article – co-authored with Raj Shah and Joe Felter – previously appeared in War On The Rocks.

After hearing from 20+ guest speakers, including two Secretaries of Defense, Generals, Admirals and Policy makers in our Technology, Innovation and Modern War class – the direction of technology and the future of national security came into sharper focus. This series of articles will offer suggestions to transform the DoD to face the challenges ahead.


As it is currently organized, the U.S. government is ill-equipped to deal with the growing number of national security challenges that exist at the intersection of commercial and defense technology. Innovation opportunities are slipping between Washington’s organizational gaps, and America’s enemies are too.

President Joe Biden has already taken several steps that suggest he recognizes the gravity of this problem. He has elevated the science adviser to a Cabinet-level position, appointed a number of talented individuals to high-level cyber security posts, and created a national security adviser for cyber and emerging technology. But more changes are needed. Most importantly, Biden should create a deputy national security adviser with sufficient staff and authority to coordinate innovation and technology policies across the entire government.

Blurred Lines
From artificial intelligence to biotechnology, U.S. national security is inexorably and increasingly intertwined with commercial technology. Unlike in the Cold War, advancements in areas with important national security implications come from private sector research labs and are driven by consumer demand rather than government directives. Yet it remains unclear who in the government now sets policy for — or has final say over — issues that cross the boundaries between academia, defense, commerce, and diplomacy. In addition to the National Security Council, bureaucratic contenders currently include the Commerce Department, Federal Communications Commission, Federal Aviation Administration, Council of Economic Advisers, Treasury Department, the Office of Science and Technology Policy, and the Office of Management and Budget.

In theory it is the National Security Council that should coordinate a “whole of government” response that brings together tools from different agencies to address emerging threats. Indeed, this was the purpose for which the council was created in 1947. However, Cabinet members who are responsible for vertical portfolios still manage the government’s large functional agencies, and there continues to be significant overlap between those who handle commercial, defense, and diplomatic policy. It is this organizational design that creates blurred bureaucratic lines and weakens U.S. national security. 

There are a number of specific areas where these blurred lines led to subpar policies that undermined America’s technological competitiveness and left the country weaker against adversaries like Russia and China. Consider four recent examples: semiconductors, drones, the SolarWinds hack, and SpaceX’s Starship.

In the absence of a coordinated technology and industrial policy, the United States has become dangerously reliant on computer chips produced in a handful of countries for all its defense and commercial needs. Originally, all of America’s computer chips were produced in Silicon Valley. Today, none are made there. The United States is dependent on two companies, Taiwan Semiconductor Manufacturing Co. in Taiwan and Samsung in Korea, for the chips used to build a substantive part of its defense electronics. Around 60 percent of the chips Taiwan Semiconductor Manufacturing Co. makes are for American companies. Even Intel, the supplier of most of the central processing units used in desktop computers and data centers, will be outsourcing manufacturing of its next generation of chips to Taiwan. Taiwan Semiconductor Manufacturing Co. has announced that it will build a chip factory in the United States, but even when complete it will produce less than 3 percent of the company’s capacity in Taiwan.

In the case of drones, the U.S. government also missed an opportunity to maintain the country’s competitive edge. Where there was once a nascent U.S. commercial drone industry, the Chinese company DJI now controls 69 percent of the global market. Not only is DJI at the cutting edge of such crucial drone technologies as motors, speed controllers, radio modules, cameras, and artificial intelligence, but it has also allegedly used its hobby drones to map U.S. military installations. As a result, the former assistant secretary of the Air Force for acquisition, technology, and logistics lamented that the lack of Defense Department support for American drone startups was a critical missed opportunity.

In some cases, the danger is more immediate. The SolarWinds cyber attacks revealed how the failure to secure commercial software can compromise even well-secured government networks. In this case, the Russian government exploited commonly used civilian network management software in order to infiltrate the Treasury, State, and Defense departments. By adding malicious code to the SolarWinds update tool, hackers gained access to the data of 18,000 customers — including many in the U.S. government.

Efforts to harden government agencies and protect sensitive information against infiltration will fail so long as adversaries can circumvent them through commercial companies. Private companies have lobbied Congress against requirements that would mandate expensive investments to secure their systems. Furthermore, financial penalties for large-scale breaches of commercial companies are trivial. SolarWinds received national attention because it was an attack on the government, but cyber attacks on private companies continue unabated.

Finally, the fate of the SpaceX Starship offers an example of how government oversight agencies can stifle innovation when they are unable to keep pace with the speed of contemporary technological development. In temporarily halting test launches of the SpaceX Starship, the Federal Aviation Administration sought a lengthy investigatory period that put unnecessary roadblocks in the way of a company that is transforming access to space. In innovation, failure is part of the process. Test rockets blow up, test airplanes may crash. If you do not push the envelope and discover the limits of your design you are not innovating fast or far enough. It goes without saying that you strive to minimize loss of life and property, but the rules governing innovation programs should recognize a heightened need for speed. The U.S government appreciated this when developing rockets and experimental aircraft in the 1950s and 1960s — better organization could help it apply this understanding again today.

Reorganize to Win
To solve these problems, the White House needs to ensure there is a single organization that has stewardship of all the issues that cross existing lines between national security, commerce, and technology. 

An effective way to do this would be to create a new deputy national security adviser. Armed with sufficient resources and influence, this position would be given real responsibility to help shape the budget, trade policy, and alliance strategy. This adviser would ideally sit on both the National Security Council and National Economic Council, where they could coordinate policies covering a range of technological and scientific issues. These would include the fields of artificial intelligence, machine learning, biotech, hypersonics, and microelectronics, to name just a few. This position would also be responsible for building a civil-military alliance for protecting civilian assets and incentivizing private companies to do work with a national security payoff. The reach of the new deputy national security adviser could also be enhanced by putting appointees in key agencies such as the Department of Defense, the Office of Management and Budget, and the Office of Science and Technology Policy who would be responsible for leading and coordinating innovation policy across the government.

The Biden administration may opt for a different form of reorganization. Several plausible alternatives have been proposed. Regardless of the approach, the important thing is for Washington to recognize and close the organizational gaps its adversaries have exploited.

Regaining America’s Technological Edge: Build a Civil-Military Alliance

A version of this article – co-authored with Raj Shah and Joe Felter – previously appeared in The National Interest.

After hearing from 20+ guest speakers, including two Secretaries of Defense, Generals, Admirals and Policy makers in our Technology, Innovation and Modern War class – the direction of technology and the future of national security came into sharper focus. This series of articles will offer suggestions to transform the DoD to face the challenges ahead.

“We need to couple the $150 billion a year U.S. Venture capitalists (VCs) spend to fund new ventures with the speed and urgency that the DOD now requires.”


We stand at a crossroads of history. The decisions this new administration makes about how to engage, incite, and rally the full force of American capitalism will determine whether we stand in the backwash of China’s exhaust or we continue to lead.

In the twenty-first century our country’s military and economic power will rely on the rapid development and deployment of new technologies—5G, microelectronics, cyber, AI, autonomy, robotics, access to space, drones, biotech, quantum computing, energy storage, and others yet to be invented.

The technologies we employed to prevail in the Cold War and the War on Terror were largely developed by big defense primes and U.S. government labs, but today most of the advances come from commercial vendors—many of them Chinese. For the first time in the history of modern civilization most of the technologies needed for the military are driven by consumer demand and the potential for profit—not government directives.

China is executing a plan to win with new technologies through its strategy of Military-Civil Fusion—they’ve torn down the barriers between Chinese companies and academia and its military. Its purpose is to improve China’s military technology by integrating Chinese industry and academia so it can develop the Peoples Liberation Army into a world-class military equal or superior to the U.S. Meanwhile, its Orwellian National Intelligence Law mandates that citizens and companies must cooperate with state defense and intelligence work. (Yes you, Apple and DJI.) Simultaneously, the Chinese Communist Party is integrating party leadership in both state-owned companies and private businesses (which account for 60 percent of the country’s output).

The takedown of Jack Ma, founder of Alibaba and Ant Group, is a warning shot to every Chinese CEO that regardless of a company’s size, the party and government are back in control.

China’s Military-Civil Fusion and National Intelligence Law stacks the deck against every U.S. and Western company. You’re not competing against an individual Chinese company; you’re competing against the Chinese government.

What it means for the U.S. military is that our national security is now inexorably intertwined with our success in leading in commercial technology. Developing, maintaining and safeguarding our technological edge will be key to prevailing in great power competition.

Today, the DOD requirements and acquisition systems are driven by a sixty-year-old model predicated on 1) predicting the future (both threats and technology), 2) delivering solutions decades out optimized for lifecycle costs and 3) assuming that government labs and incumbent prime contractors drive technological innovation. Every one of those assumptions are no longer true.

Maintaining our technology leadership will require the DOD to understand that delivering new/disruptive innovation versus execution of existing technologies happen at different speeds, with different people, organizations, culture and incentives. Buying the next incremental version of a gun, tank, plane, ship, means you can (hope to) predict its lifecycle cost, delivery schedule, etc. And you can assume that with enough dollars it will appear.  That’s not how innovation actually happens. And we are living in that new world where innovation is happening continually. We don’t need to abandon our methods of buying incremental improvements, but we must have a parallel set of activities—at scale—to keep up with our adversaries who already understand this.

How can we effectively compete? One of the reasons the United States prevailed in the Cold War with the Soviet Union was that total government control of innovation is an inherent weakness. When the Soviets launched ideological purges of their best and brightest and jailed their dissidents, they throttled innovation at scale. There were no Soviet Steve Jobs or Bill Gates. Meanwhile in the U.S., venture capital was funding startups led by iconoclasts, troublemakers and dissidents. The result has been a half-century of continuous and extremely profitable disruptive innovation.

The U.S. military must now invest in new capabilities and concepts to reimagine how it fights. It needs to use rapid/agile innovation, disposable systems that take advantage of innovations in the commercial technology base. We need to couple the $150 billion a year U.S. Venture capitalists (VCs) spend to fund new ventures with the speed and urgency that the DOD now requires. We need to leverage the inherent advantages of a capitalist democracy and align public and private sector incentives to drive technology advances.

We need a Civil-Military Alliance. One that’s driven by incentives, not coercion. By public-private partnerships, not government control. Private industry, from primes to startups incentivized at scale, will ensure our leadership in science, in industry, and in new technologies.

A few suggestions:

For a Civil-Military Alliance the DOD needs new sources of disruptive innovation and ideas. Not as replacements for the valuable prime contractors we have but as complementary partners who are agile, can attract the best and brightest, and are willing to bet their company on new ideas.

In the 20th century most military contractor-built hardware, and software was an add-on. When an upgrade was required, a new contract was needed, which took years. The new generation of military systems will be built around software with hardware as the add-on. Upgrades can happen over the air in weeks not years.

Today, the world’s innovation ecosystem for new technologies are VCs and startups. To make Civil-Military Alliance work the DOD needs to understand that no investor wants a new startup to spend three years going through the existing DOD requirements and acquisition process when there’s a viable alternative in a commercial market.  (Today, even if a startup gets an early DOD award of a million dollars it’s not enough to incent the technologies they need.)

Few inside the department of defense understand how the commercial venture capital/startup innovation ecosystem works. Going forward, inside every service acquisition arm there needs to be a deputy who can go to a whiteboard and draw how private investors make money. And more importantly work with them to get the best and brightest of their portfolio companies engaged.

One solution is that the DOD needs to pick new entrants as winners at scale. Today, no DOD acquisition chief wants to be in front of Congress explaining to a committee chair why the incumbent vendor in their district lost. That will happen unless both the DOD and Congress clearly articulate that the process for picking winners for existing incremental changes is different from disruptive ones. And that the default ought to be new vendors. Each service should pick 1-2 startup/scale-up winners and buy heavily.

As part of the Civil-Military Alliance, the U.S. needs to massively reinvest in critical technologies. China has invested over $100 billion in moving its manufacturing base from making low-tech products to rapidly developing ten high-tech industries including electric cars, next-generation computing, telecommunications, robotics, artificial intelligence, and advanced chips. They’ve raised over $50 billion just for building an indigenous semiconductor industry.  We need to pick critical industries and do the same at scale. Not only the incumbents but with new entrants.

Another part of a U.S. Civil-Military Alliance would be transforming our existing government R&D Labs. The U.S. spends $15 billion a year on forty-two Federally Funded Research and Development Centers (FFRDCs)—twenty-six R&D labs, ten study and analysis centers, and six system engineering and integration centers. Set up in WWII and scaled during the Cold War, the truth is that most of them are no longer the country’s cutting edge. We need to radically reorient their relationship with the new centers of innovation.

Finally, the existing prime contractors have sat out the innovation game. While many have a token VC arm, today they are not leading in disruptive technology, particularly if the profit incentives are to continue to lobby and build the status quo and get a new contract every time a system needs an upgrade. We need serious incentives to have primes get into the disruptive innovation game. They need to actively invest in and buy startups—at startup multiples—with billions, not millions. Their CFOs need to tell us what incentives (tax write-offs, acquisition preferences, etc.) can be tied to rapid delivery and deployment—not demos—of new technologies and systems.

This is a critical time for our nation. Our adversaries aren’t waiting. The Biden administration can choose business as usual and let U.S. leadership in technology disappear and the country founder in the backwash of the coming China age. Or it can choose to build a Civil-Military Alliance ensuring our leadership in science and in industry, while maintain our core values. Our hopes for peace, security, and self-determination around the world, require us to make this effort.

Pentagon Advisory Boards Need to Offer 10X Ideas, Not 10% Ones – P.S. You’re Fired

A version of this article – co-authored with Raj Shah and Joe Felter – previously appeared in Defense One.

(UpdateAfter this article was written the Secretary of Defense fired every member of all 40+ defense advisory boards and will start anew. Hopefully the suggestions in this post will help inform how they reconstitute the boards.)


Last week the Biden administration delayed seating several Trump appointees to defense advisory boards. It’s a welcome signal that incoming leaders recognize these groups are essential, not just patronage jobs. But the review needs to go much further than that.

One of the many changes the Department of Defense needs to make is to reimagine the role and makeup of its advisory boards and ask them for 10x advice, not 10% advice.

—-

The Defense Department is at a crossroads. Incremental improvements are no longer good enough to keep up with China; the Pentagon needs substantive and sustained changes to its size, structure, policies, processes, practices, technologies, and culture. The last administration asked most of the Pentagon’s 40-plus boards for advice on small improvements — with a few notable exceptions, such as the Innovation Board’s Software Study and the work of the National Security Commission for AI — the latter an independent effort chartered by Congress.

This is no longer sufficient. The DoD needs to ask for big ideas, boards who can deliver transformative advice, and it needs to reshape its boards to provide them.

What’s an Advisory Board?
DOD’s Advisory Boards are comprised of individuals outside of the organization who can provide independent perspectives and advice. An advisory board has no official role in managing – they can’t hire, fire, or order people to do things.  All they can do is offer advice.  But with the right membership and senior support, they can have tremendous impact. In the past decades, advisory boards have challenged conventional thinking and nudged leaders towards major policy changes.

Most of the DOD advisory boards are in the services or agencies. For example the Army and the Air Force each have their own Science Board, the military academies each have an advisory board they call the “Board of Visitors.” The office of the Secretary of Defense has 7 advisory boards: Policy, Innovation, Science, Business, Military Personnel TestingWomen in the Services, and Sexual Assault. (Steve had the pleasure of serving on one – albeit for a short time.)

Different Advisory Boards for Different Times
In times where the status quo is sufficient – when your company or country is the leader –  advisory boards are asked for advice about improvement – how to improve your existing systems. You appoint advisors who have detailed knowledge of existing systems and have long term institutional knowledge and connections. And you generally discourage Ideas that might disrupt the status quo.

However, these are not normal times. Incremental improvements no longer assure that our country can compete. For example, rapid innovation in new technologies – cyber, AI, autonomy, access to space, drones, biotech, etc. – is no longer being led by military/government labs, but instead comes from commercial vendors – many of them Chinese. The result is that unlike the last 75 years, the DOD can no longer predict or control future technologies and threats.

So it’s time for DoD leaders and staff to hand off requests for advice about incremental improvements to consulting firms and refocus their advisory boards on critical competitive issues.

The first order of business is overhauling the boards’ membership to support this turn toward rapid innovation. In the past, the DOD has had some extraordinarily effective advisory boards. During the Cold War examples included the Jasons, the Gaither Committee, the Land Panel, and numerous others. More recently the Defense Innovation Board had admirably carried that torch. Unfortunately several advisory boards have become moribund resting grounds for political apparatchiks.Today’s challenges demand the DOD’s advisory boards appoint the best and brightest regardless of party.

We believe the new administration can quickly refocus their boards in three steps: 1) reset the membership of the current DOD Advisory Boards to support rapid innovation 2) Think strategically about the future, and 3) Set high expectations for engagement and implementation.

Reset board membership and structure to support rapid innovation and transformation

  • 1/3 DOD insiders who know the processes and politics and help ensure non-standard solutions actually get implemented. They can steer the board away from dead-ends or incremental solutions.
  • 1/6 crazy DOD insiders – the rebels at work. They are the Uniformed and civilian leaders with great ideas that have been trying to be heard. Poll senior and mid-level managers and have them nominate their most innovative/creative rebels
  • 1/3 crazy outsiders. Innovators and technologists with new, unique insights in the last two years, who are in sync with the crazy insiders to build 10x solutions
  • 1/6 outsiders who represent “brand-name wisdom”. They provide top cover and historical context. Connectivity to large institutions required for implementation at scale

Once the new members are in place, DoD should ask for big and bold ideas in several key areas, including:

Think strategically about the future

  • Technology and innovation: Given finite budgets, how best to evaluate, choose, and scale a plethora of new technologies and new operational concepts?
  • Business practices: Examine and explore entirely new ways of building commercial partnerships and influencing the private sector.
  • Policy: Ensure we understand our adversaries and how they are fusing together military, economic, and private markets to challenge us. What issues require educating Congress and DOD leadership?
  • Human capital: How should we reshape the DoD’s personnel architecture to attract more technologists and fit into today’s more sclerotic career paths?

Finally, DoD leaders should ask for more than ideas; they should engage and lead the boards. They should set high expectations for engagement and implementation, and work up and down the chain to ensure recommendations are achievable. Do we need new authorities, laws, organizations? Do we need to reprogram existing budgets? Acquire new ones?The boards should report to the principals of their sponsor organizations, who should regularly review whether the boards have delivered real value to the mission.

Americans are ready to answer the call to service to help the DoD and the nation reform and strengthen.  The Biden Administration and DoD leadership have the rare opportunity to completely rethink and reset its Advisory Boards.  Successfully taking on this challenge will not only repair strained ties between the public and private sectors but is essential to the future defense of our nation.

Lessons Learned

  • Flush all the political appointees from the advisory boards. (Update: Done- fired everyone not just the new appointees..)
  • Replace them with people with the experience and expertise needed to help the U.S. keep its competitive edge
  • DOD leadership needs to ask and act for transformational, contrarian and disruptive advice
    • And ensure they have the will and organizations to act on it
  • Move requests for advice for incremental improvements to the consulting firms that currently serve the DOD

Reminder

Lessons for the New Administration – Technology, Innovation, and Modern War

Our recent national security class at Stanford, Technology, Innovation, and Modern War was designed to give students insights on how the onslaught of new technologies like AI, machine learning, autonomy, cyber, access to space, biotech, hypersonics, and others have the potential to radically change how countries fight and deter threats.

With 20+ guest speakers, including two Secretaries of Defense, Generals, Admirals and Policy makers, the class emphasized that winning future conflicts requires more than just adopting new technology and developing new weapon systems. It calls for a revolution in thinking about how these technologies can be adopted and integrated into weapons and other defense platforms, and more importantly, how they can create new operational and organizational concepts that will change the way we fight.

By the end of the class there were five surprises.

  1. One was a continuous refrain from senior DoD leadership that new tech, weapons, and operational concepts are insufficient to guarantee the U.S. will prevail in a great power conflict. In fact, these new technologies/weapons change the odds against us.
  2. Secondly, our senior military leadership recognizes that now more than ever we can’t go it alone. We need allies – existing and new ones. And that depends on a reinvigorated State Department and renewed emphasis on diplomacy in general.

Unstated by any of our speakers but painfully clear by class end were three other surprises:

  1. Our national security is now inexorably intertwined with commercial technology and is hindered by our lack of an integrated strategy at the highest level.
  2. Our adversaries have exploited the boundaries and borders between our defense, commercial and economic interests.
  3. Our current approaches – both in the past and current administration – to innovation across the government are piecemeal, incremental, increasingly less relevant and insufficient.

Lessons Learned
A few takeaways from our speakers. If you’re in the DoD and conversant with the National Defense and Military strategies and have read Chris Brose’s book The Kill Chain, none of this will come as a surprise. But for the rest of you, here they are:

  • The 2018 National Defense Strategy focused the DoD on Great Power competition. It called out China as a peer competitor to America, pursuing its goal of global dominance. At the same time, Russia has reemerged as a regional power.
  • For the last two decades, while we were focused on combating terrorism, China has explicitly developed weapons and operational concepts to target every one of our advantages- in weapon systems and operational concepts, but also in alliances, economic and diplomatic power.
  • Unfortunately, China has succeeded – many of our most exquisite systems on sea, in space or in other places are at risk. A majority of these weapons have now become legacy systems eating up future budget and resources.
  • Rapid innovation in new technologies – cyber, AI, autonomy, access to space, drones, 5G, biotech, quantum, microelectronics, etc. – are no longer being led by military/government labs, but instead come from commercial vendors – many of them Chinese. The result is that unlike the last 75 years, the DOD can no longer predict or control future technologies and threats.
  • A surprise for many of us was the tacit acknowledgment from our military and defense leaders that we cannot win a war alone, without allies. These senior leaders emphasized the importance of a more collaborative embrace of existing allies and creation of new ones. They put a premium on diplomacy, and the need for a better funded and robust State Department.
  • The result is that for the first time in almost a century, the U.S. is no longer guaranteed to win the next war.

The good news is every one of our military and civilian speakers conceptually understands all of this. And even better, all want to change the status quo. However …

Most are coming to the conclusion that the DoD is at a crossroads: Substantive and sustained changes in the DoD size, structure, policies, processes, practices, technologies, and culture are needed.

  • For example, our requirements and acquisition systems are driven by a 70-year-old model predicated on predicting the future (both threats and technology) and delivering solutions decades out; and optimized for lifecycle costs, not rapid innovation or disposable systems.
  • In the last four years we modernized the acquisition process, but it remains hindered by the requirements processes from the services, which still result in 88 Major Defense Acquisition Programs – where we spend our acquisition dollars – to buy legacy systems mostly built for past threats.
  • Some hints of the future force came from multiple speakers. Admiral Lorin Selby, the Chief of Naval Research, for example, had a compelling vision of the future fleet and an expanded industrial base.
  • The DoD has over 75 incubators and accelerators. We lead the world in demos of new technology but not in deployed systems. Few of these innovation activities have resulted in a major program of record. The DoD is making the right baby steps but needs to quickly focus on scaling innovation. This, of course, will require the difficult conversation of what legacy systems will be retired.
  • DoD’s relationship with startups and commercial companies driving these new technologies is hindered by a lack of understanding of their own and their investors’ interests. Venture capital and startups have institutionalized disruptive innovation. In the U.S. they spend $150 billion a year to fund new ventures that can move with the speed and urgency that the DoD now requires. While we’ve made progress, a radical reinvention of our civil/military innovation relationship is necessary if we want to keep abreast of our adversaries. This should include:
    • A Civil-Military Alliance driven by incentives not coercion. By public-private partnerships not government control. Private industry – from Primes to startups – incentivized at scale will ensure our leadership in science, in industry and in new technologies.
    • Reduce the dependence on bespoke and exquisite systems. Rapidly bring commercial technology into the DoD while adding proprietary defense components
    • Create new technology ecosystems around DoD technology needs by encouraging commercial interoperability around DoD standards. Awards and contracts to each new ecosystem.
    • Encourage and incentivize dual-use startups, scale-ups, and companies
    • Overhaul Federal Labs and Federally Funded Research and Development Centers (FFRDCs) to promote collaboration at scale with startups and venture investors
    • Each service should pick 1-2 startup/scale-up winners and buy heavily
  • Pentagon leadership will need to be selected on the ability to innovate – empower the innovation insurgents and elevate risk takers that understand technology.
  • We’ve failed to engage the rest of the populace in our mission. Americans – including extraordinarily talented students from our top universities — are ready and willing to serve in some capacity. We’ve shown little interest in providing the incentives and expanding the opportunities required to make that happen.

However, these observations about changes needed in the DoD surfaced a much bigger problem, one that civilian leadership has not yet acknowledged: National security is now inexorably intertwined with commercial technology and is hindered by our lack of a national industrial and economic policy. There is an urgent need for an integrated strategy and policies.

These are not problems of technology. They’re problems of organizational design, incentives, out of the box thinking and national will.

The American people will need to demand more of their government and elected officials. The status quo will need to be broken. Substantive change will require new ideas, not better versions of the ones we have. For example:

  • The new Biden senior White House organizational structure still treats technology as a standalone issue. That’s a status quo position and a losing hand. We need to recognize that the boundaries between our defense, commercial and economic interests are interrelated.
  • We need to build the innovation capacity across the interagency- coordinated and synchronized by senior executive branch leadership. One way of implementing this would be creating a political appointee in key government agencies that acts as the interagency single point of innovation leadership cutting across organizations including but not limited to the DoD, National Security Council, Council of Economic Advisors, OMB, FCC, and OSTP.
  • Create a new Deputy National Security Advisor to coordinate and synchronize innovation and industrial policies across these multiple agencies
    • With real influence and responsibility on budget, trade policy, and alliance strategy
    • Specifically coordinate national policies of 5G, AI, machine learning, autonomy, cyber, access to space, biotech, hypersonics, microelectronics, etc.
    • Owns Civil/Military alliance for engaging and incentivizing new entrants and incumbents and protecting civil assets
    • Sits on the National Security Council and National Economic Council

These changes will require Congress, defense contractors and the executive branch to pull in the same direction to change that equation.

The good news is that we have all the tools needed to succeed, we just need the willpower.  And we must not forget what’s at stake. Democracies, while messy, are a force for good.  Self-determination with codified freedoms is the most moral system of organization mankind has developed.  Getting the reforms we examined in this class is essential to the preservation of democracy and maximization of peace.  It is most certainly a noble endeavor.

In future articles we’re going to offer specific solutions to transform the DoD to face the challenges ahead, not behind.

Steve, Joe, and Raj

Technology, Innovation, and Modern War  – Wrap Up

This class, Technology, Innovation, and Modern War was designed to give our students insights on how the onslaught of new technologies like AI, machine learning, autonomy, cyber, access to space, biotech, hypersonics, and others has the potential to radically change how countries fight and deter threats.

Our 20+ guest speakers were an extraordinary collection of military and policy leaders including two Secretaries of Defense, Generals, Admirals and Policy makers.

The class emphasized that winning future conflicts requires more than just adopting new technology and developing new weapon systems. It calls for a revolution in thinking about how these technologies can be adopted and integrated into weapons and other defense platforms, and more importantly, how they can create new operational and organizational concepts that will change the way we fight.

By the time we got to the end of the class we had a firehose of perspectives on technology, weapons, and policy. It took us awhile to process it all, but out of that mass of data five surprises emerged – insights about what’s happened to the DOD and the country and how we should organize to meet these challenges. We’ve summarized them in part 2 that follows this post. But first here’s a summary of what we covered in this class.

An overview of the history of military innovation
In the first part of this course, we reminded the students that the national power of a country – its influence and footprint on the world stage – is more than just its military strength. It’s the combination of a country’s diplomacy (soft power and alliances), information/intelligence, and its innovation / economic strength as well as military prowess.

Nations decline when they lose allies, decline in economic power, lose interest in global affairs, experience internal/civil conflicts, or the  nation’s military misses disruptive technology transitions and new operational concepts.

In our opening lesson, Ex-Secretary of Defense Ash Carter shared insights and experiences from his extensive and impactful career in DoD that included tours as Undersecretary for Acquisitions, Technology and Logistics, Deputy Secretary, and Defense Secretary. Subsequent teaching team lectures provided students an overview of the history of military innovation, all the way from long bows to nuclear weapons and offset strategies with the observation that innovations and adoption in military systems follow a repeatable pattern. Max Boot, the author of “War Made New helped us understand that pattern. Next, we described the US strategies developed since World War Two to gain and maintain our technological and competitive edge during the Cold War with the Soviet Union.

Finally, we discussed the challenges raised in the 2018 National Defense Strategy. In addition to  the non-nation states (Al-Qaeda and ISIS) we’ve fought for the last two decades, our military now faces “Great Power” competition from China and Russia. Today the U.S. faces “two plus three” threats –  the two peer adversaries China and Russia, plus the three – regional threats from Iran, North Korea as well as the non-nation state actors. The strategy called for a pivot of our defense from fighting terrorists to preparing for confrontations with the “two plus three.”

To help us understand how and why the “two plus three” strategy was created, we had Bridge Colby, former Deputy Assistant Secretary of Defense for Strategy and one of the strategy’s authors lead this discussion.

Next, Christian Brose, the former staff director of the Senate Armed Services Committee and the author of The Kill Chain, (one of our required readings for the class) helped us understand what our adversaries have done to put our military and country at risk over the last two decades and the consequences for the country.

Military Applications and Operational Concepts in Space, Cyber, AI, and Autonomy
Once we laid out the new 2+3 threats to the nation, we segued into the second part of the class where we examined how emerging technologies in AI, cyber, space, and autonomy would create new weapons systems and operational concepts.

We heard from the DOD officials who are acquiring these technologies including Ellen Lord, the Under Secretary of Defense for Acquisition and Sustainment, and Will Roper, the Assistant Secretary of the Air Force for Acquisition Technology and Logistics.

We got a deep understanding of the impact and deployment of AI in the DOD from recently retired Lieutenant General Jack Shanahan, the founding director of the Joint Artificial Intelligence Center (the JAIC) and Nand Mulchandani the JAIC CTO. Chris Lynch, the ex-head of the Defense Digital Service and now CEO of Rebellion Defense described his company’s experience as a new defense contractor trying to build and deliver these AI-systems for the department defense.

For autonomy, Maynard Holiday, former senior advisor in the Pentagon who helped the Defense Science Board define autonomy, gave us a tutorial on the technology. And for cyber, we had Sumit Agarwal, a former DOD cyber policymaker, do the same.  For understanding space as a new contested domain, and the role of the new Space Force, we had General John Raymond, Chief of Space Operations and Commander of the Space Force. And for the impact new technologies will have on the Navy, the best person to hear from was Admiral Lorin Selby, the Chief of Naval Research, which includes ONR, and the Naval Research Lab.

Major General Mike Fenzel, Vice Director for Strategy, Plans and Policy, J5 for the Joint Chiefs, educated us on how the DOD develops operational plans and courses of action. And finally, Michele Flournoy former Undersecretary of Defense for Policy offered what a new Secretary of Defense might do to organize to match those 2+3 threats and new technologies.

Midterms and Finals
For their midterm we asked students to describe how they would reallocate the defense budget to better serve US national security interests and to make their case to Congress on why and how our defense priorities should change. They had to determine and argue how much budget should shift from legacy systems to new systems and why.  We selected the author of one of the top student submissions to present her argument and recommendations to Congressman Mike Gallagher of the House Armed Services Committee.

For their Final, students teamed up in groups of 4 to tackle thorny challenges that may face the US and its Allies in the coming decades, including misinformation, cyber, logistics, networks, and new military platforms.  The students, with the help of a military member, developed broad proposals and wrote a policy paper for the President of the United States.  In the next to last class, as prep for the students final presentations, Safi Bahcall observed that one of the most significant barriers to innovation and adoption is how organizations are designed. And he offered that the DOD needs a different organization to facilitate rapid adoption

Finally, in our last class one of our student teams presented their final project – how they would address real threats, with new operational concepts, policies, and strategies – to former Secretary of Defense General (ret) James Mattis.

To Our Students
This class has given us a lot of hope that our nation and free nations around the world will be in good hands if the students in this class-and the best and brightest of their generation beyond Stanford – make the decision to serve and to use their amazing skills for the betterment of the world. We hope you take on the challenge that General Mattis posed to “Be the change in the world that you want.” All of us are cheerleaders to all of you and in that journey. So thank you for letting us be part of this.  We are excited to see how much positive change you will make happen in the coming years.

Lessons Learned

In our next post we’re going to describe the five surprises, the insights we’ve derived and offer specific solutions to transform the DOD and country to face the challenges ahead, not behind.

Steve, Joe, and Raj

A Quick Course on Lean

Over the weekend I got asked the best way to teach students the principles of Lean via Zoom.

One of the key lessons from our Educators Conference is that when teaching online complex information needs to be delivered to students in small, easily processed parts.

I realized that pre-pandemic I had put together a series of two-minute videos called “See Why.”  They’re not only helpful for a formal class but for anyone who wants to review the basics. Here’s what I suggested they offer their students:

Lean in Context

No Business Plan Survives First Contact With Customers

How did we build startups in the past?

The Business Model

An introduction to The Business Model Canvas

The Minimal Viable Product

How to Get, Keep and Grow Customers?

How to Get Out of the Building and Test the Business Model

What is Customer Development

What is Customer Discovery and Why Do it?

Why Get Out of the Building?

A short article on how to do Customer Discovery via Zoom

Jobs to be done

Customer Validation

The Pivot

The Harvard Business Review Article “Why the Lean Startup Changes Everything” ties the pieces together here

The Mission Model Canvas

What is the Mission Model Canvas

The Mission Model Canvas Videos

Extra’s

Why Customer Development is done by founders

What Do Customers Get from You?

What are Customer Problems/Pains?

Users, Payers and Multi-sided markets

How do I Know I Have the Right Customers – Testing

How big is it?

How to Avoid Pricing Mistakes

More two-minute lectures here

Tools for educators here

Tools for students here

Lessons Learned

  • Break up online lessons into small parts

 

 

 

What I Learned from 500 Educators – Build Back Better Summit – Results

With the theme “Build Back Better” Jerry Engel, Pete Newell, Steve Weinstein and I co-hosted nearly 500 Lean Educators from 63 countries and 235 universities online for a three-hour session to share what we’ve learned about educators on how we can help our communities rebound, adjust, and recover.

We got insights from each other about tools, tips, techniques and best practices.

Here’s what we learned.

Background
When we last ran this virtual summit in July, our 400 educators were just coming to grips with teaching remotely. The two questions on the table were, 1) Could the lean methodology work remotely? 2) And what kind of pedagogy would support a class that depended on “getting out of the building” to work virtually? Tactically, how effective would customer discovery be for the students? Would customers sit for virtual interviews? How would you show them minimal viable products if not in person? How do you keep students engaged?

This Summit
This summit discussed how the pandemic has shifted the way we teach, but also what we learned teaching and how we can use the Lean methodology to make an impact on our communities.

COVID-19 has dramatically altered the business landscape. Main Street businesses are severely affected. While many parts of the high-tech sector are growing, others are either contracting or shutting down. Amid these uncertain times we believe that Lean educators can prepare students for this new investing climate and help communities recover.

The summit opened with a panel of Investors sharing their insights of what the funding environment for entrepreneurs, non-profits and small businesses will look like as the economy recovers. See here for a video of the investor panel.

Next, Lee Bollinger President of Columbia University in conversation electrified the audience with description of the fourth purpose of a university. (I’ve summarized our conversation with the video and transcript of the entire talk following the summary.)

The core of the summit was gathering the collective wisdom and experience of the 500 attendees as we split into 20 breakout rooms. Besides sharing tips for teaching traditional entrepreneurs the discussion also included how we could help Main Street businesses. The output of the breakout sessions provided a firehose of data, a ton of useful suggestions, teaching tips and tools. Following Lee’s talk I’ve summarized the collective notes from the breakout session.

Lee Bollinger – Columbia
The three purposes of a university are research, education, and public service. But universities should take on an additional role. To try to impact and affect the world in good ways, it’s what I call the fourth purpose of a university. No university has said, “We should design the institution to have a bigger impact using academic work and join up with outside entities and organizations and partners to do that.” And that’s what the fourth purpose is all about.

If one looks around in the world, you see these huge problems, massive inequality, hunger, poverty, climate change, issues of how to set up a global trading system. You have national problems. So, there’s no shortage of major issues.

NGOs play a very important role, but they tend to be very focused on some particular issue. If you look at think tanks, again, many of them are captured by particular interests. And universities have this incredible sort of filled-with-public purpose people who want to have an effect on the world.

One of the things that’s been striking to me over the course of my career is that those people probably will not get credit for that work in the promotion and tenure process. And that strikes me as crazy. We should embrace in the appointment process people who have incredible talents of that kind. People who are extremely gifted and talented at making things happen in the world. I think all of us have known people like that.

[If we do this] we would have a cohort of people within our institution who are of equal standing, with the greatest scholars and the greatest teachers. But they are the greatest at having impact on the world. We do this to some extent. That’s why it’s interesting in a way. It’s not even like it’s completely novel. I mean, the great surgeon, the great lawyer will be an adjunct in the law school, or the great business person will be an adjunct in the business school. But we don’t embrace it in the way that I’m thinking about. So there’s who do you embrace within the university and what do you value?

I think it’s a very pragmatic and practical –  where do you situate in your mind universities in the context of the world? Should they be highly removed and only focused on teaching and scholarship with some public service on the side? Or should they be actively engaged with problems, ready to work with outside people and organizations?

See Columbia World Projects.

If you can’t see the video of Lee Bollinger’s talk click here. The transcript of his talk is here.

Breakout Sessions
The core of the summit was gathering the collective wisdom and experience of the 500 attendees as we split into 20 breakout rooms.(There were also a special breakout room for those interested in the new Hacking for Environment/Oceans course that started at UCSC and UCSD this year.)

The output of the breakout sessions provided a firehose of data, a ton of useful suggestions, teaching tips and tools. I’ve summarized the collective notes from the breakout session. 

The consensus in our July summit and reinforced again in this summit was, yes you can teach via Zoom and “get out of the building” when you physically can’t. And it’s almost good enough. Further, our 3-hour long classes which were challenging in person required a redesign to be taught online. Zoom fatigue was real.

General Observations

  • Crisis accelerates certain trends. COVID broke the myth that distant learning was problematic and isn’t as effective.
  • It forced everybody into remote learning, and a lot of people came away with the feeling of, hey, for a lot of things, this works much better than we thought it was going to work.
  • Now everybody has lived through a pivot. Everybody has experienced disruption and perhaps is now more open to looking at new ideas.
  • That’s going to be a carryover into when we can go back. How do you use that as a technique and not be afraid of it?
  • We need to remember in these difficult times that many of the skills we’re teaching – problem solving and running around the brick wall or through it – are life skills that we’re teaching. They’re not restricted just to entrepreneurs.
  • Not pretending it’s business as usual was a great lesson

Pedagogy – How We Teach Remotely

  • On-line has made it easier for teams to meet, mentors to meet, easier access to world-class speakers.
  • The importance of actually doing good instructional design, was pointed out as is time consuming and significant work to do up front. But it pays off in enabling much better engagement and retention.
  • it’s forced educators to become much more coherent and clearer about what they want to achieve with their teams and their students.
  • Understand that that it takes longer for people to absorb information when delivered online.
  • The flipped classroom approach – lectures as prerecorded homework – reduces remote class load. It can make your synchronous time more focused on collaboration, both with you and the students as an instructor, but also among the students themselves.
  • Make each lecture available in advance of the class.
  • Reinforce the lectures with examples during the zoom session
  • We’re doing better online than ever. In classes it’s easier to get people to participate but it’s difficult to keep momentum, especially when you get into the hard part of customer discovery
  • Overall, there’s a higher pressure to be more entertaining
  • Some institutions have asked the students to design the class. They choose a topic, then the students design the class or help design the class
  • It’s really difficult to maintain that one-on-one intimacy, but zoom has been a passable kind of safe option.\
  • We’ve had faculty say that hybrid classes – teaching both in person and virtually, simultaneously – are probably the most detrimental learning environment
  • Hybrid teaching – some students physically in classroom wearing along with others online was pretty detrimental to the quality of instruction
  • One way of ensuring that students go through the advanced materials, is to have the students come up with a question about the material in advance
  • When it all changes, we’ll go back in person. But zoom is simply a classroom which just happens to be electronic. And the breakout rooms are simply a breakout, a study session, it just happens to be electronic. I think you can build an argument that there are more innovative and more interesting ways to do this. I just don’t know what they are yet.

In-Class Timing

  • Shorten the time that you’re going to do things. You can no longer do a full day, you can perhaps do three hours max virtually
  • Break things into very, very small chunks, bite sized chunks, one-minute, not 20-minute presentations for teams. And micro videos so people can watch to learn things
  • You got to break everything up – 10/15 minutes, it can’t be anything longer than that
  • Keep everything very, very short
  • Make things very bite sized, even when you’re all together online
  • The chunking concepts worked really well for students, and array those chunks of information in a buffet rather than a monolith – to make it easier for students to access
  • Make certain you boil your class down and reiterate, “these are the five things you needed to take away from this discussion.” Because at the end of the day, there are a little bit overwhelmed
  • Students being overwhelmed was a running narrative

Reach of the Classroom

  • Educators can reach a much larger audience, even a worldwide audience. And that really opened up the educators minds that they can teach not just to a single group, but to a much larger group
  • Having a worldwide audience is now possible. Which is a huge strength and has network benefits that people couldn’t anticipate
  • Teaching remote enabled being able to increase access. There were some great examples of enabling students in Africa to participate in programs from Australia, which had never happened before
  • It made me think of us as teachers without borders. That access is really pushed out to everybody now, to a much, much increased attendance
  • We can bring more people in from outside the classroom. Not only the theorists but constituents for customer development, or with Main Street businesses and local constituents
  • We’re no longer restricted by the size of the classroom. This year we’ve gone from 8 teams and 32 students to 16 teams and 80 students

Guest Speakers

  • A year ago students would have looked down on not having in-person guests. Today they’re blown away by who we can get
  • Remote teaching offers a broader access to more guest lecturers. It’s a lot easier for guests to say yes in because they don’t have to drive in, they can do it from their offices
  • Pre-recording some guests enables access to guests who normally would say no because of their schedule

Customer Discovery

  • Getting out of the classroom in some ways can be a lot easier when you’re never actually in a classroom. There aren’t the same travel and logistical challenges.
  • Getting zoom interviews is actually easier. So some of the discovery process has been easier online
  • Mixed results, we were able to get more people engaged to get more people do more interviews, and because people are more available online. But we couldn’t go as deep and couldn’t do more of the informal observation, that part of really getting to some insight
  • We need to know what sweet spots for customer development work best with zoom and that don’t work best with zoom. We need to give our students greater guidance around that point.

Minimal Viable Products

  • The very important role the MVP plays today, especially when you’re working in zoom. If you can get the product quickly, cheaply, and without using a lot of funding, you need to do that, because that’s going to get you a lot further along in terms of what you can learn from a customer development perspective.

Breakout Sessions

  • Organize to have more class time in the breakout rooms in smaller groups, because this is where engagement really happens
  • As soon as you jump into a breakout session as a professor, you’re going to kill the discussion. Be sensitive, don’t jump in, let them finish the discussion on their own
  • Breakout sessions held via zoom help maintain team chemistry
  • Keeping the same team composition in the breakout sessions make those sessions work really well, compared to when they had split teams

Student Engagement

  • The wallflowers within the class get to use chat, versus in person where they’re not going to participate at all.
  • How do you create energy during zoom sessions, especially during international calls?
  • There’s a drop off in engagement after one hour. Basically, they just disappear from the zoom
  • Some of the good things was being able to institute virtual pitching, virtual customer discovery, and in some cases a hosted special session to motivate faculty and students
  • Encourage students to learn the skill to consult with each other. This is a crucial skill. To be able to guide each other and say, “Well what did you learn about your customer discovery. And what did you learn about the value proposition.” Have them take the role of the educator a little bit

Collaboration

  • We need to find ways to allow students/teams do distance socialization. Find those kinds of activities in a way that gels the team and make that work out
  • Socializing happens naturally in person. You go out to dinner after things, you go get pizza, you hang out. That’s much harder to do virtually
  • Finding collaboration tools which can be used both during the zoom sessions but also outside of class. So students and the overall class can interact, both during the official hours, as well as during the unofficial hours
  • We’re no longer having these bigger networked conversations where you can have the serendipitous meet at the watercooler or the trade show, and kind of increase the creativity. But because of that some of these interactions have become more meaningful and purposeful because they’re very focused
  • Providing those tools is really important because they can’t just go have a cup of coffee after class, they can’t all get together at seven o’clock
  • Eventually bonding does happen if the teams meet on a regular basis and really connect over time
  • Community building is very challenging in remote context. Even though you’re able to get across a lot of the learning objectives, you’re missing a lot of these intangibles
  • Using tools like Mural, Discord, Slack and ClassEdu creates a sense of community and ongoing collaboration
  • Get the input from the students about what collaboration tools they’re most fluent in 

Team Formation

  • Teams have more trouble forming and norming under the current circumstances
  • There’s the forming, storming, norming, performing kind of thing about teams that happens through working together over time, and socializing
  • Team formations to really gel as a team can happen in this kind of remote environment – but it takes longer

Students/Teams

  • Continually push more for diversity in students/founders; older people, Hispanics, women, brown and black – people of all of all flavors
  • Having someone who looks like them lead the class info/recruiting sessions for diverse students. This dramatically changes the class makeup
  • Be sensitive to students’ personal situations
  • Students will turn off their videos, not because they’re checking out, but because of their location (bedroom, basement, sitting in their underwear, etc.)
  • Suggest a class rule that participation is part of the grade. When they do talk, they have to put the camera on. That’s a compromise on the sensitivity
  • In the online environment, it is a little bit more difficult to gauge feedback from teams
  • You need to work hard helping build highly engaged and motivated teams
  • You want to push them to take advantage of being virtual and conducting extreme customer discovery
  • On the other side, teams who might have started out strong at the beginning of COVID, and found it easier to do things virtually, have now hit a serious virtual fatigue, and are kind of disengaged and not excited about it. And just really exhausted, too exhausted to take anything more on

Mentorship

  • Mentorship becomes a lot easier. Rather than having to get people face to face meeting, we are now able to connect people. we are being able to bring the right mentors from around the country or even around the world to help our students with mentoring
  • There are three kinds of mentors; process mentors – those that know what’s coming up. Technology mentors, and then market mentors. Zoom makes it easier to have more people involved
  • Reach out to older/retired entrepreneurs, find them and put them into the mix as mentors, sometimes as founders and coaches and so forth. They’ve got time they’re willing to help
  • Keeping mentors and investors engaged over the video was a bit of a problem. They managed to shorten and simplify the process that tended to help. But q&a engagement is still a little bit of a struggle.

Exams

  • Exams need to be testing more of the understanding in the application of the concept
  • You can have an exam that is open for six or 24 hours. And then you’re able to actually ask the students to demonstrate more of the understanding of the concepts

Post-Covid Teaching

  • How do we make sure that our students who may be falling behind and may not have been able to keep up because of the COVID pandemic?
  • How do we make sure that they’re on track after we get back?
  • How do we make sure that we are adjusting for their return and the return to normalcy after we get back?

Main Street

  • We as educators need to not treat solopreneurs or Main Street businesses as second-class citizens in our classrooms or incubators, or our meetups. They’re embracing the risks and challenges that big tech startups are embracing
  • Main Street customers had product market fit, and now they’re experiencing for the first time falling out of product market fit
  • Business owners are distracted, focused on day-to-day issues. And they’re impacted personally
  • Looking at all aspects of the entrepreneur has been a real focus on prioritizing the human element, when folks are dealing with layoffs, or cash flow issues, or potential eviction
  • How do we work with companies/startups that are maybe not so much innovation driven, but necessity driven? Because of the dislocations being created by COVID-19, and economic dislocation
  • How do we provide services at scale to help coaching? We had some people who had sent their students to help those local businesses in this time of need and pivoted their classes from doing the next step to helping mainstream businesses do it
  • And we had people doing that, both in Africa and in San Jose. And with Hacking for the community in Hawaii and going out to rural areas. But we still struggle with how to engage, especially with rural communities to help them do that
  • When you go out to rural areas, that younger people who are already fluent in the tools are more are more likely to engage
  • Similarly, the idea that empathy and engagement is extremely scalable. So some of the core principles here have really scaled a lot
  • One of the things that was really interesting was connecting entrepreneurial students with waitresses and bartenders to help them figure out how to get additional funding to compensate for the lack of subsidies they might not have been able to receive
  • It’s not always a sexy company that the student gets to work with. But they get to see real impact. And it’s something that they can use in their skill sets as project managers as they continue forward
  • SBDC (Small Business Development Centers) have a very strong demand for a modified lean Launchpad curriculum program for Main Street businesses. The individual Small Business Development Centers are doing the best they can to come up with a “just getting started” program. They’re all unique. They could benefit from what universities learned from the Lean Launchpad/Lean Startup approach
  • Getting businesses online, giving them social media skills, coaching on the canvas, as a critical thing they were doing for Main Street businesses
  • The teams aren’t done when they’re done with the class. In fact, they’re actually starting a real business during the class.

University Experiments

  • At Ryerson the University incubators are open to entrepreneurs throughout the community, not just enrolled students.
  • UT Rio Grande, where many students did not have access to good internet connections, improved their WiFi to extend it to their parking lots
  • To graduate from the University of Buckingham, you must found a startup before you get your diploma. The startup doesn’t have to succeed. And if it fails early enough, you get to do another one
  • We talked about a need to extend beyond the canonical I-Corps to post class curriculum to understand how the larger ecosystem can be part of that. We also talked about the need to track more than team activity more than just interviews. But to measure engagement with mentors and instructors. And the insights that come from those engagements

Hacking for the Environment and Oceans

  • Real benefit in teaching smaller niche cohorts more focused on a specific problem area
  • All of the coastal universities are finding that this methodology should have impact in these spaces
  • These courses are more complex to put on than even Hacking for Defense type classes, because you’re trying to bring a diverse community together
  • The types of sponsors are 1) nonprofits and from foundations, 2) Coastal Conservancy organizations 2) CEOs who hoping people will help them solve problems. 34) venture funds that are starting to be impact funds, particularly. It’s kind of a very diverse group.
  • For anyone interested in offering this class see –https://www.commonmission.us/sustainability-and-prosperityhacking-for-environment-oceans

The video of the entire breakout session reports is below.

If you can’t see the video click here.

Summary
When the National Science Foundation stopped holding their annual conference of I-Corps instructors, it offered us the opportunity to embrace a larger community beyond the NSF – now to include the Hacking for Defense, NSIN, and Lean LaunchPad educators.

When we decided to hold the online summit, we had three hypotheses:

  1. Educators would not only want to attend, but to volunteer and help and learn from each other – validated
  2. Instructors would care most about effective communication with students (not tools, or frameworks but quality of the engagement with students) – validated
  3. Our educator community valued ongoing, recurring opportunities to collaborate and open source ideas and tools – validated

A big thanks to Jerry Engel of U.C. Berkeley, the dean of this program. And thanks to our organizers The Common Mission Project which provided all the seamless logistical support, and sponsors VentureWell and GCEC and every one of the breakout room leaders:

Ali Hawks – Common Mission Project UK, Chris Taylor – Georgetown, Philip Bouchard – TrustedPeer, Jim Hornthal- UC Berkeley, Michael Marasco- Northwestern, Bob Dorf – Columbia, Tom Bedecarré – Stanford, Dave Chapman – University College London, Paul Fox – LaSalle Univ Barcelona, Phil Weilerstein – VentureWell, Stephanie Marrus – University of California, San Francisco, Jim Chung – George Washington University, Babu DasGupta -University of Wisconsin, Todd Warren – Northwestern, Jeff Reid – Georgetown, Micah Kotch – Urban-X, Radhika Malpani – Google, Todd Basche- BMNT, Todd Morrill – VMG

Join our educators slack channel here

Save the date for our next Educator Summit – June 3, 2021 online.

%d bloggers like this: