Steve Blank

We just held our eighth session of our new national security class Technology, Innovation and Modern War. Joe Felter, Raj Shah and I designed a class to examine the new military systems, operational concepts and doctrines that will emerge from 21st century technologies – Space, Cyber, AI & Machine Learning and Autonomy.

Today’s topic was Artificial Intelligence and Modern War.

Catch up with the class by reading our summaries of the previous seven classes here.

---

Some of the readings for this class session included What The Machine Learning Value Chain Means For Geopolitics, How Artificial Intelligence Will Reshape The Global Order, An Understanding Of Ai’s Limitations Is Starting To Sink In, and The Panopticon Is Already Here.

AI and The Department of Defense
In our last class session General Shanahan, described the mission of the Joint Artificial Intelligence Center (the JAIC) which is to insert AI across the entire Department of Defense. He also said, “The most important hire I made in my time at the JAIC was the chief technology officer, Nand Mulchandani.” Nand changed the culture of the JAIC, bringing in Silicon Valley tools for product development, product management and for the first time a culture that focused on UI/UX, MVPs and continuous integration and deployment.

In this class session Nand Mulchandani, JAIC CTO who just completed an extended stint as Acting Director, continued the discussion of AI and the role of the JAIC.

In addition to Nand, the class also heard from Chris Lynch, founder of the Defense Digital Service (DDS), now the CEO of Rebellion Defense, a new vendor of AI to the DOD. One of the main purposes of the class is to expose our best and brightest to DoD challenges and inspire them to serve. Chris’s story of why he started DDS and how he built the team is a model for how you create a movement.

I’ve extracted and paraphrased a few of their key insights below, but there are many others throughout this substantive discussion, and I urge you to read the entire transcript (here and here) and watch the video.

—-

Nand Mulchandani

How the JAIC is Organized
The JAIC is a tiny team, yet we have a product directorate with 32 products that we’re building across six verticals – all the way from warfighter health to Joint warfighting to Business Process automation, humanitarian assistance, disaster relief, cybersecurity and predictive maintenance. And we own the AI autonomous weapon systems ethics and policy for the DOD.

We have an acquisition arm and Congress is probably going to give us acquisition authority this year. And we have a missions team headed up by a one-star Flag Officer with six O-6 level officers (Colonels and Navy Captains) heading up the different missions.

Selling JAIC Like Enterprise Sales
When General Shanahan brought me in a year and a half ago, we were focused on building one-off products that in some sense were what I called “using a peashooter against a tank.” One product at a time wasn’t going to change the trajectory of the DOD. This is where I brought in the thinking of how we build businesses here in Silicon Valley. We needed to be building leveraged business models in scale. Leverage is a key part of how you change a 3-million-person organization through technology.

Everything that we do at the JAIC has to be built and done with leverage in mind. I organized my missions teams like an enterprise sales team. We took all the colonels and the captains and said, “You are now going out there and finding the repeatable customer patterns. You’re not finding the one-off custom projects to come build. Instead you need to find the patterns, where I can build a single piece of IP or technology and then “sell it” to all the combatant commands and services equally.” Because doing one-offs is never ever going to change the DOD.

JAIC Is Applied AI – Focused on Impact
AI is not a single technology. It’s picked up every piece of technology in statistics and regression and we’re dealing anywhere from string data to numeric data to audio, language, still image things, full motion video object recognition.

But we take this technology and apply it to a particular set of customer problems and focus on the impact. I take a very practical approach to this. I’m an entrepreneur and an implementation guy. So, my focus is on practical ways of moving this big rock, in a in a tactical, tangible way, slowly and surely, while big thinkers will the work on the broader theoretical pieces.

For instance, inside the DOD we have aging equipment and we have a number of business processes that are really less than optimum. How do we tackle those with AI which can become quick wins so that success begets success?

However, using AI for new warfighting capability that’s much, much more complicated. That’s the stuff that Rebellion and Anduril and a number of other companies are helping us with.

Connecting Systems End to End
We’re applying AI across the board. Take a look at the diagram below. On the left is the Pentagon with  highly connected data centers with high bandwidth. And on the right, you’ve got the tactical edge, where we’ve got UAVs, tanks, spacecraft, etc.

At the JAIC we’re focusing on what we call the autonomy space – close combat, AI for small unit maneuvers, around things for SOCOM, and what the Marines and others are going to have to do with the new National Defense Strategy and new operational concepts.

And then we’re also focused on autonomy. The magic words – Joint All Domain Command and Control (JADC2) and the Advanced Battle Management System (ABMS) – are the automation pieces. How do you connect all of our systems end to end? That requires targeting logistics, fires, everything else. Today, it’s a process that’s half manual, half PowerPoint, half Excel, half whiteboard, and half a bunch of lieutenants, majors and colonels all drawing stuff and cobbling it together. If we agree that the next generation of warfare is going to be fought at the speed of software, you need these end-to-end systems all connected together in a backplane, a platform.

Now the problem is, in the diagram below on the left, this is what our DOD architecture looks like today. It’s a bunch of vertically integrated snowflake applications with a mouse and a keyboard and a screen. Applications are developed in silos, making it impossible to enable centralized functionality.

And for students of economics, you see the cost curve. The graph on the left just doesn’t work. Your marginal costs and your average costs of building software do not go down over time. What’s ironic is that software is one of the few businesses where the more you build, the cheaper it gets, it should.

The curve you want is on the right – lower marginal cost per application, rapid development, increased developer productivity, etc. And the only way you’re going to get there is through building common platform services and the application architecture that internet-scale companies do today. And in the DOD that discussion and ideas are completely missing. And so we continue to build stuff as it is on the left. The discussion we’re having in the Pentagon is how do we move things from left to the right.

Moving from Vertical Silos to Horizontal Platforms Enables “Software defined warfare”
Our observation is that the architecture of DOD systems is vertically scaled. What that means is that for that last few decades the goal for every weapon system is about how does each generation get a bigger and bigger version of the same weapon system?

Today that means we just made our targets bigger for our adversaries. Think of a giant aircraft carrier sitting out in the ocean with a hypersonic missile targeted against it. It’s like a giant, vertically scaled data center that we used to have 15, 20 years ago. The architecture changes we need to make in our design for the next generation Combat Systems are precisely the same things that we had to do to change the way we operated data centers. Which is moving from stateful, long running, individual systems, to horizontally scaled, stateless systems. We need them attritable, able to work in denied and degraded environments.

This is what we’re thinking when we call it “software defined warfare.”

However, AI as a services-oriented architecture has to get built out on top of an infrastructure platform. The biggest problem is that no one in the DOD owns running that. So while others can run the JEDI cloud for the DOD, nobody owns running application services for the DOD. And the JAIC can’t run them because we’re not an operational software organization.

Joint Common Foundation
One of the biggest mandates we’re focused on now is the Joint Common Foundation, the JCF. It’s an AI development and data environment for the DOD that democratizes access to compute and data.

The vision is a tactical team sitting out in one of our remote bases. Imagine if they can power up a secure laptop, crank out 30 lines of Python code; and grab a set of services – logistic services, mapping services, targeting services, etc,– that are DOD wide available directly through the JCF; and crank out a piece of code that they get into production, use it for a month and then throw it away.

That is the reconfigurability and speed that we need to have at the DOD. So if we are ever in combat, the reconfigurability of our infrastructure, whether it be hardware infrastructure at the tactical edge or a back end systems to react, that is the level of game that we have to have, or we’re toast. That’s it. It’s that simple.

—–

Chris Lynch
Culture and the culture of change – Getting the Job to Run the Defense Digital Service
In 2015, the Chief Technology Officer at the White House, Todd Park, mentioned I ought to meet with the Department of Defense who were thinking of creating the Defense Digital Service, a DOD version of United States Digital Service. The goal was to bring modern, private-sector tools, technology, and talent into the DOD to solve high-impact challenges.

And Todd Park said, there’s going be a lot of people who want to do this, who want to lead innovation and a lot of people are going to come through, they’re going to pitch their version. I thought about the job like a startup. I came in with a pitch around a “SWAT team of nerds,” one that would work on problems of impact and things that matter. I remember the first time I walked into the waiting room for the Office of the Secretary of Defense. It felt like “Shark Tank.” There were a bunch of other people all going in to do their pitch and talk about their idea for what they wanted to build. Some of them had presentations, and all of them had things printed out of course, because it’s the Department of Defense. Here, we’re talking about technology and innovative ideas, and doing things different with technology and software, and of course, nobody was able to do an actual presentation with a computer.

I chose to show up that day representing what I thought was the culture of what it had to be from the very beginning. That if we tried to build the bridge by being what they were, it wasn’t going to be right. It wasn’t going to be authentic. And we would never attract the best people into the most important mission in the entire world – the mission of defense and national security.

So I showed up in a hoodie and I showed up as me. I didn’t print off a presentation. I pitched the idea, let’s do the SWAT team of nerds. And, and I can remember it felt really, really, weird standing in that office with people in suits and people in uniform in the military. It was just such a different world, completely and totally unlike anything that I had ever seen.

And you know, that’s what they wanted. And that’s what I wanted to be in that part of the story. Because I felt it was important that we actually get people to show up to do the mission. I think that that’s probably one of the most critical things that became the basis of how I thought about what we were building.

Recruiting for the Defense Digital Service
I told people, I want you to leave your job where you’re paid more than what I’m going to pay you. I want you to leave your job, where you’re getting free meals, where you’re living and working around a bunch of other software engineers. And I want you to come into a place that is going to be so unbelievably difficult, frustrating, sometimes demoralizing, and very, very difficult. I want you to come into that.

And I want you to give me six months to one year. But I also just want you to be you.

And when you leave, if you do your job here, if you do the thing that we showed up to do, and if we are successful in what we’re trying to accomplish, it will change your life until the day you die. You’ll never be able to get it out of your blood. And that’s the place that we built. I think that that’s the culture, I think that’s the right place to be.

Because it allowed us to show up in what we were comfortable wearing, we became known as, the people in hoodies. And you would have the Vice Chairman of the Joint Chiefs of Staff doing a presentation for newly minted generals. And they would show a slide with a picture of me with orange tennis shoes on. And it just became our uniform. And it also made it really easy to be like, “Oh, that’s Chris Lynch. He’s a nerd, right? That’s the person that I want to talk to.”

And that was the culture that we decided to build. Just keep focused on results and not getting caught up in sizzle. But getting caught up in real things, in building real tangible stuff. And I think we were successful.

Starting Rebellion Defense
When I left the Defense Digital Service, I felt like I was standing at the shore of this vast sea of all the things that I might do next.

Rebellion Defense was built around the idea of let’s do something spectacular. Let’s continue that mission, but at a scale that would be impossible inside the DOD. And we would be able to bring in incredible new people who wanted to work on something meaningful and impactful.

Surprises from Being Outside the Government Selling In
It’s funny, one of the things I never had to worry about on the inside of the Pentagon was budget. As it turns out, finding who has a budget and can allocate funds is a lot more complicated than you’d think. Just because you find someone who has a budget, they have to find a contracting office to do the buy.

My second point is, if you simply believe that you’re going to show up with an awesome product and awesome technology, guess what? That doesn’t mean anything. if you’re selling this customer technology you already lost. The DOD is a mission-driven organization. Which means you have to understand what they are doing. If there’s a person saying, “Oh I need more AI and ML,” they’re likely not the person who has the budget at the end of the day.

Next, this space has a lot of companies in it that have been here for much longer than you. And you probably don’t know anybody in this space. They do. They know them all. They previously have occupied the jobs that the current people in those roles are doing. And so they have all those relationships. That’s a moat.

And incumbents and competitors say they do pretty much anything that you will say that you do. They’ll say, AI and Machine Learning? We do AI/ML on quantum clusters with Cheddar cheese. And literally, I have had people who send me the most random word-salad construction of words and phrases, I don’t even understand what you’re saying. I get that all the time. I can’t tell you the weird stuff that I get sent my way.

So you have to find that partner that has a belief in solving a problem. They have to know what is real and what is not.

How Will AI Be Used? Why Should Silicon Valley Engineers and Our Students Get Involved?

We either show up or we cede our ability to influence and decide on that policy.

If the world’s greatest technologists do not show up and build the systems for the Department of Defense, to lead in this area and decide the ethical and moral boundaries that we believe in, and that we as a nation are comfortable with, somebody else is going to show up and lead in the way that those technologies are built. And they will decide how they’re used and that will be the definition of ethics and morality.

So you have one very, very simple choice: Show up. If you don’t show up, then you don’t get to play a part in the discussion. Because if other countries who do not share your beliefs become the leaders in how these capabilities are used in a military context and they deploy them, it doesn’t matter what you think. That becomes the norm.

I believe that technologists – people like you, and people like me – we can’t give that up to somebody else. I don’t want somebody less capable than the best in this country to show up and build those systems.

Postscript

The Department of Defense awarded a $106 million contract to build the JAIC’s Joint Common Foundation to… Deloitte Consulting.

Read the entire transcript of Chris Lynch and Nand Mulchandani talks here and here and watch the video below.

If you can’t see the video click here

Lessons Learned

 Nand Mulchandani

• The JAIC acts like DOD’s AI service bureau, building leveraged business models in scale.
  • It’s working on a portfolio of 32 products in 6 verticals across the DOD
  • It’s looking for repeatable customer patterns
• It’s applying AI to both autonomy and automation applications
  • For example, the Joint All Domain Command and Control (JADC2) and the Advanced Battle Management System (ABMS)
• The DOD needs to move from vertical silos to horizontal platforms
  • AI needs to become a services-oriented architecture built out on top of an infrastructure platform
  • Hopefully the Joint Common Foundation will do just that

Chris Lynch

• Built the Defense Digital Service as a SWAT team of nerds
• How he motivated his team is inspiring:
  • “If you do your job here, if you do the thing that we showed up to do, and if we are successful in what we’re trying to accomplish, it will change your life until the day you die. You’ll never be able to get it out of your blood. And that’s the place that we built.”
• Surprises at Rebellion Defense included:
  • How to find customers with a budget
  • Competing with incumbents who came from the revolving door
  • Competing with other companies who claim “we can do that, too”
• Why work with the DOD on AI?
  • “If the world’s greatest technologists do not show up to lead and decide the ethical and moral boundaries that we believe in, and that we as a nation are comfortable with, somebody else is going to show up and lead. And they will decide how they’re used and that will be the definition of ethics and morality.”
• Show Up

 

Filed under: National Security, Technology Innovation and Modern War | Leave a comment »

We just held our seventh session of our new national security class Technology, Innovation and Modern War. Joe Felter, Raj Shah and I designed the class to examine the new military systems, operational concepts and doctrines that will emerge from 21st century technologies – Space, Cyber, AI & Machine Learning and Autonomy.

Today’s topic was Military Applications of Artificial Intelligence.

Catch up with the class by reading our summaries of the previous 6 classes here.

---

If you can’t see the slides above, click here.

Our guest speaker was General Jack Shanahan LTG (ret), former Director of the Joint Artificial Intelligence Center (JAIC).

Some of the readings for this class session included: Can The Pentagon Win The AI Arms Race?, The Ethical Upside To Artificial Intelligence, The Coming Revolution In Intelligence Affairs, Artificial Intelligence For Medical Evacuation In Great-Power Conflict, Congressional Research Service, Artificial Intelligence And National Security.

AI and The Department of Defense
As a lead up to this class session we’ve been talking about the impact of new technologies on the DOD. AI is constantly mentioned as a potential gamechanger for defense.

General Shanahan founded the Joint Artificial Intelligence Center (the JAIC) to insert AI across the entire Department of Defense. The goal is to use AI to solve large and complex problem sets that span multiple services; then, ensure that all of the DOD has real-time access to libraries of data sets and tools. A key part of the strategy was to work with commercial companies to help build these solutions.

Prior to the JAIC General Shanahan ran Project Maven, an unintentional dry-run for how the DOD and commercial companies could partner (or not) to build AI-enabled apps. Maven partnered with Google to build a computer vision tool for imagery analysts to automatically detect objects/targets. The relationship ended when Google employees forced the company’s withdraw­­­ from the project.

There were lots of lessons on both sides — about transparency, about why companies in the 20th century had “federal systems divisions” that worked exclusively on government projects, while the rest of their company pursued commercial business — as well as a lot of relearning lessons the Valley had forgotten (see The Secret History of Silicon Valley).

This entire class session was a talk by General Shanahan and Q&A with the students. It’s interesting to note how many of his observations echo ones Chris Brose and Will Roper made in the previous sessions.

I’ve extracted and paraphrased a few of his key insights below, but there are many others throughout this substantive discussion. I urge you to read the entire transcript and watch the video.

—

Tactical Urgency and Strategic Patience
We have a relatively small window to transform our respective Defense Department from Industrial Age hardware-centric organizations, to Information Age software-centric, more risk tolerant ones. That demands the right combination of tactical urgency and strategic patience, but there’s no question we have to move with alacrity on this right now.

War is a very uncertain endeavor run by humans. If we find ourselves on the verge of a conflict with China in 20 years, the idea that we might have a fully AI-enabled force by then will not by itself guarantee victory. If on the other hand, in that same 20-year period, China has a fully AI-enabled force and we do not, I believe we will incur an unacceptably high risk of defeat.

Project Maven – Start With the Problem – Automate Imagery Analysis
We didn’t start Project Maven with an AI solution. We started with a problem: far too much information coming in from the intelligence enterprise; intelligence, surveillance, reconnaissance. It was done manually — intensive, mind-numbing work with analysts staring at video screens for 12 hours at a shift. They couldn’t ever get through that much. It was really hard for analysts to absorb that amount of data and the data was increasing – in volume, speed and in tempo — and it was coming from all sources simultaneously, from unclassified to the highest classification levels.

We weren’t looking to do a 1x solution, a 5x solution or even a 10X solution. We wanted a 100X solution. We needed something that would really change the way we did processing exploitation and dissemination of all this intelligence coming in from all these platforms and centers in the world.

We went out and solicited everything we could find in the Department of Defense. There was tremendous AI work going on in the research labs, but nothing available for doing processing exploitation and dissemination in the near term. The problem was, while the military research labs were doing some of the best research in the world, it wasn’t getting across the classic technology valley of death. (I like to talk about Maven and the JAIC as “AI Now,” and then places like DARPA and the military research labs as “AI Next.”)

Where else did this AI technology exist? DIUx, In-Q-tel and others pointed out to us, “You lament about how much information you’re having to take in and process. Look at how much information YouTube takes in every single day. Your problems are not an overwhelming problem. There are solutions available in commercial industry.”

So how do you take technology from commercial industry, adapt it for the Department Defense purposes, and do it fast enough and do it at scale across the entire Department of Defense? The only way we could do that was to start bringing in those commercial technologies faster and faster.

So, Project Maven started us on the path of bringing an AI-enabled solution for the purpose of intelligence. Largely computer vision in going after the intake from drones. At first tactical drones, then mid-altitude drones, eventually to high altitude manned airplanes and even commercial imagery, working with the National Geospatial Intelligence Agency.

The JAIC – Go at Scale and Speed
After a couple of years of doing Project Maven, Bob Work (then-deputy Secretary of Defense) was itching to move just beyond intelligence. He said Maven was never designed to be only about the intelligence enterprise, we needed to bring in every single mission in the Department of Defense, and we needed to go much faster. There was a revolution happening in commercial industry and the DOD was not was not getting it.

I was chartered to stand up the JAIC in a form that Joe and Raj, and I’m sure Steve would appreciate. Somebody signed a memo saying, “Go stand up a joint AI Center. You don’t have any people, you don’t have any money, you don’t have anywhere to live, but you’ll figure it out.”

Lean, MVPs and the DOD
Project Maven became the basis for what we did in the JAIC. We started with a cross-functional team. In commercial tech you call that an integrated product team, It’s acquisition with software engineering with UI/UX. This idea of U/UX user interface/user experience, a ruthless focus on user experience, the DOD has never been very good at that.

It’s also learning how to put things on contract fast and get things done and field them quickly. It was whatever you can do to get away from the classic DOD stovepipe way of doing business to a much more agile software approach.

Josh Marcuse and the Defense Innovation Board were instrumental in helping us see what those agile principles were and how we could get moving and go fast. And we took the classic commercial tech approach of building minimal viable products: Get something that will work good enough. Let the user know what they’re going to get as opposed to just forcing something down their throat claiming that it was much better than it really was.

Minimal Viable Products
The idea of minimal viable product is not what the department of defense has lived with, it’s out there in pockets. But we need to be much better at that.

What’s “good enough” when we field these capabilities? It was never for us to say. It was for the users of the capabilities to tell us.  We would give some parameters like, “This is a 95% test and evaluation solution. Do you accept it?” “I don’t know. Let me try it out.”

The other reason that we wanted to push AI-enabled capabilities out to the field as quickly as possible is because until users get to play around with it, it’s just science fiction. You hear all these grandiose stories about what AI is. But few people are saying what AI is not. And in the Department Defense it’s more about what it’s not than what it is. There’s so much more we must do to get people to use those capabilities. As soon as they touch it, they say, “Well, if it can do that, what about this?” Every time they’ve said that, we said yes, it can be it can be done. So we’ll make this product better and better in a rapid sort of agile approach of continuous integration, continuous delivery.

That is the future for the Department of Defense. If we don’t get that right, we’re doomed because AI capabilities left to themselves six months down the road will become useless. Just like any commercial software, it’s got to be a continuous development cycle. So that idea of MVP, putting capabilities in the user’s hand, letting them tear it apart, tell us what worked, what didn’t work, what they’d like to see better. That was really the core concept about Maven, and then the JAIC.

The idea of Ops and Intel working closer together, those two worlds merging, is the holy grail idea of ops/Intel fusion. These AI-enabled capabilities are getting us closer and closer to that environment. And this idea of user-defined operating picture is with us today. The performance of that will increase exponentially over the next couple of years.

Continuous Integration, Continuous Development, Continuous Delivery
The other thing about AI is that you train it against one set of data that will reflect, to some extent, the real world. But once you put it in the real world you learn it never works as advertised. The first time you use it in Afghanistan you realize you never trained it against data that had women wearing full-length black burkas, it didn’t know what those were. Interesting little problem. So you get real-world data, feed it back into the algorithm and it performs better and better. A second, third time, so on, and so on. It was all about the user defining what success look like.

You need a cycle of continuous integration, continuous development and continuous delivery.

We were proud at Project Maven that the first updates of our initial algorithms were out the door in four to five months and then got better and better and better — in some cases about every two weeks. In our personal lives, we get updates pushed to us hourly in some of our apps. So that’s the goal. To be able to do that, you need this backend infrastructure and architecture, so that a soldier, sailor, airman, Marine, Space Force wherever they are, can design an app on the spot, relying on this backend infrastructure, that Joint Common Foundation.

We’re not close to doing that yet with some very limited exceptions like Kessel Run, which is not AI, although they’re starting to go down that path, too. But they’re getting into that model of, “How quickly can I get this in an agile sort of software pathway of doing things and push it out to the field as quickly as possible?”

Warfighters Need to Be Demanding Customers
The biggest role that the warfighters have in this is to be very demanding customers and to say what needs to be fixed and not just accept what we’ve done to operators too many times, which is, “Look, here’s the product, take it or leave it. We’re going to get you another version five years from now, and probably $200 million over budget.”

That world is gone. We have got to get to that point where, “Here it is. What’s wrong with it? We’ll fix it as quickly as we can and we’ll update it with real-world data, it will make it better and better.” Demanding customers provide ruthlessly candid feedback, which was never a problem with the Special Operations community.

This concept of leverage and UI/UX, and all of that is so essential to everything we’re doing. And the backend requires things in addition to a common foundation, a data management platform, open API, is all the T&E tools, everything should be available to everybody in the Department of Defense.

You Need a Disrupter
And then of course you need a classic disrupter. Somebody who does not take no for an answer, who breaks a little glass and makes some people a little bit upset in terms of overturning their apple carts. Because there are so many obstacles in the Department of Defense it would be easy to become disillusioned and just give up on the whole enterprise. The role I needed to play was top cover for the disrupter. It’s the combination of this top-down advocacy and pressure, and this bottom up innovation. And then you bring in a disrupter that gets this thing going.

This is what Raj Shah did at DIUX. It’s what AFWERX has to deal with. It’s what Hondo Geurts had to deal with when he was running SOFWERX. It’s what Enrique Oti has done up at Kessel Run, Chris Lynch at the Defense Digital Service. These were classic disruptors — strong personalities running innovation organizations in innovative ways.

Scaling The Organizations Across the DOD
But now you had to take all those models and begin to learn, how do you scale them across the Department of Defense? That is incredibly hard. And that’s what the JAIC was designed to do, you spark the movement for the next decade, 15, 20 years of movement.

But unless you take those organizational models and begin to scale them across the Department of Defense, they’re sitting there as one-off organizations. They’re critical but they’re insufficient. You need to inculcate a startup culture in the institutional bureaucracy of the Department of Defense.

One of the most important things we were working on in the JAIC is this thing called the Joint Common Foundation.

Joint Common Foundation
It is an architecture and infrastructure — for lack of a cleaner term, call it a platform as a service, a DevSecOps or AI/Ops environment. On top of an Enterprise Cloud environment that’s supposed to be called JEDI. To build this dev SEC ops, AI ops environment, which is to give everybody in the Department of Defense equal access to everything from data, to AI tools, to all the security environment, to Jupyter notebooks, you name it, everything you would need to develop AI. The JAIC is building that as a common foundation.

There are 200 people in the JAIC, and that includes contractors, it’s probably not going change the entire Department of Defense and their AI way of doing business.  But what it has to do is leverage more. The idea of everything that JAIC does is a product available to everybody else in the Department of Defense, they come, pull it off the shelf, so to speak, and use that to leverage the entire Department of Defense. Otherwise JAIC will be an interesting organization and last for a few years and go away.

The Joint Common Foundation is designed to make that available to everybody across the Department of Defense. And I think over the next few years that Joint Common Foundation will be what the JAIC becomes known for as much or more than anything else it’s doing. The services will figure out the technology piece, it’s just they need a little bit of a push, that flywheel has to get turning a little bit faster.

New Operating Concepts Versus Doctrine
Even more important than this, is the idea of developing new operating concepts. New operating concepts do not come out of the Pentagon. Pentagon writes great doctrine. Doctrine is all sorts of instructions and directives and great PowerPoint slides; the operating concepts will come from those on the tactical edge or the operational environment.

And what does the world of AI, Enterprise Cloud, 5G, and someday quantum look like? I don’t know, but it’s not for us to say, it’s for the users to be able to figure it out, by letting them try it out in operational settings.  I believe there is no mission in the Department of Defense that will not benefit – from the introduction of AI-enabled capabilities from the back office, to the battlefield, from undersea to outer space, in cyberspace and all points in between.

Read the entire transcript of General Shanahan’s talk here and watch the video below.

If you can’t see the video of General Shanahan’s talk click here

Lessons Learned

• The DOD recognized that AI was one of the potential game changers
  • They set up the JAIC (Joint Artificial Intelligence Center) to see if they could Leverage AI across the DOD
• They built products using modern processes – continuous integration, continuous development and continuous delivery
• The group required a “Disrupter,” someone willing to break glass to push these ideas
• Over time, JAIC has realized that scaling AI across the DOD will not come from delivering individual solutions
  • but by having a Joint Common AI Foundation that’s accessible by all DOD app developers

Filed under: National Security, Technology Innovation and Modern War | 1 Comment »

We just held our fifth session of our new national security class Technology, Innovation and Modern War. Joe Felter, Raj Shah and I designed a class to examine the new military systems, operational concepts and doctrines that will emerge from 21st century technologies – Space, Cyber, AI & Machine Learning and Autonomy.

Today’s topic was The Challenges of Defending America in the Future of High-Tech Warfare.

Catch up with the class by reading our summaries of Class 1 here, Class 2 here, Class 3 here and Class 4 here.

---

Our guest speaker was Christian Brose, author of The Kill Chain and now the head of strategy for Anduril Industries.

Some of the readings for this class session included:Brookings webinar moderated by Michael O’Hanlon with Christian Brose, Mara Karlin and Frank Rose, The New Revolution in Military Affairs: War’s Sci-Fi Future.

War Made New
The required reading for this class was Chris Brose’s book The Kill Chain. We thought the students would find having the author discuss the thinking behind the book enlightening. It was.

There are few people as qualified as Chris Brose to opine on the state of national defense. Before Brose moved into the civilian world at Anduril, he was the staff director of the Senate Armed Services Committee overseeing all the programs, policies, and resources of the Department of Defense, as well as confirming the Department’s senior civilian and military leaders. He was also responsible for leading the production, negotiation, and passage of the 2016-19 National Defense Authorization Acts. He previously was the senior policy advisor to Senator John McCain supporting his work on the Senate Armed Services Committee, the Senate Select Committee on Intelligence, and the Senate Foreign Relations Committee. And before the Senate he was senior editor of Foreign Policy magazine and served as policy advisor and chief speechwriter to then-Secretary of State Condoleezza Rice.

This entire class session was a talk by Chris and Q&A with the students. It would be easy to just put up the video and the transcription in this blog and be done with it. But that would do a real disservice to the insights that Chris offered.  I’m going to extract and paraphrase a few below, but I urge you to read the transcript and watch the video.

Why Did Chris Write The Kill Chain?
Chris was increasingly concerned that the United States was falling behind our adversaries and nobody was paying attention to the extent of the problem.

Chris observed that the reality is that we are being disrupted in a way that most Americans and most members of Congress don’t fully understand and appreciate. China has specifically and explicitly focused on undermining the core assumptions on which the United States has been planning to project military power for 25 to 30 years. The assumption was that we would:

• be able to fight on timelines of our choosing
• control the timing and tempo of military competition and operations
• be able to build up forces and operate from sanctuaries that an adversary couldn’t contest
• be able to move combat power into places that we needed it
• have military technological superiority over any competitor
• be able to move and shoot and communicate with near impunity and qualitatively
• dominate even quantitatively superior adversaries

These assumptions are no longer true.

The Defense Industry
Since the end of the Cold War our defense industry has become increasingly concentrated, consolidated, uncompetitive and essentially hollowed out. We have gotten extremely good about building a force around very small numbers of very expensive exquisite, heavily manned and hard to replace military systems. We built up a system to produce a certain type of military power at a time when that whole business model is being disrupted and undermined — much as Blockbuster Video’s business model was undermined by Netflix and Apple.

Disruptive Technology
The new technologies and capabilities that will be central to military advantage in the future – artificial intelligence, machine learning, autonomous systems, distributed networking, advanced manufacturing, and commercial space, etc. are technologies largely driven by commercial innovation and commercial companies.  The future will be dominated by large quantities of small or cheaper, more autonomous, more intelligent military systems. This is also true of things that are not military platforms: networking, the movement of information, and the weaponization of data.

The Threat Landscape
We don’t know what the world is going to look like. We don’t know what our competitors are going to do. We don’t know what new technologies are going to be developed next month or next year or next decade. And we ultimately don’t know how we’re going to want to organize ourselves and build operational concepts to employ these new technologies.

We need to have more humility around the best way to experiment and feel our way through the future. And take account for what will inevitably happen: We’re going to get things wrong. We’re going to fail to predict the future and we’re going to need to end up in places we didn’t foresee.

We’ve got to get out of the trap of trying to define the requirements for our inputs. We need to value new, innovative, completely unpredictable and surprising capabilities, concepts and organizational Innovation that allow us to solve these problems differently.

Our system is not designed to do that. Our system is designed to try to predict the future in ten, 20 or 30 years. Then write requirements to what we think it’s going to look like and then throw a lot of money at industry to deliver that future on very long timelines. “Shockingly” many of those things are irrelevant when they show up, if they ever show up at all.

Changing Acquisition
As a buyer of technology and capability, the Department of Defense now can decide to buy different capabilities to match this new world. They can create different incentives for different types of industries to work with them and for them. They can create incentives for private capital that’s sitting on the sidelines to flow back into the Defense sector in a way that hasn’t happened for a very long time.

The only way we’re really going to change is by trying to create more and more pockets in the defense portfolio and programs that are open to real competition. We have a system that’s geared around valuing and buying inputs rather than defining what we want our outcomes to be. We need pockets of marketplace-type behavior where actual systems are competed out based on outcome-oriented metrics. And we buy new things more regularly.

The DOD and Congress can create incentives to take advantage of the willingness and ability of leading technology developers to solve these problems. However, they won’t create a new commercial ecosystem if they continue to dole out small SBIR account grants and million-dollar OTA’s (door prizes for showing up) while the same five national defense contractors they’ve been paying for the past three decades still get the billion-dollar programs. The DOD has to write checks to new vendors for programs at scale.

Making Change Happen
The DOD admits they have a problem. They admit they need to do things differently. Now we get down to the difficult questions of execution and implementation, which is where they have foundered in the past. They haven’t ended up in this position due to a lack of people saying the right things. We’re here because they have failed to do so many of the things they have said (in many cases for decades).

From an organizational standpoint, change won’t come internally. Major kinds of organizational reforms tend to originate outside of bureaucratic institutions. It’s going to take an external act, such as the Secretary of Defense coming in to work with the Congress, to essentially say we do need to do things differently. It is going to involve more risk and the only people in our system capable of doing it are our senior leaders, whether they’re confirmed by the Senate or elected by the American people.

Read the entire transcript of Chris Brose’s talk here and watch the video below.

If you can’t see the Chris Brose’s talk click here 

Student Takeaways From Chris’ Talk

Lessons Learned

• The military advantages we had as a nation in the 20^th century are gone
  • China has systematically negated each of our strengths
  • Our nation is no longer guaranteed to win a war
• Our Defense industry has built expensive and exquisite systems that are few in number and now extremely vulnerable
  • That’s no longer the correct model
  • AI, machine learning, autonomous systems, distributed networking, advanced manufacturing, commercial space, etc. are largely driven by commercial innovation and companies
• The Department of Defense has given lip service to change but institutional inertia – measured by actual spending – is holding us back
  • We need to rapidly pivot to using new contractors at scale
• The DOD needs to rapidly integrate commercial tech into new capabilities that
  • Replace attrited assets
  • Enhance existing capabilities
  • Create new capabilities and concepts that leapfrog our adversaries

Filed under: National Security, Technology Innovation and Modern War | Leave a comment »

We just held our fourth sessions of our new national security class Technology, Innovation and Modern War. Joe Felter, Raj Shah and I designed a class to examine the new military systems, operational concepts and doctrines that will emerge from 21st century technologies – Space, Cyber, AI & Machine Learning and Autonomy.

Today’s topic was Defense Strategies and Military Plans in an Era of Great Power Competition.

Catch up with the class by reading about all the class sessions here.

---

Our guest speaker was Bridge Colby, former Deputy Assistant Secretary of Defense for Strategy and then Secretary of Defense Jim Mattis’ point person for articulating his vision for the National Defense Strategy. Some of the readings for this fourth class session included: National Security Strategy, 2018 National Defense Strategy, National Military Strategy Summary, The Age of Great-Power Competition, The China Reckoning: How Beijing Defied America’s Expectations, The Administration’s Policy Toward China, The End of American Illusion. Trump and the World as It is, Indo Pacific Strategy Report 2019

In this session we provided the students with an appreciation of how the United States National Security Strategy arrived at the conclusion that we are in an era of great power competition with Russia and China. Next, we introduced the National Defense Strategy (NDS) which describes how the military supports the overall National strategy. The NDS observed that we not only faced non-nation states (terror organizations,) but going forward we have to plan for 2+3 adversaries (China, Russia, Iran, North Korea and the non-nations states.) The NDS provided an outline of what we need to do (called Lines of Effort) to transform our military.

If you can’t see the slides click here.

Joe Felter (who was the Deputy Assistant Secretary of Defense for South and Southeast Asia) began the lesson providing background and context for understanding – What happened? Why did we shift our strategies and military plans? And what do these plans look like today.

Great hopes for international security
In 1989, when the Berlin Wall came down, it marked the symbolic end of the Cold War. The United States emerged as the dominant power in the international system and its Cold War rivals appeared to be moving down a path of reform. We had great hopes for an international security environment that would advance common interests among large and small nations through international cooperation and engagement. Russia at the time showed promising signs of moving closer to democracy.

The break-up of former Soviet states put the country on the path of increasing liberalization and reform. Former Warsaw Pact nations expressed interest in working with and aligning more closely with its former rivals. Several joined NATO. Meanwhile, China’s economy was growing at an extraordinary rate and becoming more integrated with countries across the region and beyond. All prevailing theories of modernization predicted that this growth and would lead to increasing liberalization and reform in China. It was considered to be on a trajectory towards becoming a “responsible stakeholder” willing to play by the rules of the established order. Beyond these encouraging developments with our former Cold War rivals the US assumed a position of unparalleled military dominance. Shortly after the fall of the wall this overmatch and dominance of US military power was put on display during Desert Storm where the US achieved quick and decisive victory destroying the world’s 4th largest Army in 100 hours of ground combat.

Optimism turns into reality
Fast forward to 2017. Conditions were far from where we hoped in the heady optimism following the Cold War. Putin’s Russia is intent on undermining the US and West in any way it can –  aggression in the Crimea and Ukraine and destabilizing activities in Syria; Venezuela and beyond. Adding to this are it’s state sponsored poisonings and assassinations, cyber-attacks against nations and election meddling in the U.S. and other countries. In China Xi Jinping and the CCP pursuing a deliberate whole of government approach to projecting influence if not dominance of the Indo-Pacific region and beyond. Disappointingly, the liberalization and reforms so many assumed would accompany its rapid economic growth did not occur.  

The CCP explicitly states its intention for China to be a dominant power with benchmarks and years identified. For example, President Xie leads the Central Military Commission that in 2012 committed to building a military that can dominate the region and “fight and win global wars by 2049.” To do this, China is pursuing a military build-up of an historic scale with a seven-fold increase in its defense budget in the last two decades. It is investing in high tech weaponry to close the gap and in many cases extend their advantage in a range of military capabilities and technologies. Beijing engages in predatory economics – driving states into significant debt burdens forcing them to make “debt- for equity” swaps in places that undermine their sovereignty Its Belt and Road Initiative makes infrastructure and other investments with a clear nationalist agenda. It is increasing its de facto project power projection capabilities by developing and establishing access to a network of dual use ports, airfields, and other facilities across region. 

Some argue that China is even in the early stages of establishing a strategically located naval base in Cambodia which course co-instructor Joe Felter raised the official alarm following a visit to the southern port while serving as a senior official in the Department of Defense. China’s militarization of features in the South China Sea is perhaps the most egregious example of its illegal efforts to build military capabilities and extend the PLA’s ability to project power. Despite Xie’s promise to President Obama in Jing Peng Rose garden 2015 and the international tribunal ruling by the Hauge in 2016 that its claims have no basis in international law, China continued to fortify its illegal claims building runways, radars, missile sites, storage facilities and other improvements. See time-lapse videos of the reefs turning into a military base here and here.

The U.S. National Security Strategy and National Defense Strategy
These were the conditions we confronted in 2017 when the current National Security Strategy (NSS) and National Defense Strategy was developed. These strategies reflected the realization we are in long term competition with Russia and China and must make a clear-eyed assessment and treat these competitors for who they are and not as we want them to be. As the NSS states Just as American weakness invites challenge, American strength and confidence deters war and promotes peace. The 2018 National Defense Strategy (NDS) pulled no punches. It was a real wakeup call for the military and the country. As Bridge Colby said in his talk to the class, “others described it as the first realist document we’ve had as a country in a long time.” Bridge points out that after the Berlin Wall fell we were the sole superpower and the country really didn’t need a defense strategy. We had so many resources relative to the plausible threats that we could essentially overwhelm any adversary. Besides explicitly acknowledging we are in long term strategic competition with Russia and China. It said that our regional priorities would shift from the Middle East to Indo-Pacific and China. And China is recognized as the more powerful and potentially dangerous threat. The National Defense Strategy outlined three major lines of effort that the Department of Defense needed to execute to face these new 2+3 challenges: 

• Line of Effort I: Build a more lethal force
• Line of Effort II: Strengthen Alliances and Build partnerships
• Line of Effort III: Reform the Defense Department

And the US has made important progress across all three of these lines of effort. (Our students heard this quarter about some of the efforts aimed at reforming the Department of Defense – requirements and acquisition reform from Will Roper, new innovation organizations like the JAIC (Joint Artificial Intelligence Center,) from General Shanahan, AFWERX, Kessel Run, NavalX,…and they’ll hear more later this quarter from General Raymond about standing up a new service branch – the Space Force.) So how are we doing so far? First the bad news. China is making gains and many are at the expense of state sovereignty across the region which in some cases will be difficult to reverse (ie Hong Kong.) Under Xi and the CCP, China is structurally set up in many ways to compete more effectively e.g. with its coherence and continuity of leadership, civil/academic/military fusion. Other examples include how China’s State-Owned Enterprises can be employed by the CCP for coordinating and projecting influence more efficiently. But there is good news that bodes well for the outcome of this long term competition.  The US has a vision that is largely shared and embraced by those that wish to see the region remain free and open and for the rules-based order to endure. Significantly we are not asking states to choose between the US and China- but rather to choose their own sovereignty and a vision for future. Our challenge, however, is to ensure our actions match our strategy- demonstrating that the US is a reliable partner and will deliver on its stated goals and objectives. Bridge Colby gave us some compelling insights on the 2018 National Defense Strategy and participated in an informative Q&A session with our students. He provided an insiders account of the development of the National Defense Strategy and an informed assessment of its execution.

Read the transcript of Bridge Colby’s talk here and watch the video below.

If you can’t see the Bridge Colby talk click here

Lessons Learned

• The National Security Strategy arrived at the conclusion that we are in an era of great power competition with Russia and China
• The National Defense Strategy (NDS) describes how the military supports our nations overall strategy
  • It observed that we still face non-nation states (terror organizations,) but have to plan for 2+3 adversaries – China, Russia, Iran, North Korea and the non-nations states
  • Our regional priorities shifted from the Middle East to Indo-Pacific
  • China is recognized as the more powerful and potentially dangerous threat
• We want our adversaries to choose diplomacy not war. To do so, we..
  • are developing a lethal force (the NDS Line of Effort I) to decisively defeat adversaries in future conflict
  • this ensures no state calculates it can successfully use force against the US to achieve its objectives
  • and therefore it must rely on diplomacy and other means short of war
• The US has a significant advantage in its network of alliances and partners
  • Strengthening these alliances and building new partnerships (the NDS Line of Effort II) will be critical to our ability to compete effectively

Filed under: National Security, Technology Innovation and Modern War | 1 Comment »

We just held our third session of our new national security class Technology, Innovation and Modern War. Joe Felter, Raj Shah and I designed a class to examine the new military systems, operational concepts and doctrines that will emerge from 21st century technologies – Space, Cyber, AI & Machine Learning and Autonomy. Today’s topic was Sourcing, Acquiring and Deploying Technology for Modern War.

Catch up with the class by reading about all the class sessions here.

---

Class 3:
Our guest speaker for session 3 was Anja Manuel, former State Department official, founding partner of Rice, Hadley, Gates and Manuel and author of This Brave New World: India, China and the United States. Some of the readings for the session included: Esper’s Convenient Lie, How to Win the Tech Race with China, The Age of Great-Power Competition.

If you can’t see the slides click here.

Winning the Wars We Knew
Joe Felter, former Deputy Assistant Secretary of Defense for South and Southeast Asia, started the class showing excerpts from General MacArthur’s famous Duty, Honor, Country speech given to the Corps of Cadets at West Point in May 1962. In what would be his final address to his alma mater, MacArthur admonished these future leaders of the United States military that, “Through all this welter of change, your mission remains fixed, determined, inviolable: it is to win our wars. Everything else in your professional career is but corollary to this vital dedication. You stand as the Nation’s war-guardian, as its lifeguard from the raging tides of international conflict.”

Back in MacArthur’s day, fighting a conventional conflict akin to the wars America experienced in the 20th century was certainly not expected to be easy. Confronting the massive armored formations of the Soviet Union in the Fulda Gap or engaging in a proxy war fought in another theater would be costly and difficult to prevail (not to mention the specter of escalation to a nuclear exchange). But with known adversaries and technologies the weapon systems and operational concepts we expected to rely on to win our future wars were, however, easier to anticipate and simpler to define. 

For example, in the aftermath of Pearl Harbor the U.S. knew how – and largely where – to respond. The country mobilized its resources and industrial base, raised powerful military forces and projected power – directing it at a defined enemy and the enemy’s industrial base. In conventional state-on-state warfare, the operational and tactical level activities that support a strategy to win are often clear. You mass fire power on objectives. You destroy the enemy’s military and industrial capabilities and seize terrain. All those things are missions that the military can get their head around. 

In MacArthur’s time we defeated our enemies and drove them to unconditional surrender. We did so by using the superior power (both quantity and quality) of our weapons and how we employed them. 

After WWII the weapons and defense systems we acquired and deployed reflected this experience. In the 1950’s we leveraged our industrial capacity and innovated by producing five new fighter designs and three new classes of aircraft carriers, and nuclear-powered attack and ballistic submarines.

As we pointed out in previous class sessions, in the 20th century, requirements were known years ahead of time and the DoD built incrementally better versions of the same platforms. (Although our experience in Vietnam would foreshadow the issues of unconventional warfare the U.S. faced in Iraq and Afghanistan.)

Winning our wars remains-as MacArthur characterized- the military’s fixed and inviolable mission. However, the conditions we will fight in the future are much different, than in the wars we prevailed in during MacArthur’s time. How we prepare for and fight future wars must reflect these new realities of modern war. Adaptability has always been an essential attribute of successful militaries. 

We will discuss these ideas further in later class sessions.

Two Acquisition Paradigm Shifts 
Raj Shah, former head of the Defense Innovation Unit, pointed out that men and women in uniform have signed up to support national security with the equipment that they are given and must make do with what you give them. These men and women are quite resourceful to achieve the mission as best they can with the gear they have.

However, if we give them equipment that fails to keep up with the threat or state of the art, our warfighters bear a cost (ultimately with their lives) that they and the nation will pay.  So, it’s incumbent on us to think about the ramifications of these acquisition decisions.  It’s better to take risk in the hallways of the Pentagon than on the battlefield – risk aversion in the former will force risk acceptance in the latter, with potentially grave consequences.

There are two paradigm shifts going on in the DOD. The first, the transition from buying a small number of exquisite systems versus large number of low-cost systems. And the second, the shift from the DOD contracting everything from defense primes to building software themselves or serving as the integrator for off-the-shelf commercial systems.

To illustrate the escalating cost of military hardware, Norm Augustine, former CEO of Lockheed famously graphed out how much each airplane costs. On the bottom left, a Wright brothers plane in 1910 cost ~$5,000 in today’s dollars. If you follow the cost line up and to the right, the F 22 Raptor – is a $300 million a plane (if you include all the R&D costs).

Augustine’s tongue-in-cheek conclusion was that if we followed this trend line, by 2050 the entire defense budget will purchase just one aircraft. And that aircraft will have to be shared by the Air Force and the Navy for three and a half days a week, except on a leap year when it will be available to the Marines for that extra day. 

While Augustine was being facetious, the consequence of escalating costs of these exquisite systems plays out in the way he described. The Air Force said we needed 750 F-22s to meet all the threats. They ended up buying 187. They said they needed 132 B-2 bombers. They ended up buying 21. We design these world-beating systems, but because they’re so expensive, and it takes so long to build them, and the threats change before they get deployed, we’re going to be left behind.

The same story is being played out in our satellites in space. The National reconnaissance office builds satellites the size of school buses and they can do more than any other countries. But we just have a handful of them — all of them big, fat targets. But Planet Labs and SpaceX are launching thousands of satellites that individually aren’t as good, but collectively illustrate the trend of mass commodity versus exquisite. 

At the same time, the Department of Defense has finally realized how important software is. In fact, many of our most advanced airplanes and ships are really software delivery vehicles, meaning the software, not hardware, is the primary driver of capability. Over the last few decades, the ability of the DoD to design and even understand modern software design had atrophied. The good news is that DoD has recognized this and has announced a new policy for acquiring software, and have start building ‘software factories’ with names like Kessel Run (USAF) and Kobiyashi Maru (Space Force). Raj had a front row seat in this revolution: 

 

If you can’t see the video click here.

Many of the innovations that will shape future conflicts will increasingly occur in the commercial technology base. Advancements in these technologies will be driven by consumer demand and the potential for profit- not government directives. Requirements are not known years ahead of time. So, the DoD needs a new way of engaging and acquiring these fast evolving technologies. Fortunately, real progress is happening across the DOD. There had been a wellspring of new initiatives and reform. Hopefully the most successful of these initiatives will be broadly scaled across the department and federal government. These positive trends include: Software color of money reform, Middle-tier acquisitions, Other Transaction Authorities (OTAs), Commercial outreach organizations, SIBR reform, software factories, talent pipelines, rapid prototyping, digital engineering, and more (it’s a very exciting time to be a reformer in the DoD). But these initiatives will need to overcome institutional barriers to scale; our hope is that Congress, uniformed leaders, political appointees, and traditional contractors will continue to work together to improve the ability of democracies to deter and prevail against potential adversaries.

Guest Speaker – Anja Manuel
Anja Manuel is the author of This Brave New World, an overview of the political and economic relationships between India, China and the US.

If you can’t see Anja Manuel‘s talk click here 

Lessons Learned

• 20^th Century U.S.-centric rules for war were built around known adversaries and technologies
  • The conditions we will fight in the future are much different 
  • The Vietnam War would foreshadow the issues of unconventional warfare the U.S. faced in Iraq and Afghanistan
• The Department of Defense is coming to grips with two major transitions
  • from buying a small number of exquisite systems to a large number of low-cost systems
  • from the DOD contracting everything from defense primes to building software themselves or serving as the integrator for off-the-shelf commercial systems

 

Filed under: National Security, Technology Innovation and Modern War | 2 Comments »