A new look at an old problem: the Pentagon’s weapons procurement

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne

The Government Accountability Office is out with its latest assessment of the Pentagon’s most expensive weapons programs. It’s the 20th annual look-see. GAO says it’s seen some improvement and DOD’s management of major acquisitions during that time, but the cost and schedule growth are still big problems. And while every program is unique, one common theme is that DOD tends to commit itself to big systems before it has enough information about technology risks and cost estimates. Shelby Oakley is director of contracting and national security acquisitions at the GAO. She spoke with Federal News Network’s Jared Serbu on the Federal Drive with Tom Temin. about some of the trends in this year’s report.

Interview transcript:

Jared Serbu: A lot of different systems assessed in this report, as there are every year. Can’t talk about every single one of them in a 10-minute radio interview, obviously. But can you take us through any common themes you found in terms of what the trends are toward what’s leading to capability delays in cost overruns, again, understanding that every system is unique?

Shelby Oakley: Absolutely. You pinpointed it right there, you know, we cover a lot of different systems, and they experience a lot of different challenges. And with these being cutting-edge technologies, they are just uniquely challenging to develop. But the bottom line, and the consistent themes that we’ve seen over the years are that these are big investments, billions of dollars being made without information to reasonably assume that they’re going to actually pan out. We consistently see a push to commit to programs before information is available. And then once that happens, once they move forward without information available, it’s difficult for programs to catch up. And as they proceed through development the problems just compound and inertia sets in and more financial commitments are made. And at that point, it just is what it is. We’ve seen over the years that very few programs actually get cancelled. And so DOD has really recognized this consistent challenge that it has, and what the result has been, has really threatened our military advantage and has taken some steps over the past several years to really accelerate capability development to address these things. But we still continue to see that over half of the major defense acquisition programs that we reported are delayed. They’re delayed from achieving their initial operational capability. And none of them are reporting accelerating any cycle times. And so that can be a little rough. And so let me give you an example that I think is illustrative of the kinds of things that we see. So the KC-46, you’re familiar with the tanker program. The program began with a contract valued at about $5 billion, but DOD didn’t really have appropriate knowledge of the requirements or the design for the program. And so as a result, as we all know, experienced a lot of cost growth, and importantly, schedule overruns. And it continues to have a lot of design and stability issues. These aren’t insignificant things, the program has critical deficiencies that still need to be addressed. The air force is forced to accept planes that have deficiencies and deal with the ramifications of that. And so it’s that kind of lack of knowledge at the beginning of programs that we see really have a significant impact across DOD as a body of programs.

Jared Serbu: I want to stick with the theme of moving ahead with insufficient information. The air tanker example seems like a really good one to me, because there you’ve got a program that’s built on a commercial airframe and the technology concepts behind air tankers are not exactly new. Okay. But in other cases, DOD really is inventing new stuff here. And so there, it seems like there’s got to be cases where you just don’t know what you don’t know, how much of that goes on. Are there common pieces of missing information that DOD could get after and make decisions based on?

Shelby Oakley: Absolutely. So that’s exactly what our body of work is based on is knowledge-based acquisition. It identifies those common pieces of knowledge, that programs should have to decrease the risk of unmet expectations. Whether that’s in the form of schedule, or cost or capability, or all three. And so our work over the past several decades has looked at the defense acquisition cycle and identified the types of information that is necessary to make good decisions and avoid some of these challenges. And so, you know, let me give you an example, at milestone B or when development starts, our work has shown that a program should have a sound business case. And that means that the requirements of the program match the resources. And when we’re talking about requirements, we mean, the time and the technology, for example, or what it’s being asked to do can actually be done within existing resources. And, there’s specific types of information that help decision makers determine that and so understanding whether or not the technologies some of which may be new, have been developed enough to be able to be tested in an operational environment. And that gives really good information to say okay, we can move forward with basing the design of our program off of these technologies that are going to be the game changers that are going to be able to provide us with that capability. And so we have often advocated for spending a little bit more time before you begin a program to mature those technologies such that  information is available upon which to base a good sound business case for the program. And same thing with resources, like what you’re doing is a realistic or not, assessment of the cost or schedule that it’s going to take to get what you want within the timeframes you want that can provide good information for decision makers of “Can we fit this program within our portfolio of programs? Can we afford it over the long term? And what adjustments can we make if we can’t?” That often just doesn’t happen with these programs. And so while we’ve seen some improvements over the years, for sure, DOD has definitely made some good strides. We still continue to see the lack of information in some of these big decisions. And why that matters is because once the program is approved, and you start awarding contracts, and the industrial base is engaged, it’s really difficult to stop that inertia, even when things aren’t going exactly the way you want them to.

Jared Serbu: Do you have any good news stories to tell us out of these 63 programs? I mean, are there any that you looked at that really did embrace some of the principles that you just talked about and move forward with good knowledge ahead of time,

Shelby Oakley: Just generally, we’ve been able to directly tie the use of the knowledge-based practices that we advocate to significantly less unit cost growth and scheduled delays on programs. And so we’ve been able to do that analysis. But if you wanted to ask me like, is there one sterling example of a program that implemented every knowledge-based practice? The answer’s no, there’s not. But I think what our data shows is that even implementing some of them, especially at the beginning of development can have a positive effect. And so one good example that we saw. We reported in 2018, the navy has a program called the expeditionary transfer dock expeditionary sea-based program. And it put a lot of work into attaining good design and construction knowledge before making the commitments. Before agreeing on the cost and schedule and you know, awarding the contract and that enabled the program to be able to achieve its initial capability with $697 million in cost savings and no cost growth. And so just having that level of knowledge before committing to the program allowed them to see those outcomes. We also have the middle tier of acquisition programs or a new pathway within DOD that are really intended to be the vehicle for DOD to get capabilities to the warfighter faster. And we have seen some successes in that pathway as well. One MTA program that we could highlight is the F-15EX. That program is intended to fill a gap between fourth generation fighters and fifth generation fighters like the F-35. So this program did a lot to be able to execute under the rapid fielding pathway. It had most of the elements of a business case in place before it began. And as a result, it remains on schedule, and it’s likely gonna achieve its initial operational capability in 2023. So, you know, there are some successes that we can point to, but you know, again, not an overwhelmingly sterling example to point to though.

Jared Serbu: Fair enough. I wanted to ask a little bit more about those MTA programs, because it seems like the congressional intent there was to incentivize DOD to move ahead on programs where there was a reasonable amount of knowledge or expectation that you could get something into the field within five years. Or at least prototyped within five years. Has DOD seem to have been using section 804 In that way, as a vehicle for these programs with bigger foundations of knowledge?

Shelby Oakley: Not always. So as you mentioned, that is the intent of the pathway, right, that DOD is able to provide an operational prototype or field capability within a five-year time period. So five years in the acquisition world is very short, obviously, right? I mean, most programs take upwards of 20 years to get out, which is the problem. So at its very core, it would have to be based upon a high level of knowledge about what capabilities are available. And really, from what we’ve seen, so far, the pathway has enabled the services to get programs started faster. They’re able to start them much more quickly because they’re not beholden to traditional acquisition policies, like the joint requirements process or the DOD 5000. But based upon the kinds of programs that we’re seeing using the pathway, it’s really unclear that many of them will actually finish any faster and provide capabilities to the warfighters faster. So you just give you an illustrative example. As you know, more than half of the programs and MTA programs we reviewed or are planning to transition to the major capability acquisition pathway. So they’ll become MDAPs, eventually to either pursue, like additional development or production through that pathway. And so the question is, are these programs really meeting the intent of actually delivering capabilities to the warfighter if they become a series of strung together programs across different pathways? And so we really hope that DOD will use the experience gained and the MTA efforts to shorten the overall timeframes once in the major capability pathway, but that’s really dependent on DOD taking advantage of the knowledge that’s gained in that MTA pathway and structuring those acquisition programs in a good way. And so we are seeing some outcomes like the Army’s MPF program, mobile protective firepower. It began in 2018, as one of the first MTAs. And you know, it has conducted vehicle assessments and limited-user testing all within three years. And it’s going to be on track to award a low rate production contract this fiscal year. And that seems like the kind of thing that is you know, exactly what the MTA pathway is intended for. But other programs like the Air Force’s ARRW program, the air launch rapid response weapon, has experienced a lot of challenges that indicate that its schedule and its approach before that program under the MTA pathway was a bit aggressive. And importantly on that one, you know, they were moving forward despite these challenges with wanting to buy additional missiles, and Congress had to step in and say no, you know, you need to fund more development for this program before you go into buying, you know, production quantities.

Jared Serbu: And speaking of pathway Shelby, before you go, I definitely want to ask you about the new software acquisition pathway, because that does seem to me like one of the biggest reforms. DOD has made its own acquisition instructions in recent years. I was a little bit surprised to find that out of those 63 programs, you only found one that was even starting to use the software acquisition pathway. And I know DOD is using it for other things. Is the answer just that they’re really just applying it to things that are more pure IT rather than weapons systems that are software intensive?

Shelby Oakley: To be fair, since the software acquisition pathway is newer than almost all of the programs that we’ve reviewed in our annual assessment of weapons programs, it’s not really that surprising to us that most haven’t switched their software development efforts over to that pathway. And that’s something that we plan to continue to kind of keep an eye on is the extent to which programs are kind of parsing off those software development efforts within their MDAPs, or their MTA programs to be executed within that software pathway. Because that’s a capability that has been provided through the adaptive acquisition framework is to be able to use multiple pathways to achieve your intended purpose. So we expect that we’re going to see a growth in that number going forward. But  an important thing that we’re really seeing is that while the pathway is a super important element of improving how DOD goes about acquiring software, the practices have to catch up to what those policies are. And that’s really where we see challenges. We see a majority of MDAPs and MTA programs who tell us they’re using modern software development approaches. And  modern software development approaches are really contingent or hinged on or identified by frequent deliveries and user feedback and that kind of thing. But we’re just not seeing that within the programs that we review. We’re seeing longer delivery timeframes, lack of user feedback, those kinds of things that would characterize a modern approach. And so we just really think that DOD needs to make sure that programs are indeed embracing those practices, not just giving them lip service, and we have a lot of ongoing work looking at DOD reforms to its software acquisition practices, and we’re hoping to be able to get that issued early next year, so stay tuned for some good findings there.

        Read more: Acquisition

Tom Temin: Shelby Oakley, director of contracting and national security acquisitions at the GAO speaking with Federal News Network’sJared Serbu.