At the heart of the problem lies a critical shortage of skilled personnel within the agencies tasked to enforce the sanctions. Faced with the most comprehensive sanctions in a generation and a thinning workforce to implement them, government officials are left with very few options but to take a page from the private sector and integrate AI technology into their investigation operations.

AI’s speed, scope, accuracy and efficiency would optimize sanction enforcement efforts. The technology’s capacity to analyze vast amounts of data and rapidly identify criminal activity and potential risks make it a formidable tool in the enforcement of financial regulations, including international sanctions.

Challenges of scaling Russian sanctions

President Joe Biden recently announced a major scaling-up of Russian sanctions, targeting two of Russia’s largest financial institutions — Sberbank and Alfa-Bank — along with an expanded list of individuals tied to the Kremlin. These joined the already voluminous list of sanctioned entities that were targeted directly or severely limited through exclusion from SWIFT, the global payment system for cross-border trade.

But the West’s capacity to maintain, let alone expand, its current sanctions program is already experiencing significant gaps and limitations. In early 2020 — at the onset of the Coronavirus pandemic — the U.S. Government Accountability Office released a report describing how several U.S. agencies responsible for enforcing sanctions were short-staffed and have been unable to fill enough full-time positions to operate effectively for years.

Without adequate human or technological support, regulators may be compromised in their efforts to counter the bad actors working to circumvent the sanctions. Many of them may be counting on the possibility that overwhelming mountains of data in the global financial system could slow down or block regulators’ actions. If enforcement teams adopt advanced AI technologies to boost sanctioning efforts, however, the same violators would soon realize the unfavorable odds in making such a wager.

Risk discovery, detection speed and accuracy

Risk detection lays the groundwork for all sanction enforcement actions. Moreover, detection speed and detection accuracy, via the elimination of “false positives,” play a critical role in determining the probability of success.

Here’s where the case for AI is most compelling. AI can help private financial institutions more than double the number of risks detected, reduce false positives by 60%, and increase the pace of risk detection by 40%. Through advanced machine learning technologies, AI systems are capable of analyzing large, complex, noisy and incomplete datasets (a.k.a., topological data analysis) to identify the latest and riskiest criminal behaviors. AI detects anomalies in payments and transaction patterns in a discreet manner that doesn’t depend on interrogations and won’t tip off the institutions, companies, or individuals seeking to sidestep sanctions.

AI can also help analysts develop behavioral models based on past sanction violations or similar financial crimes. Based on those models, it can analyze large volumes of data from a variety of sources to automatically pinpoint current violators. It can even identify emerging threats, to uncover the “DNA” of complex crime behaviors on its own.

Equally important is the capacity of an AI application to present its findings in an easy-to-understand format suitable for end users that are not data scientists or IT specialists. And all of this analysis can be executed in just a fraction of the time it would take most trained investigators — a crucial advantage in an endeavor in which time is of the essence, and there’s already an enormous drain on resources, time and capital.

AI could be a game-changing ally for regulatory and law enforcement agencies in their efforts to thwart sanctions violators. And, with political relations with Russia evolving by the day, if not the hour, the sooner support is brought in to help track new and existing sanctions against the country, the better.

Raj Srivatsan is Vice President, Civilian at SymphonyAI.

The center will focus on tactical autonomy, meaning systems that act independently under the bounded authority of human support. The systems support missions like situational awareness, force protection, cyber defense and logistics. The center will also focus on system collaboration and man-machine learning.

The collaboration will be the first academic research center affiliated with the Air Force and the 15th connected to the Defense Department.

“It’s a gap in our suite of research institutes right now that we don’t have one focused on this area of autonomy,” Air Force Secretary Frank Kendall said at the Pentagon Monday. “I’m very focused on the threat of Chinese military modernization and what that means in terms of our viability of our forces for the future. Part of the future of the military is going to be autonomy, there’s no doubt in my mind to that. We’re seeing increasing evidence, evidence for almost in every conflict that occurs.”

The Air Force is committing $12 million per year for the next five years to the collaboration. DoD will be adding an extra $2 million per year.

The Air Force and DoD are currently working with the 11 HCBUs that qualify as high-research activity schools as designated by the Carnegie Classification of Institutions of Higher Education to decide which school will lead the center. That school will be responsible for building a consortium of educational institutions to conduct research.

The collaboration is also working a secondary goal of increasing diversity and inclusion, a mission DoD has been working on since the national response to the murder of George Floyd.

“This is an opportunity to tap into universities that have enormous amount of capability in science and technology,” Kendall said. “The HBCUs put out about 30% of the scientists and engineers that are that are produced by that community.”

DoD will also work on growing the STEM community within HBCUs. The center will not work like other academic research consortia. The Pentagon wants the consortium to build capabilities, but also build its research abilities.

“We understand that there are historical inequities, and we want help them build capacity, as well as deliver results to us,” said Victoria Coleman, DoD’s chief scientist. “We want to advance their efforts to move at least one, hopefully more than one, institution from the Carnegie Foundation Research Classification from ‘R2,’ which means a high research institution, to an ‘R1,’ which is very high research.”

Carolyn Parent, president and CEO of Conveyer, joins Aileen Black on this week’s Leaders and Legends to discuss leadership in the technology sector.

Conveyor is a provider of an AI-driven document virtualization platform that transforms instructions into friendly, easy-to-follow, mobile-first applications that customers can retrieve with QR codes and view on any device.

Prior to joining Conveyor, Parent led companies at the forefront of SaaS technology innovation in real-time digital communications, mobile, AI, data and security analytics. She founded companies, managed new growth-oriented software technology business units, and helped navigate organizations through six profitable evolutions or exits (IPO, M&A, privatization and large-scale divisional spin-offs).

Throughout  her career, Parent has built high performance teams and led organizations to their highest levels of success. She did that by leading with “optimism and enthusiasm,” and keeping the lines of communication open so that all stakeholders from the individual contributors to the board of directors have the information they need meet their objectives.

According to Parent, she tries to ensure that there are “no surprises” during the decision making process. That way everyone knows where they stand and what to do.

She says “The more informed people are the better they can do their job.”

Parent believes that leaders of any organization must empower and support their teams then stand aside to let them find their own paths to success. She tries to lead by being what she calls a “velvet hammer” — strong but not abrasive. It’s a lesson she learned from a former mentor.

Parent said strong effective leaders must be able to “work with people and get to know them so they can see your point of view. Coming on too strong without understanding where or how that person may perceive it could cause issues. ”

She said leaders must always be able to adapt to change because “nothing lasts forever,” and she recommends networking with other leaders and executives as you make your way up the corporate ladder.

Networking comes naturally to Parent because she loves people and learns from everyone she meets. She added that networking has paid off over the years and has fueled her success. In fact, she said networking is the key to having a successful career.

Parent is passionate about helping the next generation of women leaders and she has some advice for them.

“Don’t let someone else pick your path,” she said. “Every company is a tech company these days. You don’t need to code to be in tech. You don’t need to be an engineer to work at STEM companies. They need sales, marketing and customer service people too. Remember if you can see it, you can be it. With hard work you can make anything possible.”

The IRS announced Friday that individuals delinquent on their taxes, who receive a mailed notice from the agency, can call an artificial intelligence-powered bot and set up a payment without having to wait on the phone to speak with an IRS employee.

Taxpayers are eligible to set up a payment plan through the voice bot if they owe the IRS less than $25,000, which IRS officials said covers the vast majority of taxpayers with balances owed.

Taxpayers who call the Automated Collection System (ACS) and Accounts Management toll-free lines and want to discuss their payment plan options can verify their identities with a personal identification number on the notice they received in the mail.

Darren Guillot, IRS Deputy Commissioner of Small Business/Self Employed Collection & Operations Support, told reporters Friday that the agency’s expanded use of voice bots and chatbots will allow the IRS workforce to assist more taxpayers over the phone.

The IRS earlier this year answered about three out of every 10 calls from taxpayers.

“If you don’t have more people to answer phone calls, what are the types of taxpayer issues that are so straightforward that artificial intelligence could do it for us, to free up more of our human assisters to interact with taxpayers who need to talk to us about much more complex issues,” Guillot said.

IRS Commissioner Chuck Rettig said the automation initiative is part of a wider effort to improve taxpayer experience at the agency.

“We continue to look for ways to better assist taxpayers, and that includes helping people avoid waiting on hold or having to make a second phone call to get what they need,” Rettig said in a statement

Guillot said the IRS in December 2021 and January 2022 launched bots that could assist taxpayers with questions that don’t require authentication of the taxpayer’s identity or access to their private information.

These bots could answer basic questions like how to set up a one-time payment, and answered more than 3 million calls before the end of May.

But this week, the IRS expanded its capabilities and launched bots that can authenticate a taxpayer’s identity and set up a payment plan for individuals.

“It verifies you really are who you say you are, by asking for some basic information and a number that you will have on the notice you received. That gives you a phone number to call and speak with the bot,” Guillot said.

Guillot said taxpayers can name their own price for the payment plan, as long as a taxpayer pays their balance within the timeframe of the relevant collection statute or up to 72 months.

Once a payment plan is set up, the bot will close the taxpayer’s account without any further enforcement action from the IRS.

“Those taxpayers didn’t wait on hold for one second,” Guillot said.

Guillot said the IRS is ramping up its bot capability incrementally to ensure the automation can handle the volume of calls it receives. The bot, he added, is currently at about one-quarter of its full capability, and will reach 100% capacity by next week.

The bots are available 24/7 and can communicate with taxpayers in English and Spanish.

Later this year, Guillot said the bots will be able to provide taxpayers with a transcript of their accounts that includes the balance of their accounts.

Guillot said the IRS worked closely with National Taxpayer Advocate Erin Collins on the rollout of the voice bot.

“She raised legitimate concerns that some taxpayers, because they can name their price, may get themselves into a payment plan that’s more than they can afford,” he said.

The IRS is working to ensure that the bots ask some additional questions to ensure taxpayers are able to afford the payment plans they set for themselves.

Guillot said that this week’s rollout marks the first time in IRS history that the agency has been able to interact with the taxpayers using AI to access their accounts and resolve certain situations without having to wait on hold.

“I have friends and family that have to interact with the Internal Revenue Service, and when I hear them talk about how long they’re on hold that bugs me. It should bug all of us,” Guillot said.

Guillot said the IRS also added a “quick response” QR code to the mailed notices that went out to taxpayers. The QR code takes taxpayers to a page on IRS.gov showing them how to make a payment.

Guillot said the IRS originally expected to launch this capability by 2024, but was able to expedite the rollout given the perceived demand for this service.

The IRS in recent years has seen low levels of phone service that have decreased further since the start of the COVID-19 pandemic.

IRS is looking to further expand the range of services voice bots can provide, and is part of a broader effort to improve taxpayer service.

“We never lose sight of our first interaction with every single taxpayer is never enforcement. It’s a last resort. Our first effort is always around that word ‘service,’ and trying to help customers understand the tax law and almost always work out a resolution with them meaningfully,” Guillot said.

The idea is to determine priorities for greater investment in the near future.

Dave Frederick, CYBERCOM’s executive director, said DoD already integrates basic machine learning applications and commercially available products that incorporate machine learning in its cyber defense mission. And he hopes to have a better idea of what’s still needed in terms of machine learning this fall. But he did offer one example of the new directions CYBERCOM wants to explore: synthetic users.

“Our offensive cyber operators, the way they get ready for a mission is we set up a simulated network that is supposed to emulate the adversary environment as much as possible. Just setting up a static network isn’t sufficient in terms of realism, because on a real network, you’ve got system administrators, you have users, you’ve got a lot of people using computers, and some of them may notice that something’s odd, and tip off to their [system administrator], or tip off their security operations center, and take action,” Frederick said on June 14 during the Defense One Tech Summit. “And so to elevate our game in training and mission rehearsal, we want to see if we can develop, in partnership with industry, some abilities to emulate the actual presence of people operating the network, to where we’re not just looking at routers and switches and software packages. But we’re also dealing with this uncertainty that can be introduced, when you have a system administrator who happens to notice, ‘hey, this just doesn’t look right to me, I’m going to investigate this a bit,’ which could throw us off our game. And so that’s just one of many examples that the team is doing pretty early in the survey work right now.”

Frederick said continuous monitoring is another area CYBERCOM is looking at, which aligns pretty well with industry priorities, as well as detecting misinformation. Meanwhile, he said, CYBERCOM will be looking to partner with DARPA for innovation in offensive capabilities.

One thing Frederick is counting on machine learning to help with is taking some of the load off of cyber analysts and defenders. He said CYBERCOM is exploring one such way, involving tools that would allow defenders to compare suspicious activity against massive databases of known malware. That way, if the defenders aren’t sure whether what they’re looking at is malicious, they can confirm much more quickly. And if there’s not an exact match, the algorithm can also help determine with a fair amount of confidence whether it might be a new variant.

“So that’s an important example; it just takes work off the protection teams, it allows a more junior operators to be more effective in their mission,” Frederick said. “So not everybody has to be master level, because they can use these tools to help them in their job.”

That’s why Frederick wants to begin including more machine learning tools in training for cyber operators. Not only do they need to know how to use these tools to get the maximum effect out of them, but they also need to be able to recognize their limitations. The algorithms can only do so much; they only process data, albeit much faster than humans can. But human judgement is still required to evaluate and implement the insights machine learning produces. So that needs to become a basic part of cybersecurity training.

That will also require improving DoD’s talent pipeline for cybersecurity and machine learning. Toward that end, Frederick said CYBERCOM launched an academic engagement network about six months ago, which is starting to pay dividends. He said the network is closing in on 100-member universities. That’s helping with student outreach; Frederick said students outside of universities with strong ties to the military rarely considered DoD as a potential career option, and often didn’t know DoD employed civilians.

CYBERCOM is also leveraging the network to innovation in the area of unclassified threat analysis.

“There’s so much information available at the unclassified level, that we want to work with students on research projects on that topic,” Frederick said. “And we’re also trying to encourage student research and faculty research on our hard innovation problems. So this fall, we’ll be sponsoring at least 10 student research projects focused on our innovation problems. We’ll be launching that in August, advertising it out, and going through the process there.”

And the network is also facilitating two way education opportunities between DoD and academia. Frederick said two professors from partner universities have spoken to the CYBERCOM workforce about both the technical and disinformation aspects of election security, which is one of CYBERCOM’s mandates. Frederick said the CYBERCOM workforce asked to be educated about those topics.

And a CYBERCOM expert has already held a talk at one university on the subject of zero trust. And next week CYBERCOM will be participating in a faculty workshop at the University of Cincinnati on the topic of persistent engagement strategy.

“We’re trying to help scholars that look at cyber strategy and policy and at the technical side of cyber have a better understanding of persistent engagement, and the defend forward strategy, some of the core aspects of how CYBERCOM fits into DoD in an integrated deterrence effort,” Frederick said.

The Defense Department is evaluating its own processes with an eye toward making it easier for small companies to provide innovative solutions. Deputy Defense Secretary Kathleen Hicks said the DoD innovation steering group is central to those efforts, which includes mapping out the transition process from prototype or commercially available product to DoD contract.

“I think transition clearly is one of our biggest problems. The so-called ‘Valley of Death,’ scaling up to fielding and full-scale production is a piece of that. There are many places you can fall off the cliff in transition for the Defense Department. So we are mapping that system, which to my knowledge has never been done, which surprises me. So I keep waiting for someone to tell me it has,” Hicks said June 13 during the Defense One Tech Summit. “But right now we have [Research and Engineering, the Chief Digital and Artificial Intelligence Office (CDAO)] and our Acquisition and Sustainment folks sitting down and just building out what is that map? And what are the friction and pain points on that map … where are we falling down in the transitions? And the answer right now, early on, is there are dozens of pain points, spots where we are losing innovation through our system.”

Mapping that system is the first step to understanding how contractors interface with DoD, so the department can start thinking about how to improve that process and any bottlenecks that might exist. But that’s not the only way DoD is attacking the issue.

Hicks said the innovation steering unit also recently developed a DoD “storefront” website for small contractors to begin their interactions with the department. Hicks said that “front door approach” will help the department keep track of all of DoD’s innovation projects across its enterprise, many of which fall under smaller DoD innovation storefronts like SOFWERX, AFWERX and SpaceWERX, while making the department itself more accessible to small businesses.

“If you’re a major contractor with the defense industry, you know how to get to us most likely,” she said. “If you’re a small innovator, you probably don’t. So those little things, like just a webpage that can let you in the door, help.”

One example of the need for that enterprisewide view of innovation is artificial intelligence. That’s why DoD recently brought on Craig Martell, DoD’s first CDAO. AI is a major DoD innovation effort across the department, but it’s happening in various small pockets without a clear path to scale across the enterprise.

Hicks said Martell and his organization will provide a compromise between a federated, enterprise approach to AI, and the ability to model unique solutions and get them to warfighters quickly.

“I have high expectations that CDAO can really be a vanguard for us in terms of demonstrating both that we can create the tech stack we need to get after AI — so that’s the data pieces, the digital awareness, the workforce — and then building up to AI that can enable everything from the boardroom, in terms of efficiencies that we can make on anything from logistics to audit, to making it down to the warfighter in terms of the sensor-to-shooter advantages,” she said.

Hicks also said that the department is taking lessons learned from the original DoD innovation storefront, DIU, and looking at how to multiply those kinds of investments tenfold. For example, DIU taught DoD that many of these innovations that can help support the mission are dual-use commercial technologies. But DIU operates primarily at the prototype level; Hicks also wants to improve focus at the other end of that pipeline, where academia is doing more generalized research on the technologies that inform these prototypes. That way DoD understands the base concepts and can better distinguish the more mature technologies that deliver directly for the warfighter, versus shiny new tech solutions in search of problems.

And that’s because, while most innovation happens in the commercial sector, Hicks said DoD innovation funding is still central to those efforts.

“We have to have better insight into what’s happening in the external world. That’s where I think we have a lot of potential,” she said. “We have the potential to increase partnerships, make sure our workforce has the flexibility to go out into the commercial sector, to spend fellowship time. We can bring more people like Craig Martell into the government, places like Digital Services … where we routinely bring folks in. And of course, the flagship of that actually is [the Defense Advanced Research Projects Agency], where we have long had the ability to attract the best and the brightest into the department. We need to continue doing all of that. And then we need to be able to work with Congress to make sure that we have the room and flexibility to actually bring that technology to bear inside the department.”

DoD’s chief digital and AI office (CDAO) has only been fully operational for several days, but its chief, Craig Martell, says he plans to hit the ground running by working with DoD’s famous bureaucratic processes to add 21st century value to the military.

Talking at the virtual DoD Digital and AI symposium on Wednesday, Martell said he’s already noticing the administrative slow down — he has yet to receive his access card to the Pentagon, and still needs to go through the visitor’s entrance. However, Martell said the point of his office is to use DoD’s power, even the frustrating parts, to expand AI capabilities.

“We’re not going to change the bureaucracy of the whole,” Martell said. “That’s not a challenge I want to put before the team. We need to find the right gaps, the right places where we can leverage value. That value is going to drive a virtuous cycle of change. There’s a lot of things about the DoD that can’t be more like industry. We shouldn’t try to force that square peg in a round hole, right? We need to find out how to keep it the DoD but also make it more efficient and work better.”

Deputy CDAO Marie Palmieri said the office will scale a different operating model for delivering digital technologies. The point of the office is to build end-to-end cohesion on everything from data collection and curation to advanced analytics that will give the agency an advantage in decision-making and operations.

“It really is a collective ecosystem. We had the parts of it, but [we’re] putting it together in a way we haven’t before to deliver that decision advantage that our leaders need,” DoD Chief Information Officer John Sherman said in February. CDAO brings together DoD’s Joint Artificial Intelligence Office, the Defense Digital Service the chief data officer and the Advancing Analytics planform Advana.

Despite trying to veer away from making DoD like industry, Martell has extensive Silicon Valley experience. It’s no surprise DoD chose the former head of machine learning at Lyft to head the CDAO. DoD often refers to its future military command and control plans by using ride sharing apps as an analogy.

“DoD uses ride-sharing service Uber as an analogy to describe its desired end state for JADC2. Uber combines two different apps — one for riders and a second for drivers,” the Congressional Research Service report on Joint All Domain Command and Control (JADC2) states. “Uber relies on cellular and Wi-Fi networks to transmit data to match riders and provide driving instructions. JADC2 envisions providing a cloud-like environment for the joint force to share intelligence, surveillance, and reconnaissance data, transmitting across many communications networks, to enable faster decision making.”

The CDAO will be heavily involved in JADC2 as DoD plans to inject the program with AI components to quicken that decision making.

Martell said part of his job will be easing the way for industry to bring in their technologies, especially ones that can be applied right off the shelf. That ease will be extended down to even the smallest companies.

“One of the things that I want us to spend a lot of time thinking about is how do we not just go to the big players?” Martell said. “How do we make it easy for other businesses too? How do we create a marketplace for startups, for medium-size, for small businesses? Because particularly in the AI space and I’m sure in many other spaces as well, there’s a lot of innovation happening in two-person shops or five-person shops. You know, a good brain with a good idea, we want to be able to leverage all of that.”

Deputy Defense Secretary Kathleen Hicks noted at the same conference that there is a massive innovation ecosystem focused on software in the United States and she wants the CDAO to tap into that.

“There’s a real a part of the impetus I had in the CDAO is there’s a power in bringing a vanguard organization with direct reporting relationship to me and to the secretary at the four star level that can push us in these areas,” she said. “They can build on work that’s been underway. We have a number of procurement vehicles already available. I think there’s five that are really focused on expanding our access in DoD to nontraditional companies.”

Hicks said she hopes the office can also bring in a talented workforce that wants to work for and stay with DoD. The CDAO will be an exemplar for people who want to innovate within government.

The Pentagon’s new chief digital and artificial intelligence office is quickly bringing together multiple tech specialists under one roof in the latest bid to scale a “different operating model” for delivering digital technologies across the Defense Department.

The CDAO hit full operational capability on June 1 and is hosting an online “DoD Digital and AI Symposium” this week. The office merges the Joint Artificial Intelligence Center, the Defense Digital Service, the chief data officer and the Advancing Analytics (Advana) platform that originated in the DoD comptroller’s office.

Margie Palmieri, the deputy chief digital and artificial intelligence officer, compared it to a major merger and acquisition activity in industry.

“Companies go into mergers and acquisitions to be competitive,” she said on day one of the symposium. “And that’s exactly what the Department of Defense is doing. We are increasing our competitive advantage by bringing these different groups together. And for the first time in my career of over 15 years in government at this point, all the right levers of change and influence are coming into play in the CDAO.”

While recognizing that DoD’s industrial age acquisition approaches are still appropriate for the thousands of aircraft, ships and vehicles the Pentagon buys every year, Palmieri said software, digital technologies and data analytics require an alternative approach to scale across DoD. The various organizations coming together under the CDAO all have experience in piloting alternative approaches to the Pentagon’s traditional development and buying processes.

“This team coming together to show what a different operating model looks like is one of our top priorities,” Palmieria said.

Who’s on the CDAO team

In late April, the Pentagon announced Craig Martell would be the first chief digital and AI officer. Martell was most recently head of machine learning at Lyft, previously led machine learning at Dropbox, and he led a number of AI initiatives at LinkedIn. He also was a tenured computer science professor at the Naval Postgraduate School.

“I’m doing it because of the mission,” Martell said during the symposium. “It’s extremely important that we get this right. And there are not a lot of folks who have the intersection of AI and a government background. And when the deputy secretary of defense calls you up and says, ‘We would like you to take this job,’ you have to think really hard about why you wouldn’t take the job, and not the other way around. And I think getting this mission right is extremely important.”

As the No. 2 at the CDAO, Palmieri has more than a decade of DoD experience, most recently as special assistant to the vice chief of naval operations. She was also founding director of the Navy Digital Warfare Office, a relatively new organization focused on data analytics and Project Overmatch, the sea service’s contribution to the military’s Joint All Domain Command and Control concept.

The CDAO also has filled out its ranks with officials from the JAIC, DDS, the CDO’s office, Advana and other DoD organizations:

• Clark Cully, previously DoD’s deputy chief data officer, is now the deputy CDAO for policy, strategy and governance.
• Sharothi Pikar is the deputy CDAO for acquisition and AI assurance. She joined the JAIC in the spring to lead acquisitions. She also previously held positions as acquisition executive at U.S. Cyber Command and as the associate director for cyber strategies in the office of the under secretary for research and engineering.
• William Streilein, who had a long career at MIT Lincoln Labs before joining the JAIC as chief technology officer this spring, is now CTO for the CDAO.
• Joe Larson, currently chief of Project Maven, is leaving the project to be Deputy CDAO for Algorithmic Warfare. Larson helped co-found Project Maven, the Pentagon’s AI pathfinder program, which is now transitioning to the National Geospatial-Intelligence Agency.
• Greg Little is the deputy CDAO for enterprise capabilities. He was previously deputy comptroller for enterprise data and business performance. Little was also program lead for Advana, a big data program that now falls under the purview of the CDAO.
• Katie Olsen, who was the deputy director of the Defense Digital Service, is now the deputy CDAO for digital services.

The DDS in particular excelled at bringing tech talent from outside the government into DoD for temporary tours of duty to work on high-profile problems. Recently, DDS led Project Rabbit, an effort to allow employers to verify their previous or current Afghan employees seeking asylum in the United States.

Olsen says she think the CDAO can help scale the DDS model.

“We’ve been a ragtag team of 50 to 60 people, give or take, for the past seven years,” Olsen said during the symposium. “I’m excited about being part of the CDAO because I think it’s an opportunity to replicate and scale that talent and the [idea that] when you give smart people agency to do things, what can happen.”

CDAO operating model

Palmieri said the groundwork laid by CDAO’s predecessor organizations have helped establish good practices for how to approach digital and AI technologies within DoD.

“The good news is everything that the department has to do to scale digital analytics and AI is relatively known at this point,” she said.

In particular, she said determining the right “feedback loop” between users and developers will be key as CDAO looks to tailor the budget, requirements and acquisition processes to its technology goals.

“We really see the opportunity for the CDAO to put together a different operating model for how do you actually deliver these types of capabilities in a meaningful way to users and create a more rapid feedback loop where requirements, acquisition and funding are all in response to that capability need, instead of driving the pace, as happens today.”

Former Google chief executive Eric Schmidt told Martell he should avoid a wholesale reliance on prime contracting. Schmidt has been highly influential in driving DoD’s technology organizations and strategies in recent years as chairman of the Defense Innovation Board and then co-chairman of the National Security Commission on AI.

“My guess is that at the end of the day, your success will be your own software people in the government, as well as essentially small firms where you’re essentially contracting with a firm, but it’s really one person. And then you’ll have some priming. If you start with a full private approach, you’ll never get there.”

Martell said it will likely be a combination of in-house coders and contracted work, suggesting a collaborative approach.

“We’re not going to grow the talent fast enough to have all of the coders in government to do what we need to do,” he said. “We’re not going to replicate these agile new AI companies. But if we have the authority to say, ‘You’re sitting with us, and you’re our agile team, and you’re going through the loop with us as we’re building it,’ I think we have a greater chance of success.”

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

More than 25% of federal contracting dollars end up in one of six markets. That’s according to analysis by Bloomberg Government. For details and the opportunities they represent, we turn to the team lead for government contracting news and analysis, Amanda Allen joined the Federal Drive with Tom Temin.

Interview transcript:

Tom Temin: Ms. Allen, good to have you on.

Amanda Allen: It’s great to be with you, Tom.

Tom Temin: And the six markets, let’s begin with what they are.

Amanda Allen: Sure. Kind of split up into two categories, we’ve got tech oriented markets, and then we’ve got some ones that would be kind of not tech. And in the not-tech, we’ve got business and financial management services, facility services, and logistics. And then we’ve also got cloud, artificial intelligence and machine learning, and digital services.

Tom Temin: Yeah, that pretty much covers the waterfront for what they’re buying.

Amanda Allen: It really does.

Tom Temin: But it’s not a lot of hardware, like it was in days of yore.

Amanda Allen: Sure. One thing that we’re finding a lot of is that interoperability and consolidation of requirements, the idea that kind of in the business world, things need to go enterprisewide and in the government world, of course, they need to go agencywide or even governmentwide. And so the big money is finding ways to get all of what a contracting office needs or all of what an agency needs or all of what the government needs in one of those areas.

Tom Temin: Right, you hear more and more discussion anyway, at the policy level of shared services among agencies, but that hasn’t really emerged yet as a market that’s discernible?

Amanda Allen: So the Shared Services seem to be, actually, most of the same team who had done this webinar last week, we also did one in March about some particular contracts. And the way that we were framing that webinar, as well as most of our analysis, really, for the last eight or nine months has been on governmentwide acquisition contracts multiple award contracts that are either agency wide or multiple agency. And I think that there isn’t yet a market with boundaries around it. But we are seeing it. It is unignorable at this point.

Tom Temin: And that brings up a good question on the six areas that you did identify on the tech side and the nontech side. Are most of those dollars that are going through there, and it’s a couple hundred billion dollars and its 25% of contracting dollars, are they through the established GWACs or is some of its still full and open competition?

Amanda Allen: In the tech area, we’ve got major vehicles like SEWP and Schedule 70 for GSA. There are smaller, fully competitive opportunities. There are also what I would call kind of good slots to find where you can fit in. Sub contracting is a really big opportunity for folks even who might not consider themselves as subs in most settings. But so much of this work is so wide scale that I think a lot of folks who are winning on the major prime end of things are going to be looking for really experienced partners to be working with down the line, especially with the contracts that have the longer period of performance.

Tom Temin: And anyway, sub or prime it’s still revenue. And that’s all green, I suppose. And getting into the cloud question, what does the spending look like in terms of how much is going directly to the big commercial cloud suppliers services, the Googles, AWS, and so on? And how much is services related to getting agencies to the cloud? Because that seems to be, qualitatively speaking anyway, a pretty big part of the cloud market.

Amanda Allen: It very much is. What we do know is that, one way I’ll take an answer to this, I guess, is saying that the small business share of the cloud computing market, I think it’s 43%, I just actually was able to pull up the number, for the last five full fiscal years. And what that tells me, right, those are not folks who are building their own cloud platforms and trying to break into the market share for the big players. Those small businesses are going to be doing quite a lot of these services-related aspects of cloud computing for the government.

Tom Temin: That’s interesting. So small business gets almost half of the cloud dollars, meaning only half goes to the big players.

Amanda Allen: That’s what we’re seeing in the numbers and what we’re taking a look at analyzing the market.

Tom Temin: Interesting.

Amanda Allen: And the market is growing, so it’s growing in both arenas.

Tom Temin: Got it. We’re speaking with Amanda Allen, she is the team lead for government contracting news and analysis at Bloomberg Government. And that artificial intelligence/machine learning market, how big is that? Do you have the dollars there? And what does that look like in terms of large and small business?

Amanda Allen: Sure. So the AI market is one of the two smaller markets that we have. The AI/machine learning market and the digital services market are significantly smaller dollarwise than the other four. So AI is between about $1.5 billion and $2 billion right now. And we are expecting a slight dip based on our forecasting model for this full fiscal year before it goes up again in 2023 and we anticipate it continuing to go up beyond that, only because everything that we see that’s coming out that’s a solicitation, that is a policy document, that’s a forecast from an agency. Everyone in government needs to have AI-based efficiency in order to get their jobs done. We know that there are budget pressures, we know that the annual appropriations process is increasingly fraught. And the only way that agencies can get things done are by finding ways to get it done quicker. And folks seem to think that AI is the way to do that. What’s particularly interesting is that over the last five full fiscal years, the share of civilian AI market spending had gone down for a bit and is now looking to even back out the Defense Department had really kind of jumped in 2019, 2020, 2021. And what we’re anticipating moving forward is a much more even share between civilian and defense in AI, at least for the unclassified.

Tom Temin: And getting back to the nontech side here, business management, facilities and logistics, how does that look? When you say business management and financial management, what kind of dollars are there and what are the services they’re actually buying?

Amanda Allen: So what we did with that one, so four of our markets are the cloud, AI, the logistics, and the digital services are all Bloomberg Government analysts-defined markets. And the business and financial management market is one of the two that we have that is actually based on PSC codes, the standard for everyone. And what we did is we took three of those sub markets and combined them because what we’re seeing is that Management Advisory Services are super heavyweights. I don’t actually know boxing, but whatever comes after, big guys throwing big punches at each other? And we’ve got north of $60 billion in the last full fiscal year for how big this kind of subcategory combination is. And what this market does is it’s the same conversation, right? The themes of efficiency and consolidation of requirements and rationalization of what an agency or office is doing. And so rather than having government officials learn how to be, “good business people,” it’s bringing in the contractors who already know how to do that type of integration, interoperability and efficiency.

Tom Temin: So are these primarily software buys? Or are they body shop-type buys for people that do, I don’t know, accounting and whatever else they do that relates to business management?

Amanda Allen: It is absolutely, yep it is absolutely some of that accounting. It is absolutely – what did you just call it the –

Tom Temin: Body shop.

Amanda Allen: Body buys? Yeah body shop. It is anything that is technical that, inventories and who’s going where, and so there is a bit of a, we did separate out what the business management part of the accounting books would look like, from the tracking where something is going from warehouse to truck to user. But I think those two are, nicely, two sides of the same coin.

Tom Temin: And just a quick question about facilities, is that simply rent coming from GSA or the agencies that have their own rental authority? Or is there something more that is a trend here?

Amanda Allen: So what I think is the most interesting trend about the Facilities Services market is that it’s modernization is a lot of what’s happening here. So, and definitely it’s not just rent. This is the shipyard infrastructure project for the Navy. This is the Department of Energy, making sure that all of their labs are run in the way that they need to be run for safety, security, national security. This is what GSA in terms of leasing has been making decisions about future demand, because of potential telecommuting as a long-term answer for the government workforce. We’ve also got several DoD depots that have infrastructure improvement projects underway. So this is everything from really big Los Alamos, all the way to, is GSA going to need this building in the future?

Tom Temin: Right. So there’s a pretty good technological and engineering component, then, to all of the facilities work.

Amanda Allen: Absolutely. Because it is not just being a landlord somewhere, right? This is being, or I guess, the building supervisor somewhere. This is making sure that if you’re putting up a new building, that it’s going to have all of the right energy efficiency and that it’s going to have the right wiring so that it will be future-proof in terms of technology and the load that will be required based on how many people are going to be working there. And how many computers they’re going to have hooked up to all of the, all of the wires.

Tom Temin: Sure, and just adding it all up, what is a generally good strategy for these types of technology and services contractors to pursue as we move into fiscal ’23, if there is a fiscal ’23 and beyond?

Amanda Allen: I think what’s really interesting to look at is that you’ve got very few companies probably who could go out for a, let’s say $20, $30, $40 billion modernization and facilities management contract. But those companies are not going to be able to do the personnel security, they’re not going to be able to do the real basic striping the parking lot, right, as the example that everyone likes to use. So this is another one of those areas where even if it looks like it is too small of a market to get into, figure out who’s going to be getting that big prime for doing all the management and be able to offer your services.

Tom Temin: Amanda Allen is team lead for government contracting news and analysis at Bloomberg Government. Thanks so much for joining me.

Amanda Allen: Absolutely. Thanks for having me.

The task force behind the National AI Research Resource (NAIRR) issued an interim report Wednesday outlining how the AI data-and-research hub can put federal AI resources in the hands of more U.S. researchers. A final report is expected this November.

The task force, co-chaired by the White House Office of Science and Technology Policy (OSTP) and the National Science Foundation, envisions NAIRR as a shared national cyberinfrastructure that will allow greater access to federal AI research, high-performance computing capabilities, data sets and testbeds.

Private-sector partners would also have the ability to make resources available through the NAIRR portal.

Lynne Parker, the deputy U.S. chief technology officer and director of OSTP’s National AI Initiative Office, said the interim report marks the “first step toward a more equitable future for AI R&D in America.”

“It’s a future where innovation can flourish, and the promise of AI can be realized in a way that benefits all Americans,” Parker said.

Manish Parashar, director of NSF’s Office of Advanced Cyberinfrastructure, said the NAIRR will be run by a single, non-governmental management entity.

The task force recommends that several federal agencies leading AI research should be funded to work cooperatively and support NAIRR’s resources and management.

“These federal agencies would make new and existing cyberinfrastructure resources relevant for AI R&D available via the NAIRR,” Parashar said.

NSF, the Energy Department and the National Institutes of Health rank among the top spenders on AI research in the civilian federal government.

The NAIRR would primarily serve AI researchers and students based in the U.S. who are pursuing foundational AI research.

The NAIRR taskforce is a congressionally directed federal advisory committee authorized by the National AI Initiative Act. Members include AI experts from the Energy Department and National Institute of Standards and Technology, as well as top technology companies and universities.

The NAIRR seeks to lower the barriers to participate in AI research, and allow more researchers to get involved in AI R&D.

AI research relies on access to large amounts of computational power and data. Parker said this demand for resources often limits this research to large technology companies and well-resourced universities.

“AI is transforming our world, and a growing resource divide between those who have access to the resources needed to pursue cutting-edge AI and those who don’t threatens our nation’s ability to cultivate an AI research community and workforce that reflects America’s rich diversity. It could also prevent us from harnessing AI in a manner that benefits all Americans,” she said.

Parker said the NAIRR promotes more equitable access to federal AI resources, and opens up opportunities for individuals that would not otherwise have access to engage in this kind of research.

“To prevent unintended consequences or disparate impacts from the use of AI, it matters who is doing the AI research and development,” Parker said.

The interim report outlines four main goals for the NAIRR — to spur innovation, increase the diversity of AI talent, increase capacity for AI research in the U.S. and advance trustworthy AI.

The task force released a request for information seeking feedback on its interim report. OSTP and NSF will also host a public listening session on June 23.

Parker said that the task force is particularly interested in hearing who perhaps have not been as engaged in AI research and development

The task force plans to release a second report in November that provides a detailed roadmap and implementation plan for the NAIRR.

“This interim report is its findings and recommendations on sort of the ‘what’ that we think the NAIRR should be all about. But the details on exactly how we achieve all of these high-level recommendations are something that will be left for the final report,” Parker said.

Parashar said NAIRR will primarily host open scientific data, which has fewer constraints around sharing, but he said the task force recognizes there will be some cases where data sharing agreements are needed.

“Overall, it’s really looking at setting up the incentive mechanisms, where the data providers find value in sharing data and being able to provide a broader accessibility to data and amplify the impact of that data, as well as creating this value ecosystem around data, so that it becomes more natural to be able to share data broadly,” Parashar said.

The Federal Data Strategy and the Foundations for Evidence-Based Policymaking Act direct agencies to make federal data more accessible to the public, businesses, and researchers while upholding data privacy.

The Biden administration also sees greater access to federal data as a more urgent priority since the start of the COVID-19 pandemic. OSTP on Thursday issued guidance outlining ways to make federally funded research have a greater benefit for underrepresented communities.

“One of the lessons of the COVID-19 pandemic for data sharing is that public access policies can benefit all of America,” OSTP officials wrote in a May 26 blog post.

Jennifer Bisceglie, founder and CEO of Interos, joins Aileen Black on Leaders and Legends to explain how she turned a small startup into a dynamic supply chair risk management and operational resilience company.

Interos has disrupted the traditional supply chain risk  management discipline, bringing 24/7 real-time visibility into every supplier at every tier  and at every location around the world.

It gives companies the tools they need to detect and respond to supply chain crises before they happen. Clients include the Defense Department, NASA and a host of Fortune 500 companies.

Bisceglie described Interos as a “see everything, do anything, operational resilience company” that is now valued at more than $1 billion.

According to Bisceglie,  she was able to set a clear vision for what she wanted to accomplish at Interos. She then “hired a great team” to carry that vision out.

Bisceglie said one of the keys to effective leadership is to give  your teams all the resources they need to do their jobs and then get out of their way. She describes her leadership style as “up and out, not down and in,” and now focuses most of her attention on moving the company forward.

Bisceglie also believes that culture is key to a successful strategy execution. She interviewed the first 150 people hired at Interos because she wanted to be sure they shared her vision on the organization’s culture.

“It is important that we have a culture that values mutual respect. It is OK to have healthy friction, but with respect. My door is always open and you need to value everyone’s opinion,” she said.

Bisceglie’s leadership style was also influenced by one of her favorite books: Sharon Hadary’s How Women Lead: The 8 Essential Strategies Successful Women Know. She said the book taught her how to use information to drive results and how to make that information your ally.

That lesson stuck with her when she went to Sand Hill road in the Silicon Valley to meet with potential investors for her company.

She learned that “first you need to be a ‘subject expert.’  Second, you need to be sure you are solving a problem. Third, you need to have ‘grit and be scrappy.’ The fourth is you need to know your numbers and the story and the results that can be driven. Build business acumen.”

Bisceglie also shared her thoughts on what government can do to build a more resilient supply chain.

“Get rid of uncertainty and mandate transparency in the supply chain. This will allow people to visually see the supply chain, the risks and the ethical issues associated with it. This will allow us to de-risk our supply chains and allow people to make choices on where and what they consume,” she said.

Bisceglie also offered some advice to the next generation future women entrepreneurs.

“Just do it. If you have an idea, you can make it happen,” she said. “Ask for help. Have persistence and apply grit. Just do it.”

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

The civilian and Defense sides of the government have taken a big step together to move the Defense Department’s innovative, nontraditional contractors to the mainstream of federal contracting. It takes the form of a deal between the Defense Innovation Unit and the General Services Administration. GSA is in the midst of moving about a dozen such companies from DIU’s prototype roster to the GSA Multiple Award Schedules Program at the ACTIAC Emerging Technology Conference in Cambridge, Maryland, GSA’s Senior Technology Adviser Sam Navarro shared details on the Federal Drive with Tom Temin.

Interview transcript:

Sam Navarro: We talked about today about enhancing digital transformation and innovation in the government space, one of the things we want to do is we want to take good ideas and scale them across government. So the Defense Innovation Unit’s been a key partner now to bring in innovative small companies and show that they have a proof of concept to add value within the Defense space. So we just inked a brand new deal to make sure that those vendors and their capabilities are available for utilization by our civilian counterparts. And so we’re excited to fast lane them on the schedule as fast as we can, so that their capabilities are available for civilians.

Tom Temin: So these are vendors that have come through DIU, perhaps on an OTA now graduating them, if you will, to regular Federal Acquisition through the schedules?

Sam Navarro: Yes. And as we know, the OTAs have their constraints. And the constraints are there for prototyping proof of concept kind of work. So when you want to scale that work to the enterprise, it’s valuable if they’re on a governmentwide solution. And that’s the value we want to bring to the table.

Tom Temin: These are companies that are not otherwise on the schedules or on any, say GWAC that you might know of. These are absolutely new to the federal market through DIU?

Sam Navarro: Yes, that is correct. And we could fastlane them. They have a sponsor, so we could get them on a lot faster. We’re looking at at least from 15, anywhere to 30 days getting them on schedule so they’re readily available for government competition.

Tom Temin: How many companies are we talking about here?

Sam Navarro: Right now we’re looking at about a dozen, which DIU is working with, but they are constantly recruiting innovative companies. And our objective is as soon as they prove value to the government, we onboard them on our site to the schedule.

Tom Temin: Well give us a couple of examples. I mean, can you name a couple of men what it is they have brought to DIU?

Sam Navarro: Yes, and precisely for not going into endorsements, I will not say specifically which companies, but we we are looking at six different focus areas. Area in particular is AI and ML. And in the area of robotics, there’s been a lot of use cases where they’ve shown innovation within the DoD space. And when we think about AI, ML and robotics we just had the CTO of DHS here today. And there’s a lot of concepts for Customs and Border Patrol or keeping the federal officers safe from different scenarios where robotics can be applied, in a comprehensive manner.

Tom Temin: Yeah. So in some ways, the mission of in that sense of the Defense Department look like the mission of DHS in terms of tactics and requirements and field work, and so on.

Sam Navarro: Precisely.

Tom Temin: When you say these companies that have a sponsor, that would be the DIU?

Sam Navarro: Yes, yes and Mr. [Michael] Brown, who’s in charge of the DIU, signed an agreement with Laura Stanton, who is the associate commissioner for ITC, and they have set the critical metrics for delivering results for the government. So they want to make sure that within six focus areas, we get vendors in the emerging technology space, that citizen-servicing agencies can leverage and benefit from as well.

Tom Temin: And how do we ensure that just coming on to the schedule doesn’t become a valley of death for them? There’s 50,000 vendors on the schedules, you’re adding 12 more into the ocean. Is there a way of, again you can’t promulgate certain companies, you’re the referee, but how will anyone know about them?

Sam Navarro: Great question. So GSA has an Office of Customer and Suppliers Engagement. And so we’re working in collaboration with them to make sure that the use cases that they’ve been able to apply within the Defense space align to the civilian space by engaging agencies. So we’re looking at making sure that we’re proactive in engaging agencies that have use cases which align to these vendors, not to endorse a particular vendor, but to let them know that we have emerging technology vendors that may be able to meet their needs, and then compete solicitations among them.

Tom Temin: And the GSA does operate conferences and training sessions for vendors from time to time and it helps them work their schedule better, so they’ll be able to participate in those?

Sam Navarro: That is correct. And we want to also not only on-ramp them, but educate them, train them on like, market research as-a-service is a tool that we utilize within the GSA space. So how do they respond to those? And when they respond, making sure we have industry days, reverse industry days for them as well so that they can show their capabilities to civilian agencies.

Tom Temin: And just to reiterate, you expect they could be on the schedule within a month or so?

Sam Navarro: Yes, we’ve already actually got about four or five already on board. And they’re already looking to compete for solicitations. And we’re slowly but surely ramping up to get the last batch that doesn’t onboard.

Tom Temin: And just an acquisition question, does that enable DIU to use regular acquisition to buy from them if they want to ramp them up to production scale?

Sam Navarro: Yes, DIU and other entities within DoD, we’ve kind of heard that same theme within Research and Engineering, and other areas, such as software factory solutioning, where DoD has a need to scale. Once they find out a use case works, how do they get it applied across the force? And so we’re working heavily to make sure that they have that availability.

Tom Temin 5:52
Right. And the other thing they’re looking for is a, I guess for lack of a better word, a legal way to be able to buy at scale under the FAR from people they bought, not at scale under the OTA authority.

Sam Navarro: Yes, “fair opportunity” means fair opportunity. Everyone gets a bite at the apple so precisely, making sure that they stay within the rules.

Tom Temin: And a little closer down to earth: You’ve been working on supply chain security, and supply chain assurance issues through ACT-IAC.

Sam Navarro: Yes, ACT-IAC conducted a survey. And with a lot of the challenges we’re seeing in the supply chain 75% of industry vendors have had some sort of impact because of supply chain challenges. One out of five face challenges with keeping the doors open from those. So the question is how do we collaborate with industry to create best business practices, which enables us to break down barriers, but also keep cyber supply chain risk management principles in place?

Tom Temin: And the output of this will be some kind of a playbook?

Sam Navarro: Yes. First and foremost, we’re doing a shark tank this June, June 6, where we’re bringing in folks who have great ideas. And we want to hear what they got to say and vote on which ones are the best for government utilization.

Tom Temin: And who votes?

Sam Navarro: We’re having sharks from across government and academia. So right now we have Allen Hill, who’s the senior executive sponsor, and we’re cycling in some other sharks for June 6. And so by the end of that, we’ll have some winners to announce to the broader public.

Tom Temin: And the playbook comes out by the end of the fiscal, roughly or so?

Sam Navarro: Yes, we’re breaking up – well, bottom line, what’s the challenge? The challenge is there’s a lot of content for cybersecurity practitioners, not a lot of content for acquisition workforce. So the objective is, by the end of the fiscal year release a playbook where we can talk about software bill of materials, quality assurance plans, terms and conditions that the acquisition workforce could place in solicitations. Builds on the predictability to industry on what they’re going to look for in solicitations when it comes to C-SCRM [cybersecurity supply chain risk management] needs for the government.

Tom Temin: Yeah the software bill of materials (SBOM), I think is a particularly difficult case, because the term only came into popular parlance a year ago or so maybe with the presidential executive order, a little more than a year. I think a lot of people are trying to get their arms around what an SBOM is, what form it takes, and then how you use it in evaluating a vendor or evaluating the product you’re getting. And I think a lot of the COR’s probably aren’t real up on SBOMs either.

Sam Navarro: Correct, and it has a valuable role to play in pre-solicitation, identifying what are the licenses you need, and what types of software you need, but even in post, right, in identifying what intellectual property belongs to the government, in terms of software, and this new AI world that we live in, and what intellectual property still remains with the industry partner. So we’re looking at fleshing that out and seeing how that works for government and industry in the playbook.

Tom Temin: And a final question: You mentioned software licensing. There have arisen lately, especially in academia, questions about the need to make sure that your software licensing and contract language for AI in particular, have the right elements you need because of particular requirements of AI and who has rights to the training data and all of this. Are you aware of that? And is that something you’re working on?

Sam Navarro: At this point in time it’s all on the table. We’re vastly aware of different challenges. I think right now, it’s definitely from a C-SCRM perspective is focusing on security and understanding that the acquisition workforce is part of the team now. It’s no longer a cybersecurity practitioner line of field only. It’s everyone and all hands on deck, including the acquisition workforce, so like Dave Lattimore said, cybersecurity is no longer the CISO’s job. It’s all of our jobs, including the acquisition workforce. And we’re focusing in precisely on those aspects is how can we increase security so that these things don’t come out of the box with vulnerabilities that we have to deal with later on.

The Department of Health and Human Services’ first chief artificial intelligence officer no longer works in the federal government. But he says more agencies and health organizations are following his example and naming their own chief AI officers.

“That’s a good sign we are moving in the right direction,” Oki Mek, the former Chief AI Officer at HHS, now the chief information security officer at Equideum Health, said on the latest episode of All About Data.

In an exit interview looking back on his tenure at HHS, Mek said his former position came into focus after agency officials inventoried more than 100 AI use cases across the department, but weren’t sure how to proceed on many of those projects.

Mek said the department wasn’t sure whether those use cases followed federal mandates, recommendations and executive orders.

Former President Donald Trump signed two executive orders promoting the use of trustworthy AI in the federal government and maintaining American leadership in AI.

“There was no clarity, in terms of, are they following that policy. Is there any governance and compliance? So there was little oversight,” Mek said.

Recent Gartner research estimates about 85% of AI projects will fail, and provide inaccurate outcomes because of bias in the data, algorithms or the teams managing them.

“They’re kind of stuck in proof of concept. That creates a lot of risks, not only from costs and reputation, but because we work a lot in AI and machine learning,” Mek said.

“We wanted to solve those issues and dive into it. That’s why we started looking at the department, and how do we come up with a strategy on AI,” he added.

Mek said data issues stood out as a persistent challenge for AI use cases at HHS — especially trying to understand where the data resides.

“Most of the work in AI and machine learning is run on data, and most of it is really processing the data, getting the data way from either hard copy or scanned documents to make that machine-readable. I think that’s the biggest challenge,” he said.

Mek said his goal was to work with the HHS chief information security officer to develop a policy in which any system that goes through the authority to operate (ATO) process includes a data dictionary.

“My goal was that I wouldn’t want to be in the business of negotiating for data. I would leave that up to the business owner, the process owner, but I would provide a mechanism for them to know where the data resides, where they could search for data and know who to contact to get those data sets. The data thing is definitely a major hurdle to overcome,” he said.

Among the agency’s 100 AI use cases, Mek said using AI for cybersecurity stood out as a particularly promising development.

“Integrating AI into cyber makes sense to me because the process is quite antiquated at the moment to fight the bad actors who are using advanced technology, AI and machine learning against us,” Mek said.

Mek said the Food and Drug Administration has also looked into using AI and machine learning to assist in reviewing and approving food, drugs and medical devices.

HHS named Mek to the CAIO position in March 2021, and he left the position less than a year later.

But during his tenure, Mek oversaw the next steps under the agency’s AI strategy, developed a Trustworthy AI playbook and launched an AI Community of Practice and AI Council.

Mek said the HHS AI strategy served as a valuable roadmap for other organizations just getting started with AI.

“A lot of people are starting from scratch in terms of AI strategy, and how do we align with all the executive orders that are coming down from OMB? I would highly recommend people, even if you’re not in the health industry, to look at the AI strategy, as well as the trustworthy playbook. We’ve been getting a lot of feedback on the materials that we have created,” he said.

Mek said the HHS community of practice helped members share best practices and create awareness around projects. The AI Council, he added, ensured that every HHS division “had a voice in how we carried out the strategy.”

“We didn’t want to do it in silos. We can’t preach that we want to break down silos, but making decisions sitting in headquarters without involving the division leads on AI,” Mek said.

Mek said HHS also held a series of lunch-and-learn programs and invited guests who won the Turing Award, the Nobel Prize equivalent for AI research and computer science.

Best listening experience is on Chrome, Firefox or Safari. Subscribe to Federal Drive’s daily audio interviews on Apple Podcasts or PodcastOne.

Artificial intelligence software isn’t like other software, especially when it comes to acquiring it and licensing it. The data requirements to ensure lack of bias in AI and transparency and how it works are not part of the standard license agreements. This is all the subject of a study by the School of Business at George Mason University.  Study author and senior fellow Benjamin McMartin joined the Federal Drive with Tom Temin  to discuss some of the warnings.

Interview transcript:

Tom Temin: Mr. McMartin, good to have you on.

Benjamin McMartin: Hey, Tom, great to be on.

Tom Temin: So you looked at contracting for artificial intelligence and what are the big differences? It’s just software, but maybe not?

Benjamin McMartin: So at its core, yeah, it’s software and should be easy enough. But there are elements of AI that are particular and actually create some challenges within the acquisition environment. The Center for Government Contracting at George Mason is well as my co-author Maj. Andy Bowne who is the chief counsel of the MIT AI Accelerator in the Air Force, really looked at some of the current challenges that DoD is having in procuring AI software technologies, particularly when it comes to licensing.

Tom Temin: And what are some of the challenges?

Benjamin McMartin: The department and honestly, the federal government are looking at the issue of responsible AI. So how do we look at AI technologies and identify whether there are inherent biases, whether we’re able to explain the results that we get, and so while you may not be able to explain why Spotify has recommended certain songs for you, or why Tinder has sent you on a certain date, in the Department of Defense, we must be able to identify and explain the results that we get from AI software. The results and the impact of the results are much more dire. And so those are the type of issues that we looked at on this paper, which is how do we develop licensing schemes within the current constructs that allows the department to get the type of information that you need to actually explain the results that you get from artificial intelligence?

Tom Temin: Well, isn’t that just embodied in the logic of the AI just as any outcome with software is embodied in its logic?

Benjamin McMartin: So you may be able to get results out of your AI and understand that, hey, I got results based on some algorithm. The question for the department is, can you actually have access to that? Most of these technologies are not being developed within the department. They’re being developed in private industry at very high private expense. And so these are big, upfront investments that companies are making. The department traditionally has looked for licensing rights and technologies that allow them to do a few things and these are no surprise, right? What do I want to do with data rights? I want to make sure I don’t get locked into a vendor, I want to make sure that I have the data that I need to do test and evaluation and sustain systems for a long, long time. But even that level of data rights does not give me the access I need to explain what was the background data? How was this developed? Why am I getting the results that I’m getting based on the background data? These are traditionally not things that are developed and delivered under a traditional DFARS – Defense Federal Acquisition Regulation – supplement data rights license scheme.

Tom Temin: Got it. We’re speaking with Benjamin McMartin, he’s a senior fellow at the George Mason University School of Business’ Center for Government Contracting, and an attorney we should also point out. So what can be done? What is the Air Force and the Navy and the Army that are all pursuing this? What can they do?

Benjamin McMartin: The purpose of the study that we did, again, in partnership with George Mason and the MIT AI accelerator with the Air Force was to create a framework, a practical framework for how acquisition professionals across DoD, and honestly across the federal government, can look at licensing that does two things: One, it gives access to the type of background data that you would need to understand the results that you’re getting from AI solutions. But two, it gives the opportunity to balance. And this is an issue that we kept at the forefront of our paper is, the more data and background data that you ask for from industry, the higher likelihood that folks are not going to want to work with you. And so you have to over communicate what you’re using this data for, what the limits of the use of the data are for, and how those custom licensing structures are going to work. This is a challenge. This is a communication challenge to be able to say to accompany, “I’m going to need your background data. I understand in your commercial practice you don’t give that to anybody, it’s not part of your business model. For DoD’s uses we’re going to need to look at it, but we’re going to procure a license to it, it’ll be limited, and you’ll understand exactly what we can and can’t do with it.” And so in our paper, we’ve provided that framework going through all of the DoD is responsible AI principles, which honestly were developed out of national policy and promoted by the Joint Artificial Intelligence Center. And they’ve done a great job of identifying what those principles are.

Tom Temin: Yeah, so the government is highly aware of this limitation in current licensing. Is there anything in the FAR or the DFAR that can enable this type of licensing request in the first place? Do we need a DFAR update?

Benjamin McMartin: So the nice part about the DFAR is contrary to what a lot of people might say, it’s pretty flexible. It’s got the opportunity, and in fact, it encourages the development and negotiation of specially negotiated license rates. Now, there are some limits. But for example, the joint Artificial Intelligence Center through the Tradewind [other transcation agreement] is finding a lot of success in going outside of DFARS and drafting these custom licensing agreements that are pretty close to what you could get with DFARS but there are some nuance. But within the DFARS licensing scheme, our framework that we’re proposing through our study in our white paper provides you examples of how you can achieve this within the current framework and the DFARS, or under OTAs, which gives you even more flexibility. But ultimately, there are going to be some issues that are going to come up in the future. And we expect these will be the subject matter of future white papers, which is ultimately through machine learning. There is a point where the machine is developing the data. And the current DFAR scheme is based on who has developed the data and who has funded the data. There becomes a point in machine learning where the machine has developed the data. And the current scheme has not been developed to understand how that will work.

Tom Temin: And what about the source code? Because that could be also something required to have full transparency and the audit capability that DoD wants in AI software, can that be part of this mix also?

Benjamin McMartin: Absolutely. So source code, especially when it comes to machine learning models and artificial intelligence is key to understanding how the algorithms have developed, how they’ve modified, how they’ve learned. And ultimately, you need to know what the input data is, and the source code is to understand the outputs that you’re getting. The scheme that we’re proposing, however, through our white paper, is that those should be special licenses put aside, there shouldn’t be a one-license-fits-all for these type of acquisitions. You should sit down and say, okay, source code, this is super important for us for a couple of purposes. And for a limited amount of time, we are going to negotiate a very narrow, very specific license for that piece of it. And then for other stuff, there’ll be larger licenses. Ultimately, companies wants to sell to the Department of Defense. But they want to make sure that they maintain their competitive advantage on the commercial market, and honestly, they want to make sure that they remain a preferred customer for federal agencies as well. And so you really have to get in the weeds on each type of data or software and negotiate those as custom license agreements.

Tom Temin: So the issue then is not what’s in the FAR, the DFAR or the law or regulations. It’s simply a matter of trust, and being able to craft very detailed, one-off or bespoke contract licensing agreements as you adopt AI.

Benjamin McMartin: AI suffers the same challenges as a lot of federal acquisition. And it’s communication. Ultimately, the policy of the regulation is to only negotiate the license rights you need for the purposes you need them for, that policy has never changed. There’s a couple of issues there. One is communication and this need to inform industry that these are the purposes that we’re going to use this for. There are the flexibilities in the law to allow us to do this, the policy demands it. And honestly, the policy benefits both industry and government for that. The second piece, Tom, to that is education. And I am encouraged by congressional actions in the last two [National Defense Authorization Acts] to promote and find money for AI literacy among the acquisition workforce, which is needed because these are not things that folks are going to find on a template. You actually have to sit down and develop these agreements and understand the technology at least to a degree where you can competently advise on license terms.

Tom Temin: Attorney Benjamin McMartin is a senior fellow with the George Mason University School of Business’ Center for Government Contracting. Thanks so much for joining me.

Benjamin McMartin: Thank you very much, Tom.

The Equal Employment Opportunity Commission (EEOC) and the Justice Department issued guidance Thursday outlining ways AI and automated hiring tools can violate the Americans with Disabilities Act (ADA).

EEOC Chairwoman Charlotte Burrows told reporters the guidance applies to employers nationwide, but said the EEOC is working with agencies across the federal government to ensure compliance.

EEOC, she added, is working closely on this guidance with the Labor Department’s Office of Federal Contract Compliance Programs, which ensures federal contractors comply with nondiscrimination laws and regulations.

“Obviously, the various agencies do different things, but [they] have made it a point to be in that dialogue, so that we don’t end up with sort of different focuses, and that we can really lead,” Burrows said.

Burrows said more than 80% of employers are using AI in some form for their work and employment decision-making. The guidance, she said, focuses on ensuring AI tools comply with civil rights laws, “instead of become a high-tech pathway to discrimination.”

“By and large, people are trying to figure this out, and the expertise that you might have in developing these products may not necessarily be the same expertise that you need to look at the civil rights lens. What we want to do is marry that up and be as helpful as we can, as this technology develops,” Burrows said.

The Bureau of Labor Statistics reports about 34% of working-age individuals with a disability are employed, and that the unemployment rate this population is nearly twice as high as the unemployment rate for workers without disabilities.

Assistant Attorney General for Civil Rights Kristen Clarke said the U.S. is “at a critical juncture” with this technology, and that employers are turning to algorithms and AI more frequently to help them select new employees, track performance and determine pay or promotions.

“We are sounding an alarm regarding the dangers tied to blind reliance on AI and other technologies that we are seeing increasingly used by employers. And today, we’re making clear that we must do more to eliminate the barriers faced by people with disabilities,” Clarke said.

Clarke said DOJ has made it a top priority to ensure state and local government employers are not discriminating against job applicants or employees with disabilities.

“Employers may turn to these tools thinking that algorithms or AI can prevent discrimination by removing potential biases that humans may bring to the decision-making process. But in fact, they’re using such technologies in ways that may actually lead to discriminatory hiring decisions,” she said.

Clarke said the guidance aligns with the Biden administration’s focus on promoting diversity, equity, inclusion and accessibility in and out of government.

The EEOC and DOJ guidance documents state an employer may be held responsible for discriminatory actions carried out by software vendors authorized to act on their behalf.

Burrows said employers need to exercise due diligence and ask vendors “what’s under the hood” of these algorithms before using them to vet candidates. Employers, she added, should ask vendors whether any AI algorithm used for hiring allows for reasonable accommodations, a requirement for employees with disabilities under the ADA.

“If the vendor hasn’t thought about that, isn’t ready to engage in that, that should be a warning signal,” Burrows said.

Applicants going through a pre-employment evaluation must also be able to have the opportunity to ask for a reasonable accommodation.

“If everything is automated, it is very difficult for someone with a disability to raise their hand and say, ‘Hey, I need an accommodation.’ Has the vendor thought about at what point could someone do that, and could someone interrupt that sheerly automated process, to have a conversation with a person about the options? Because that accommodations conversation is vitally important,” Burrows said.

Clarke said hiring algorithms in some cases may analyze candidates’ facial movements, speech patterns and words, then compare those behaviors to some its most successful hires.

However, Clarke said those criteria may inadvertently weed out applicants with disabilities from the hiring pool.

“Without enough data to properly assess people with disabilities, like individuals who have speech impairment or autism, people with disabilities can be unfairly locked out or screened out of the applicant pool,” Clarke said.

Burrows said AI discrimination in the workplace isn’t just limited to hiring decisions. In some cases, employers may use productivity algorithms to track how much work an employee is doing, but may not allow for breaks or reasonable accommodations.

Automated hiring tools can also violate the ADA if they make disability-related inquiries or seek information that’s considered at a medical examination before giving the candidate a conditional offer of employment.

The EEOC’s technical assistance document is part of its AI and Algorithmic Fairness Initiative to ensure that the use of AI and software tools using for hiring and employment decisions adhere to federal civil rights laws.

“We are hopeful that the actions that our agencies have taken today help to empower workers and empower people who are seeking to access the job market, we want to ensure that job applicants with disabilities know that they have a right to seek reasonable accommodations,” Clarke said.

House Education and Labor Committee Chairman Bobby Scott (D-Va.) said in a statement that the EEOC and DOJ guidance will help ensure a more equitable job market.

“While algorithmic tools have the potential to address bias in the hiring process, the research is clear that — without the appropriate safeguards in place — artificial intelligence and other automated technologies can further entrench discrimination against workers, including workers with disabilities,” Scott said.