Back to news

June 22, 2020 | International, C4ISR

Artificial intelligence systems need ‘checks and balances’ throughout development

The Pentagon's primary artificial intelligence hub is already studying how to aim a laser at the correct spot on an enemy vehicle, pinpointing which area to target to inflict the most damage, and identifying the most important messages headed to commanders, officials said June 16.

But as part of that work, the Department of Defense needs to carefully implement checks and balances into the development process, experts urged June 16.

“Fundamentally I would say there's a requirement ... that there's going to be a mixture of measures taken to ensure the governability of the system from the first stage of the design of the system all the way up through the operations of the system in a combat scenario,” said Greg Allen, the Joint Artificial Intelligence Center's chief of strategy and communications at the Joint Artificial Intelligence Center, at the Defense One Tech Summit June 16.

The JAIC is working on several lethality projects through its new joint warfighting initiative, boosted by a new contract award to Booz Allen potentially worth $800 million.

“With this new contract vehicle, we have the potential to do even more this next year than we did in the past,” Allen said.

Meanwhile, the Army's Artificial Intelligence Task Force is working on an advanced threat recognition project. DARPA is exploring complementing AI systems that would identify available combat support assets and quickly plan their route to the area.

Throughout all of the development work, experts from the military and from academia stressed that human involvement and experimentation was critical to ensuring that artificial intelligence assets are trustworthy. The department has released a document of five artificial intelligence ethical principles, but the challenge remains implementing those principles into projects across a department with disparate services working on separate artificial intelligence projects.

“We want safe, reliable and robust systems deployed to our warfighters,” said Heather Roff, senior research analyst at the Johns Hopkins Applied Physics Lab. “We want to be able to trust those systems. We want to have some sort of measure of predictability even if those systems act unpredictably.”

Brig. Gen. Matt Easley, director of the artificial intelligence task force at Army Futures Command, said the service is grappling with those exact challenges, trying to understand how the service can insert “checks and balances” as it trains systems and soldiers.

Easley added that the unmanned systems under development by the Army will have to be adaptable to different environments, such as an urban or desert scenarios. In order to ensure that the systems and soldiers are ready for those scenarios, the Army has to complete a series of tests, just like the autonomous vehicle industry.

“We don't think these systems are going to be 100 percent capable right out of the box,” Easley said on the webinar. “If you look at a lot of the evolution of the self-driving cars throughout our society today, they're doing a lot of experimentation. They're doing lots of testing, lots of learning every day. We in the Army have to learn how to go from doing one to two to three vehicle experiments to have many experiments going on every day across all our camp posts and stations.”

Increasingly autonomous systems also mean that there needs to a cultural shift in among all levels of military personnel who will need to better understand how artificial intelligence is used. Roff said that operators, commanders and judge advocate generals will need to better understand how systems are supposed “to ensure that the human responsibility and governability is there.”

“We need to make sure that we have training, tactics, procedures, as well as policies, ensuring where we know the human decision maker is,” Roff said.

https://www.c4isrnet.com/it-networks/2020/06/18/artificial-intelligence-systems-need-checks-and-balances-throughout-development/

On the same subject

  • The Army has outlined a technology wish list

    December 24, 2018 | International, C4ISR

    The Army has outlined a technology wish list

    By: Mark Pomerleau The Army is looking to expand industry partners and technologies in the next five years to inform its pilot multidomain task force. The task force will test new concepts the Army believes will be needed to fight future near-peer adversaries in areas such as long-range precision fires, cyber and electronic warfare. In a notice to industry Dec. 18, Army Pacific, which is running the pilot, is in the process of assessing the industrial base for state-of-the-art technologies that can enable the pilot. In the run up to joint exercises in the 2018-2021 time frame, Army Pacific wants to hear from industry how its solutions can provide enhanced or new capabilities to the multidomain experimentation plan. Specific solutions named in the notice include: Integrated joint/partner fires targeting (lethal and non-lethal); Long range cross-domain fires; Land-based anti-ship missile capabilities; Long range strike capabilities from air, surface and maritime platforms; Multidomain artillery; Integrated intelligence; Cyber and Electromagnetic Activities; Space effects at operational and tactical levels; Organic over-the-horizon surveillance and targeting (including elevated sensors and unmanned aerial systems); Integrated and layered (short, medium and long range) air and missile defense capabilities and low-cost effectors; Mobile and survivable sensors (active, passive, seismic, etc.); Camouflage, concealment and deception across multi-spectral; visual, infrared, cyber, active radio frequency, etc.; Multidomain common operating picture; Survivable UAS (point launch/recovery, low observable) with modular/multi-payload options: ISR, EW, SIGINT, Kinetic); Advanced logistics and sustainment; Assured Communications, PNT, on and offshore datalink transmission; Manned-Unmanned Teaming; and Development of agile leaders (human dimension). https://www.c4isrnet.com/c2-comms/2018/12/21/the-army-has-outlined-a-technology-wish-list

  • March 30, 2021 | International, Aerospace

    Rotorcraft Industry Trying to Regain Footing

  • IDV signs contract with the Italian Army to supply 1,453 tactical-logistic trucks

    December 22, 2024 | International, Land

    IDV signs contract with the Italian Army to supply 1,453 tactical-logistic trucks

    The contract is for the supply of a variety of military logistic platforms based on the new IDV truck range and equipped with tactical cabins. The value of the overall contract...

All news