Back to news

June 22, 2020 | International, C4ISR

Artificial intelligence systems need ‘checks and balances’ throughout development

The Pentagon's primary artificial intelligence hub is already studying how to aim a laser at the correct spot on an enemy vehicle, pinpointing which area to target to inflict the most damage, and identifying the most important messages headed to commanders, officials said June 16.

But as part of that work, the Department of Defense needs to carefully implement checks and balances into the development process, experts urged June 16.

“Fundamentally I would say there's a requirement ... that there's going to be a mixture of measures taken to ensure the governability of the system from the first stage of the design of the system all the way up through the operations of the system in a combat scenario,” said Greg Allen, the Joint Artificial Intelligence Center's chief of strategy and communications at the Joint Artificial Intelligence Center, at the Defense One Tech Summit June 16.

The JAIC is working on several lethality projects through its new joint warfighting initiative, boosted by a new contract award to Booz Allen potentially worth $800 million.

“With this new contract vehicle, we have the potential to do even more this next year than we did in the past,” Allen said.

Meanwhile, the Army's Artificial Intelligence Task Force is working on an advanced threat recognition project. DARPA is exploring complementing AI systems that would identify available combat support assets and quickly plan their route to the area.

Throughout all of the development work, experts from the military and from academia stressed that human involvement and experimentation was critical to ensuring that artificial intelligence assets are trustworthy. The department has released a document of five artificial intelligence ethical principles, but the challenge remains implementing those principles into projects across a department with disparate services working on separate artificial intelligence projects.

“We want safe, reliable and robust systems deployed to our warfighters,” said Heather Roff, senior research analyst at the Johns Hopkins Applied Physics Lab. “We want to be able to trust those systems. We want to have some sort of measure of predictability even if those systems act unpredictably.”

Brig. Gen. Matt Easley, director of the artificial intelligence task force at Army Futures Command, said the service is grappling with those exact challenges, trying to understand how the service can insert “checks and balances” as it trains systems and soldiers.

Easley added that the unmanned systems under development by the Army will have to be adaptable to different environments, such as an urban or desert scenarios. In order to ensure that the systems and soldiers are ready for those scenarios, the Army has to complete a series of tests, just like the autonomous vehicle industry.

“We don't think these systems are going to be 100 percent capable right out of the box,” Easley said on the webinar. “If you look at a lot of the evolution of the self-driving cars throughout our society today, they're doing a lot of experimentation. They're doing lots of testing, lots of learning every day. We in the Army have to learn how to go from doing one to two to three vehicle experiments to have many experiments going on every day across all our camp posts and stations.”

Increasingly autonomous systems also mean that there needs to a cultural shift in among all levels of military personnel who will need to better understand how artificial intelligence is used. Roff said that operators, commanders and judge advocate generals will need to better understand how systems are supposed “to ensure that the human responsibility and governability is there.”

“We need to make sure that we have training, tactics, procedures, as well as policies, ensuring where we know the human decision maker is,” Roff said.

https://www.c4isrnet.com/it-networks/2020/06/18/artificial-intelligence-systems-need-checks-and-balances-throughout-development/

On the same subject

  • How the Office of Naval Research hopes to revolutionize manufacturing

    October 16, 2018 | International, Naval

    How the Office of Naval Research hopes to revolutionize manufacturing

    By: Daniel Cebul WASHINGTON — The Office of Naval Research awarded Lockheed Martin Oct. 1 a two-year, $5.8 million contract to explore how machine learning and artificial intelligence can make complex 3-D printing more reliable and save hours of tedious post-production inspections. In today's factories, 3-D printing parts requires persistent monitoring by specialists to ensure intricate parts are produced without impurities and imperfections that can compromise the integrity of the part overall. To improve this laborious process, the Navy is tasking Lockheed Martin with developing multi-axis robots that use lasers to deposit material and oversee the printing of parts. Lockheed Martin has multiple partners on the contract including Carnegie Mellon University, Iowa State University, Colorado School of Mines, America Makes, GKN and Wolf Robotics and Oak Ridge National Laboratory. The contract covers what Glynn Adams, a senior engineer with Lockheed Martin, describes as the pre-flight model of the program's development. Initial work will focus on developing computer models that can predict the microstructures and mechanical properties of 3-D printed materials to generate simulation data to train with. Adams said the Carnegie Mellon team will look at variables such as, “the spot size of the laser beam, the rate of feed of the titanium wire [and]the total amount energy density input into the material while it is being manufactured.” This information helps the team predict the microstructure, or organizational structure of a material on a very small scale, that influences the physical properties of the additive manufactured part. This data will then be shared with Iowa State, who will plug the information into a model that predicts the mechanical properties of the printed component. By taking temperature and spot size measurements, the team can also ensure they are, “accurately controlling energy density, the power of both the laser and the hot wire that goes into the process,” Adams said.. “All of that is happening before you actually try to do any kind of machine learning or artificial neural networks with the robot itself. That's just to try to train the models to the point where we have confidence in the models,” Adams said. Sounds easy, right? But one key problem could come in cleaning up the data and removing excess noise from the measurements. “Thermal measurements are pretty easy and not data intensive, but when you start looking at optical measurements you can collect just an enormous amount of data that is difficult to manage,” Adams explained. Lockheed Martin wants to learn how shrink the size of that dataset without sacrificing key parameters. The Colorado School of Mines and America Makes will tackle the problem of compressing and manipulating this data to extract the key information needed to train the algorithms. After this work has been completed, the algorithms then will be sent to Oak Ridge National Laboratory, where robots will begin producing 3-D titanium parts and learn how to reliably construct geometrically and structurally sound parts. This portion of the program will confront challenges from the additive manufacturing and AI components of the project. On the additive manufacturing side, the team will work with new manufacturing process, “trying to understand exactly what the primary, secondary and tertiary interactions are between all those different process parameters,” Adams said. “If you think about it, as you are building the part depending on the geometric complexity, now those interactions change based on the path the robot has to take to manufacture that part. One of the biggest challenges is going to be to understand exactly which of those parameters are the primary, which are the tertiary and to what level of control we need to be able to manipulate or control those process parameters in order to generate the confidence in the parts that we want.” At the same time, researchers also will tackle AI machine learning challenges. Like with other AI programs, it's crucial the algorithm is learning the right information, the right way. The models will give the algorithms a good starting point, but Adams said this will be an iterative process that depends on the algorithm's ability to self-correct. “At some point, there are some inaccuracies that could come into that model,” Adams explained. “So now, the system itself has to understand it may be getting into a regime that is not going to produce the mechanical properties or microstructures that you want, and be able to self-correct to make certain that instead of going into that regime it goes into a regime that produces the geometric part that you want.” With a complete algorithm that can be trusted to produce structurally sound 3-D printed parts, time-consuming post-production inspections will become a thing of the past. Instead of nondestructive inspections and evaluations, if you “have enough control on the process, enough in situ measurements, enough models to show that that process and the robot performed exactly as you thought it would, and produced a part that you know what its capabilities are going to be, you can immediately deploy that part,” said Adams. “That's the end game, that's what we're trying to get to, is to build the quality into the part instead of inspecting it in afterwards." Confidence in 3-D printed parts could have dramatic consequences for soldiers are across the services. As opposed to waiting for replacement parts, service members could readily search a database of components, find the part they need and have a replacement they can trust in hours rather than days or weeks. “When you can trust a robotic system to make a quality part, that opens the door to who can build usable parts and where you build them,” said Zach Loftus, Lockheed Martin Fellow for additive manufacturing. “Think about sustainment and how a maintainer can print a replacement part at sea, or a mechanic print a replacement part for a truck deep in the desert. This takes 3-D printing to the next, big step of deployment.” https://www.c4isrnet.com/industry/2018/10/15/how-the-office-of-naval-research-hopes-to-revolutionize-manufacturing

  • Australia, Naval Group conclude sub negotiations

    December 17, 2018 | International, Naval

    Australia, Naval Group conclude sub negotiations

    By: Nigel Pittaway MELBOURNE, Australia – Australian Defence Minister Christopher Pyne confirmed that the Australian government has finally concluded negotiations for the formal signing of a strategic partnering agreement for 12 large conventionally-powered attack submarines from Naval Group. Australia is acquiring the vessels under its $50 billion (U.S. $36.12 billion) Project Sea 1000 (Future Submarine) to replace its existing fleet of six Collins Submarines from the early 2030s. The subs will be the ‘Attack' class with the lead vessel named HMAS Attack. They will be fabricated in Australia to a design previously known as the Shortfin Barracuda 1A. Recent local media reports have suggested that negotiations between the parties had stalled, placing the government's timeline for the Collins replacement in jeopardy, but Pyne said on Thursday the program was still on track. “There's been a lot of ill-informed mythmaking around the negotiations but I'm very happy to say today the negotiations are complete,” Pyne said during sod-turning event at the site of the Future Submarine Construction Yard at Osborne in South Australia. “The strategic planning agreement will be signed in February next year and we can continue to get on with the submarine project, which has been under the design and mobilization contract for the last two years.” Declining to provide details of the intricacies of the agreement due to their commercial nature, Pyne said the negotiations were officially concluded at an Australian Government National Security Committee meeting in Melbourne on Dec. 10. “Suffice to say the Australian government's interests, the Australian taxpayer's interests, have been taken care of,” he said. “Naval Group Australia will deliver 12 regionally-superior submarines on time and on budget.” Australia's Chief of Navy, Vice Adm. Mark Noonan, also denied reports of an emerging capability gap between the retirement of the first Collins submarines and the Attack boats entering service, which some analysts have suggested might require a ‘Plan B' to be formulated. “I don't believe that's the case,” he told reporters. “We've got a very solid plan to ensure that there is no gap in our nation's submarine capability, and there is a very advanced plan that will see a number of our current Collins class submarines going through a life of type extension program, which will ensure that capability gap doesn't exist.” https://www.defensenews.com/global/asia-pacific/2018/12/14/australia-naval-group-conclude-sub-negotiations/

  • Air defences down 16 airborne targets in Russia's Belgorod, local governor says
All news