20 avril 2018 | International, C4ISR, Sécurité

DARPA official: To build trust in AI, machines must explain themselves

By:

Artificially intelligent systems must be able to explain themselves to operators if they are to be trusted, according to an expert from the Defense Advanced Research Agency, who voiced concern that methods used by current AI systems are often masked by mysterious algorithms.

“A lot of the machine learning algorithms we're using today, I would tell you ‘good luck,” Fred Kennedy, the director of DARPA's Tactical Technology Office during a panel at Navy League's Sea-Air-Space on April 10. “We have no idea why they know the difference between a cat and a baboon.”

“If you start diving down into the neural net that's controlling it,” Kennedy continued, “you quickly discover that the features these algorithms are picking out have very little to do with how humans identify things.”

Kennedy's comments were in response to Deputy Assistant Secretary of the Navy for Unmanned Systems Frank Kelley, who described the leap of faith operators must make when dealing with artificially intelligent systems.

“You're throwing a master switch on and just praying to God that [Naval Research Laboratory] and John's Hopkins knew what the hell that they were doing,” Kelley said of the process.

The key to building trust, according to Kennedy, lies with the machines.

“The system has to tell us what it's thinking,” Dr. Kennedy said. “That's where the trust gets built. That's how we start to use and understand them.”

DARPA's Explainable Artificial Intelligence program seeks to teach AI how to do just that. The program envisions systems that will have the ability to explain the rationale behind their decisions, characterize their strengths and weaknesses, and describe how they will behave in the future. Such capabilities are designed to improve teamwork between man and machine by encouraging warfighters to trust artificially intelligent systems.

“It's always going to be about human-unmanned teaming,” said Kennedy. “There is no doubt about that.”

https://www.defensenews.com/home/2018/04/10/darpa-official-to-build-trust-in-ai-machines-must-explain-themselves/

Sur le même sujet

  • Netherlands, Denmark commit to delivering F-16s to Ukraine | Reuters

    20 août 2023 | International, Aérospatial

    Netherlands, Denmark commit to delivering F-16s to Ukraine | Reuters

    The Netherlands and Denmark will supply F-16 fighter jets to Ukraine with first deliveries due around New Year, the prime ministers of the two NATO countries announced on Sunday during visits by Ukrainian President Volodomyr Zelenskiy.

  • Albania receives two Black Hawk helicopters

    15 janvier 2024 | International, Aérospatial

    Albania receives two Black Hawk helicopters

    A pair of US Army surplus Sikorsky UH-60A Black Hawk helicopters were delivered to Albania on 13 January, Albanian Minister of Defence Niko Peleshi disclosed on the same day. A file photo of a UH-60A Black Hawk, of the type that Albania has ...

  • 130 House members want 24 percent more F-35s procured in FY21

    23 mars 2020 | International, Aérospatial

    130 House members want 24 percent more F-35s procured in FY21

    By: Aaron Mehta WASHINGTON — One hundred thirty members of the House of Representatives are asking key defense committees in Congress to increase the number of F-35 Joint Strike Fighters by 24 percent over the number requested by the Pentagon in fiscal 2021. “Our adversaries continue to advance surface-to-air missile systems and develop their own stealth fighters,” read the letter, released Wednesday. “It is essential that we continue to increase production of our nation's only 5th generation stealth fighter in order to ensure the United States maintains air dominance and to further reduce overall program costs.” The letter, addressed to the chairs of the Senate and House Armed Services committees and Appropriations Defense subcommittees, is authored by Reps. John Larson, D-Conn.; Marc. Veasey, D-Texas; Martha Roby, R-Ala.; and Michael Turner, R-Ohio — the four leaders of the bipartisan F-35 caucus. Last year, the four also joined forces to write a similar request, which garnered 103 signatories. The Defense Department's budget request asks for 79 F-35s, including 48 of the F-35A model used by the Air Force, 10 F-35Bs used by the Marines and 21 F-35C models used by the Navy. In the letter, the congressmen note that number is 19 less jets than Congress appropriated in FY20. However, that number creates “a capability gap that 4th Generation, or legacy, aircraft cannot fulfill,” the letter warned. “To reach the minimum 50% ratio of 5th Generation and 4th Generation fighters in the timeframe required to meet the threat, the U.S. must acquire F-35s in much larger quantities.” Instead, the members want a 24 percent increase in fighters procurement, going up to 98 total, including 12 more F-35As, two more F-35Bs and 26 more F-35Cs. Those numbers match the fighter increase listed by the Air Force in its unfunded requirements document sent to Congress earlier this year; the Navy requested only five more F-35C variants, while the Marines did not request more. The letter was first reported by Politico. In addition to the increase in planes bought, the members are seeking additional funding for “spare parts and depot level repair capability to meet the required availability rates and accelerate the stand-up of mandated, organic government repair capabilities.” Additionally, investments are sought for the program dedicated to the jet's reliability, maintainability and improvement, as well as a “long-term, outcome-based sustainment contract” that would guarantee performance metrics at a fixed price. The members then request the committees fully fund the budget request for the continuous capability development and delivery (C2D2) modernization effort and use existing funds to accelerate integration of the Joint Air-to-Surface Standoff Missile onto the jet. Earlier this year, the Pentagon's independent weapons tester called the current schedule for C2D2 “high risk” and said the program office is struggling to stay on schedule. “C2D2 is critical to meeting the evolving threat in the mid-2020s and into the 2030s. Full funding is needed for the delivery of new weapons and critical capabilities necessary to keep the F-35 ahead of our adversaries,” the members wrote. https://www.defensenews.com/congress/2020/03/19/130-house-members-want-24-percent-more-f-35s-procured-in-fy21/

Toutes les nouvelles