Retour aux nouvelles

20 avril 2018 | International, C4ISR, Sécurité

DARPA official: To build trust in AI, machines must explain themselves

By:

Artificially intelligent systems must be able to explain themselves to operators if they are to be trusted, according to an expert from the Defense Advanced Research Agency, who voiced concern that methods used by current AI systems are often masked by mysterious algorithms.

“A lot of the machine learning algorithms we're using today, I would tell you ‘good luck,” Fred Kennedy, the director of DARPA's Tactical Technology Office during a panel at Navy League's Sea-Air-Space on April 10. “We have no idea why they know the difference between a cat and a baboon.”

“If you start diving down into the neural net that's controlling it,” Kennedy continued, “you quickly discover that the features these algorithms are picking out have very little to do with how humans identify things.”

Kennedy's comments were in response to Deputy Assistant Secretary of the Navy for Unmanned Systems Frank Kelley, who described the leap of faith operators must make when dealing with artificially intelligent systems.

“You're throwing a master switch on and just praying to God that [Naval Research Laboratory] and John's Hopkins knew what the hell that they were doing,” Kelley said of the process.

The key to building trust, according to Kennedy, lies with the machines.

“The system has to tell us what it's thinking,” Dr. Kennedy said. “That's where the trust gets built. That's how we start to use and understand them.”

DARPA's Explainable Artificial Intelligence program seeks to teach AI how to do just that. The program envisions systems that will have the ability to explain the rationale behind their decisions, characterize their strengths and weaknesses, and describe how they will behave in the future. Such capabilities are designed to improve teamwork between man and machine by encouraging warfighters to trust artificially intelligent systems.

“It's always going to be about human-unmanned teaming,” said Kennedy. “There is no doubt about that.”

https://www.defensenews.com/home/2018/04/10/darpa-official-to-build-trust-in-ai-machines-must-explain-themselves/

Sur le même sujet

  • DARPA Wants to Make Underground Maps on the Fly

    16 août 2018 | International, C4ISR

    DARPA Wants to Make Underground Maps on the Fly

    The agency is challenging teams to build systems that chart caves, tunnels and underground urban infrastructure. Finding your way through caves and tunnels is both difficult and extremely dangerous, but the Pentagon's research office wants to build technology that can navigate underground environments while humans stay on the surface. The Defense Advanced Research Projects Agency is finalizing teams for its Subterranean Challenge, or SubT, a three-year competition to build systems that can rapidly map and search often treacherous underground areas. The agency on Thursday awarded a $4.5 million contract to Virginia-based iRobot Defense Holdings and a $750,000 contract to Michigan Technological University to participate in the challenge. A third team, Scientific Systems Company Inc., joined the program on July 31 with a $492,000 contract. “Even under ideal conditions, these complex environments present significant challenges for subterranean situational awareness,” DARPA wrote in the program announcement. “However, in time-sensitive scenarios, whether in active combat operations or disaster response settings, warfighters and first responders alike are faced with a range of increased technical challenges, including difficult and dynamic terrains ... severe communication constraints, and expansive areas of operation.” Full Article: https://www.defenseone.com/technology/2018/08/darpa-wants-make-underground-maps-fly/150554/

  • Romania opens F-16 pilot training hub for NATO allies, Ukraine | Reuters

    13 novembre 2023 | International, Aérospatial

    Romania opens F-16 pilot training hub for NATO allies, Ukraine | Reuters

    Red is a Romanian fighter pilot with hundreds of flight hours and countless air policing missions on the NATO state's now retired fleet of MIG21 LanceR jets who will be flying Lockheed Martin F-16 planes as early as December.

  • Navy surface drone fires its first missile during Middle East exercise

    8 novembre 2023 | International, Naval

    Navy surface drone fires its first missile during Middle East exercise

    The mini missile fire was part of an exercise by a unit working to develop the Navy's drone and artificial intelligence technologies.

Toutes les nouvelles