Back to news

April 20, 2018 | International, C4ISR, Security

DARPA official: To build trust in AI, machines must explain themselves

By:

Artificially intelligent systems must be able to explain themselves to operators if they are to be trusted, according to an expert from the Defense Advanced Research Agency, who voiced concern that methods used by current AI systems are often masked by mysterious algorithms.

“A lot of the machine learning algorithms we're using today, I would tell you ‘good luck,” Fred Kennedy, the director of DARPA's Tactical Technology Office during a panel at Navy League's Sea-Air-Space on April 10. “We have no idea why they know the difference between a cat and a baboon.”

“If you start diving down into the neural net that's controlling it,” Kennedy continued, “you quickly discover that the features these algorithms are picking out have very little to do with how humans identify things.”

Kennedy's comments were in response to Deputy Assistant Secretary of the Navy for Unmanned Systems Frank Kelley, who described the leap of faith operators must make when dealing with artificially intelligent systems.

“You're throwing a master switch on and just praying to God that [Naval Research Laboratory] and John's Hopkins knew what the hell that they were doing,” Kelley said of the process.

The key to building trust, according to Kennedy, lies with the machines.

“The system has to tell us what it's thinking,” Dr. Kennedy said. “That's where the trust gets built. That's how we start to use and understand them.”

DARPA's Explainable Artificial Intelligence program seeks to teach AI how to do just that. The program envisions systems that will have the ability to explain the rationale behind their decisions, characterize their strengths and weaknesses, and describe how they will behave in the future. Such capabilities are designed to improve teamwork between man and machine by encouraging warfighters to trust artificially intelligent systems.

“It's always going to be about human-unmanned teaming,” said Kennedy. “There is no doubt about that.”

https://www.defensenews.com/home/2018/04/10/darpa-official-to-build-trust-in-ai-machines-must-explain-themselves/

On the same subject

  • Earthcube, la start-up française qui veut marcher dans les pas de Palantir

    February 13, 2019 | International, C4ISR

    Earthcube, la start-up française qui veut marcher dans les pas de Palantir

    ANNE DRIF Fondé il y a deux ans par deux anciens d'Areva et Airbus, Earthcube travaille pour quatre « organisations » du ministère des Armées et un service britannique. L'un montait des joint-ventures en Chine, en Russie et en Corée chez Areva. L'autre élaborait de nouvelles technologies d'observation de la terre chez Airbus Defense & Space. A 37 et 34 ans, Arnaud Guérin et Renaud Allioux sont aujourd'hui à la tête d'Earthcube, la première start-up devenue, en l'espace de deux ans, le prestataire de quatre « organisations » au sein du ministère de la Défense, dont ses services de renseignement militaire. Et d'un service britannique. Identifier un pick-up dans le désert Faute d'écho à ses solutions de Big Data au sein de l'avionneur, et Space X ayant écarté sa candidature, les projets de Renaud Allioux, ont rencontré ceux d'Arnaud Guérin, qui s'intéressait lui aussi aux systèmes de surveillance stratégiques. Ensembles, ils ont lancé une solution d'intelligence artificielle qui permet d'identifier sur des images satellites prises à 600 kilomètres d'altitude, des objets de quelques dizaines de pixels en quelques secondes, comme un pick-up en plein désert, ou de suivre des dizaines de milliers de véhicules, dans de grands centres urbains, comme Deir Ezzor aux mains de l'EI. « Aujourd'hui, il faut plusieurs jours à un analyste de renseignement pour mener ce type de ciblage sur des images satellites. Avec l'arrivée des satellites espions européens CSO, qui fourniront un grand volume de données, les équipes d'analystes pourront, en l'état des outils actuels, traiter moins de 10 % des flux, explique Arnaud Guérin. Or, la rapidité d'intervention est clef dans ces missions ». Expansion américaine Rapidement identifiée par la Direction générale des Armées, Earthcube a scellé son alliance avec la Direction du renseignement militaire lors du premier défi de l' Intelligence Campus , son pôle dédié aux nouvelles technologies. Mais celle-ci n'est pas exclusive, puisque la start-up a pu se rapprocher d'autres services de renseignement. En 2017, la société a levé 3 millions d'euros. Au départ, pourtant, ils ambitionnaient de s'adresser seulement aux acteurs privés ayant de forts enjeux sécuritaires comme les groupes pétroliers ou nucléaires. « Chez Areva, je faisais mettre en place des systèmes de surveillance au sol pour des sites miniers ou chimiques, poursuit Arnaud Guérin. Mais, quand vous êtes le géant chinois Cnooc, vous devez sécuriser des dizaines de milliers de kilomètres de pipeline. Ces systèmes de surveillance sont vulnérables et onéreux ». Au lancement d'Earthcube en 2016, ajoute-t-il, « nous nous étions positionnés comme l'alternative non américaine d'analyses de données, mais c'est une illusion de croire que les services de renseignements américains ont une réelle longueur d'avance sur les Européens en la matière », poursuit le dirigeant. Earthcube est ainsi « en discussions avancées » avec des organisations américaines. Les deux fondateurs ont de sérieuses ambitions, quitte à prendre une référence qui fait polémique en France. « Pour nous, l'américain Palantir est un modèle. En très peu de temps, ils ont réussi à devenir l'égal de géants traditionnels de leur secteur ». Anne Drif https://www.lesechos.fr/industrie-services/air-defense/0600634357107-earthcube-la-start-up-francaise-qui-veut-marcher-dans-les-pas-de-palantir-2244076.php

  • Europe’s next-gen fighter club faces a dilemma: Who else can join?

    August 17, 2020 | International, Aerospace

    Europe’s next-gen fighter club faces a dilemma: Who else can join?

    By: Sebastian Sprenger COLOGNE, Germany — With mainland Europe and the United Kingdom pursuing their own sixth-generation fighter programs, differing views have started to emerge about how many nations should partake in the action. At issue is whether the British Tempest and the German-French-Spanish Future Combat Air System can coexist in the long run without cannibalizing the continent's defense budget. What's more, while leaders in the U.K. have openly advertised their appetite to pick up partner countries for Tempest, Paris and Berlin are divided about admitting additional members, besides Spain, for fear of slowing down their effort. German defense officials early this year told Bundestag lawmakers they consider it risky to keep the circle of participants too small because team Tempest could go around snagging up contributors, though that concern has yet to bear out. To Dirk Hoke, the CEO of Airbus Defence and Space, the issue comes down to the budget. His company is the co-lead for the FCAS program along with France's Dassault Aviation. Notably, the French have the lead for the Next-Generation Fighter, envisioned as the central aircraft for the larger program of supporting drones and command-and-control equipment. “Europe can't afford two new systems,” Hoke said in a virtual panel discussion organized by a German defense industry lobbying association. That is especially the case, he argued, because a competition between the U.K. and members of the European Union would reinforce the perception that Britain's divorce from the bloc has weakened the intra-continental defense alliance — a notion that leaders on both sides of the Channel have been trying to dispel. Finding a way to merge FCAS and Tempest should become a top priority for decision-makers once a Brexit agreement on future trade and defense relations is in the bag, according to Hoke. Until then, he said, “we simply have to be patient.” In order to be prepared for “an opportunity to negotiate at eye level,” both projects should continue to work through their technology development so that eventual touchpoints for cooperation are already far along, he explained. In contrast, Dassault Aviation CEO Eric Trappier has urged caution when it comes to broadening the FCAS circle too soon. “I don't know if there won't be some new partners in the future,” he said during a company earnings call late last month. “We must not exclude them.” Noting that Spain already was admitted to the program after Germany and France had kicked it off, Trappier said the companies involved were still learning to work together. “We have to get to know each other and share our work together,” he said. “If we change partners every six months, I can tell you that we will not reach 2040,” he added, referring to the envisioned in-service date for the futuristic weapon. The next big milestone for the program is fielding a demonstrator aircraft in 2026. https://www.defensenews.com/global/europe/2020/08/17/europes-next-gen-fighter-club-faces-a-dilemma-who-else-can-join

  • AFA 2019: US Air Force will maintain F-35 buy as it pursues Digital Century Series

    September 19, 2019 | International, Aerospace

    AFA 2019: US Air Force will maintain F-35 buy as it pursues Digital Century Series

    Key Points The US Air Force will maintain its current F-35A buy as it starts to experiment with its new approach to acquisition, called the Digital Century Series The F-35A will be fielded through approximately 2040, so a retired pilot believes the USAF must act fast if it wants to avoid a gap between fighters The US Air Force (USAF) will maintain its Lockheed Martin F-35A Lightning II Joint Strike Fighter (JSF) programme of record of 1,763 aircraft as it pursues its futuristic Digital Century Series (DCS) next-generation acquisition effort, according to the air force's acquisition boss. Will Roper, assistant secretary of the USAF for acquisition, technology, and logistics (AT&L), told reporters on 16 September that the DCS will be a software-focused approach to acquisition. It will enable the USAF to begin exploring what Roper calls the digital trinity of agile software development: agile software, open architecture (OA) systems, and digital engineering; and what it can provide the air force as it develops a futuristic and connected kill web. Next Generation Air Dominance (NGAD) will be one of the first USAF efforts to experiment with the DCS. The output of this effort, Roper said, will determine the acquisition strategy for NGAD and if it will fit with the funding profile that it has. The USAF requested USD1 billion in fiscal year 2020 (FY 2020) for NGAD, the concept the USAF is pursuing to prepare for warfare against near-peer nations such as China or Russia. Roper said he is going to announce a team on 1 October that will figure out how to run a programme based on those three attributes. He said it will be similar to how the USAF put together its Kessel Run software development hub. https://www.janes.com/article/91350/afa-2019-us-air-force-will-maintain-f-35-buy-as-it-pursues-digital-century-series?from_rss=1

All news