7 février 2019 | International, C4ISR

DARPA: Defending Against Adversarial Artificial Intelligence

Today, machine learning (ML) is coming into its own, ready to serve mankind in a diverse array of applications – from highly efficient manufacturing, medicine and massive information analysis to self-driving transportation, and beyond. However, if misapplied, misused or subverted, ML holds the potential for great harm – this is the double-edged sword of machine learning.

“Over the last decade, researchers have focused on realizing practical ML capable of accomplishing real-world tasks and making them more efficient,” said Dr. Hava Siegelmann, program manager in DARPA's Information Innovation Office (I2O). “We're already benefitting from that work, and rapidly incorporating ML into a number of enterprises. But, in a very real way, we've rushed ahead, paying little attention to vulnerabilities inherent in ML platforms – particularly in terms of altering, corrupting or deceiving these systems.”

In a commonly cited example, ML used by a self-driving car was tricked by visual alterations to a stop sign. While a human viewing the altered sign would have no difficulty interpreting its meaning, the ML erroneously interpreted the stop sign as a 45 mph speed limit posting. In a real-world attack like this, the self-driving car would accelerate through the stop sign, potentially causing a disastrous outcome. This is just one of many recently discovered attacks applicable to virtually any ML application.

To get ahead of this acute safety challenge, DARPA created the Guaranteeing AI Robustness against Deception (GARD) program. GARD aims to develop a new generation of defenses against adversarial deception attacks on ML models. Current defense efforts were designed to protect against specific, pre-defined adversarial attacks and, remained vulnerable to attacks outside their design parameters when tested. GARD seeks to approach ML defense differently – by developing broad-based defenses that address the numerous possible attacks in a given scenario.

“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure. The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” stated Siegelmann.

GARD's novel response to adversarial AI will focus on three main objectives: 1) the development of theoretical foundations for defensible ML and a lexicon of new defense mechanisms based on them; 2) the creation and testing of defensible systems in a diverse range of settings; and 3) the construction of a new testbed for characterizing ML defensibility relative to threat scenarios. Through these interdependent program elements, GARD aims to create deception-resistant ML technologies with stringent criteria for evaluating their robustness.

GARD will explore many research directions for potential defenses, including biology. “The kind of broad scenario-based defense we're looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” said Siegelmann.

GARD will work on addressing present needs, but is keeping future challenges in mind as well. The program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.

A Proposers Day will be held on February 6, 2019, from 9:00 AM to 2:00 PM (EST) at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203 to provide greater detail about the GARD program's technical goals and challenges.

Additional information will be available in the forthcoming Broad Agency Announcement, which will be posted to www.fbo.gov.

https://www.darpa.mil/news-events/2019-02-06

Sur le même sujet

  • Homeland defense interceptor defeats ballistic missile in test

    11 décembre 2023 | International, Terrestre

    Homeland defense interceptor defeats ballistic missile in test

    Missile Defense Agency sees successful takedown using an upgraded Ground-Based Interceptor for homeland missile defense of a ballistic missile target.

  • US Navy awards major contract to Huntington Ingalls for its newest class of amphibious vessels

    3 août 2018 | International, Naval

    US Navy awards major contract to Huntington Ingalls for its newest class of amphibious vessels

    By: David B. Larter WASHINGTON — The U.S. Navy awarded shipbuilder Huntington Ingalls a $165.5 million contract to procure long lead-time materials for the LPD-17 Flight II, according to a contract dated Aug. 2 and released Friday. The amphibious transport dock, designated LPD-30, is the first of the 13-ship LPD-17 Flight II class that will replace the current dock landing ships. The program, which was until April known as LX(R), is expected to be built exclusively at HII's yard in Pascagoula, Mississippi. “This is a significant milestone as we embark toward a new flight of LPDs,” Ingalls Shipbuilding President Brian Cuccias said in a statement. “The Flight II LPDs will be highly capable ships meeting the requirements and needs of our Navy-Marine Corps team. We look forward to delivering this series of affordable LPDs to our nation's fleet of amphibious ships.” The Navy is anticipating awarding a detailed design and construction contract either late in 2018 or early 2019. The Navy's cost goal for the program is $1.64 billion for the first ship and $1.4 billion for each subsequent ship, according to the Congressional Research Service. LPD-30 is going to come equipped with Raytheon's Enterprise Air Surveillance Radar, an upgrade over the AN/SPS-48 currently on the LPD-17 class. https://www.defensenews.com/breaking-news/2018/08/03/us-navy-awards-huge-contract-to-huntington-ingalls-for-its-newest-class-of-amphibious-vessels

  • German, French defense ministers push for Eurodrone progress

    21 septembre 2020 | International, Aérospatial, C4ISR, Autre défense

    German, French defense ministers push for Eurodrone progress

    Sebastian Sprenger COLOGNE, Germany — The defense ministers of Germany and France have pushed for speedy progress in the Eurodrone program, urging member nations to initiate the aircraft's development phase before the end of the year. The high-level endorsement means a shot in the arm for a weapons program that has slipped under the radar since Airbus, Dassault and Leonardo unveiled a mock-up drone at the April 2018 Berlin Air Show. While French Armed Forces Minister Florence Parly said she hopes to see the next phase begin by year's end, her German counterpart, Annegret Kramp-Karrenbauer, expressed hope any outstanding issues, which mostly involve cost, could be resolved “in the next few weeks.” The two leaders spoke at Manching, Germany, Airbus' hub for the Eurodrone project and a company site for another key European program, the Future Combat Air System. The unmanned aircraft's official name is “European MALE RPAS,” using acronyms for medium-altitude, long-endurance, remotely piloted aircraft system. The pan-European Organisation for Joint Armament Cooperation manages it on behalf of Germany, France, Italy and Spain. The drone program sets out to field the first new unmanned aircraft certified to fully integrate into civilian airspace, though European authorities have not yet finalized the requisite regulatory framework. Company officials hope that key design features of the drone, such as a propulsion system of two engines — one as a fallback, if necessary — will be conducive to passing future safety checks. That means the technology could cut into the business strategy of American competitor General Atomics. The company aims to be the first to sell its drones, complete with automatic collision-avoidance kit, to Europeans. Officials at the German Defence Ministry did not immediately return a request for comment on how soon the government plans to present a financing and contract strategy to lawmakers — a prerequisite for letting the effort proceed. It remains to be seen if the weapons-capable Eurodrone, whose primary mission is intelligence gathering, will get wrapped up in Germany's debate on the ethical aspects of arming aerial and ground robots. Another program, the Israeli-made Heron TP drone, is still awaiting decision by Germany's parliament, the Bundestag, on whether the government can proceed with installing weapons on the aircraft. The German military is using the drones to watch over deployed forces under a leasing agreement with Israel Aerospace Industries. They are operationally managed by Airbus. It's possible that the Heron TP armament decision will be presented to the Bundestag first, thus capping what a Defence Ministry official told Defense News will likely be a lengthy public meditation on drones and war. But that sequence of approvals is not automatic, Airbus hopes. Either way, time is of the essence for the Defence Ministry, with election years looming in Germany and France starting in 2021. “It would be surprising if we had the Eurodrone first,” said Ulrike Franke, a London-based analyst with the European Council on Foreign Relations. “Because it would amount to a signal that the Heron TP decision had been needlessly stalled.” Questions surrounding the program include whether it can provide enough utility beyond offerings already on the market, including American-made hardware, Franke said. Its success also depends on countries purchasing the future drone in sufficient quantities to get the envisioned benefits of greater European interoperability, she added. https://www.defensenews.com/global/europe/2020/09/18/german-french-defense-ministers-push-for-eurodrone-progress/

Toutes les nouvelles