Back to news

February 7, 2019 | International, C4ISR

DARPA: Defending Against Adversarial Artificial Intelligence

Today, machine learning (ML) is coming into its own, ready to serve mankind in a diverse array of applications – from highly efficient manufacturing, medicine and massive information analysis to self-driving transportation, and beyond. However, if misapplied, misused or subverted, ML holds the potential for great harm – this is the double-edged sword of machine learning.

“Over the last decade, researchers have focused on realizing practical ML capable of accomplishing real-world tasks and making them more efficient,” said Dr. Hava Siegelmann, program manager in DARPA's Information Innovation Office (I2O). “We're already benefitting from that work, and rapidly incorporating ML into a number of enterprises. But, in a very real way, we've rushed ahead, paying little attention to vulnerabilities inherent in ML platforms – particularly in terms of altering, corrupting or deceiving these systems.”

In a commonly cited example, ML used by a self-driving car was tricked by visual alterations to a stop sign. While a human viewing the altered sign would have no difficulty interpreting its meaning, the ML erroneously interpreted the stop sign as a 45 mph speed limit posting. In a real-world attack like this, the self-driving car would accelerate through the stop sign, potentially causing a disastrous outcome. This is just one of many recently discovered attacks applicable to virtually any ML application.

To get ahead of this acute safety challenge, DARPA created the Guaranteeing AI Robustness against Deception (GARD) program. GARD aims to develop a new generation of defenses against adversarial deception attacks on ML models. Current defense efforts were designed to protect against specific, pre-defined adversarial attacks and, remained vulnerable to attacks outside their design parameters when tested. GARD seeks to approach ML defense differently – by developing broad-based defenses that address the numerous possible attacks in a given scenario.

“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure. The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” stated Siegelmann.

GARD's novel response to adversarial AI will focus on three main objectives: 1) the development of theoretical foundations for defensible ML and a lexicon of new defense mechanisms based on them; 2) the creation and testing of defensible systems in a diverse range of settings; and 3) the construction of a new testbed for characterizing ML defensibility relative to threat scenarios. Through these interdependent program elements, GARD aims to create deception-resistant ML technologies with stringent criteria for evaluating their robustness.

GARD will explore many research directions for potential defenses, including biology. “The kind of broad scenario-based defense we're looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” said Siegelmann.

GARD will work on addressing present needs, but is keeping future challenges in mind as well. The program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.

A Proposers Day will be held on February 6, 2019, from 9:00 AM to 2:00 PM (EST) at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203 to provide greater detail about the GARD program's technical goals and challenges.

Additional information will be available in the forthcoming Broad Agency Announcement, which will be posted to www.fbo.gov.

https://www.darpa.mil/news-events/2019-02-06

On the same subject

  • Airbus Helicopters poursuit la militarisation du H160 et du soutien associé

    February 7, 2020 | International, Aerospace

    Airbus Helicopters poursuit la militarisation du H160 et du soutien associé

    Airbus Helicopters et la Direction Générale de l'Armement (DGA) s'engagent sur des études complémentaires pour la militarisation du H160 et du soutien associé, dans le cadre du programme d'Hélicoptère Interarmées Léger (HIL). Ce marché va permettre le lancement des activités de développement préliminaires de la version militaire du H160, aussi connue sous le nom de Guépard, afin de tenir le calendrier de livraison accéléré annoncé en mai 2019 par la Ministre des armées, Florence Parly. Ces études complémentaires comporteront également un volet visant à définir le modèle optimal du soutien pour cette flotte interarmées. Airbus Helicopters, Safran Helicopter Engines et la DGA travailleront en étroite collaboration avec pour objectif de maximiser le taux de disponibilité des hélicoptères tout en optimisant les coûts de soutien de la flotte. « Le lancement dès la phase de pré-développement de ce travail collaboratif entre l'industriel et le ministère des armées pour définir le modèle de soutien du Guépard et les processus associés est essentiel. Cela permettra de garantir un taux de disponibilité élevé dès son entrée en service au sein des forces armées » a déclaré Alexandra Cros, Directrice des Affaires Gouvernementales France d'Airbus Helicopters. « Ces études s'inscrivent dans la continuité des travaux et des engagements pris récemment dans les contrats « verticalisés » pour les flottes Cougar, Caracal et Tigre des armées françaises ». Hélicoptère modulaire par conception, le Guépard permettra de couvrir avec une plateforme unique des missions allant de l'infiltration de commandos à la lutte antinavire, en passant par l'interception aérienne et l'appui-feu, répondant ainsi aux besoins de l'armée de Terre, de la Marine Nationale et de l'Armée de l'air dans le cadre du programme HIL. Le lancement du HIL anticipé en 2021 permettra de livrer les premiers hélicoptères à l'armée française dès 2026. https://www.air-cosmos.com/article/airbus-helicopters-poursuit-la-militarisation-du-h160-et-du-soutien-associ-22521

  • How the Navy’s top officer will prep the service for war with China

    September 19, 2024 | International, Naval

    How the Navy’s top officer will prep the service for war with China

    The chief of naval operations' new navigation plan seeks to have the Navy ready for war with China by 2027.

  • US, Japan say they’ll get to F-35 wreckage first – concerns China will reach crash site dismissed

    April 25, 2019 | International, Aerospace

    US, Japan say they’ll get to F-35 wreckage first – concerns China will reach crash site dismissed

    DAVID PUGLIESE, OTTAWA CITIZEN Top defence officials from the U.S. and Japan say they are confident they'll reach the wreckage of a F-35 stealth fighter jet before China does, even though there are no indications that country is actively searching for the downed plane. There has been speculation from various media outlets that either Russia or China would seek to recover the parts of the F-35 to better understand its stealth and other technology. At an April 19 joint news conference the U.S. Acting Secretary of Defense Patrick Shanahan and Japanese Defence Minister Takeshi Iwaya addressed such concerns but dismissed any worries the aircraft would be recovered by any nation other than the U.S. or Japan. “The Japanese have the lead there, and we're working very collaboratively with them,” Shanahan explained. “And we've got a capability if what they have doesn't prove to be sufficient.” Shanahan was asked whether he was concerned China could get to the wreckage first. “No, I'm not,” he responded. Japanese Defence Minister Takeshi Iwaya also expressed confidence. “We don't have such possibility, absolutely no,” he said. “We are conducting surveillance and warning activities so we can identify and find the missing aircraft.” He was speaking through a translator. The F-35 crashed in an area in the Pacific Ocean which is estimated to be around 5,000 feet deep. Japan is using a submarine and other vessels to search for the wreckage. The aircraft went missing on April 9 about 30 minutes after taking off from Misawa air base in northern Japan. The aircraft was flying with three other F-35As in a night training mission. The next day the Japanese military confirmed that the aircraft had crashed and some debris had been recovered. “This is a very important aircraft, so we would like to locate the aircraft as soon as we can and salvage it,” Iwaya added during the joint news conference. “Japan will lead the investigation, but we're hoping and also it is indispensable to have the support of the U.S. So while we do that, we would like to find the root cause of the accident.” Pentagon spokesman Charles Summers Jr. told the Air Force Times last week that the crash had not shaken the American and Japanese militaries' faith in the F-35 program. “The U.S., and all F-35 partners, remain fully committed to protecting all F-35 capabilities and technology,” he added. https://ottawacitizen.com/news/national/defence-watch/us-japan-say-theyll-get-to-f-35-wreckage-first-concerns-china-will-reach-crash-site-dismissed

All news