Back to news

February 7, 2019 | International, C4ISR

DARPA: Defending Against Adversarial Artificial Intelligence

Today, machine learning (ML) is coming into its own, ready to serve mankind in a diverse array of applications – from highly efficient manufacturing, medicine and massive information analysis to self-driving transportation, and beyond. However, if misapplied, misused or subverted, ML holds the potential for great harm – this is the double-edged sword of machine learning.

“Over the last decade, researchers have focused on realizing practical ML capable of accomplishing real-world tasks and making them more efficient,” said Dr. Hava Siegelmann, program manager in DARPA's Information Innovation Office (I2O). “We're already benefitting from that work, and rapidly incorporating ML into a number of enterprises. But, in a very real way, we've rushed ahead, paying little attention to vulnerabilities inherent in ML platforms – particularly in terms of altering, corrupting or deceiving these systems.”

In a commonly cited example, ML used by a self-driving car was tricked by visual alterations to a stop sign. While a human viewing the altered sign would have no difficulty interpreting its meaning, the ML erroneously interpreted the stop sign as a 45 mph speed limit posting. In a real-world attack like this, the self-driving car would accelerate through the stop sign, potentially causing a disastrous outcome. This is just one of many recently discovered attacks applicable to virtually any ML application.

To get ahead of this acute safety challenge, DARPA created the Guaranteeing AI Robustness against Deception (GARD) program. GARD aims to develop a new generation of defenses against adversarial deception attacks on ML models. Current defense efforts were designed to protect against specific, pre-defined adversarial attacks and, remained vulnerable to attacks outside their design parameters when tested. GARD seeks to approach ML defense differently – by developing broad-based defenses that address the numerous possible attacks in a given scenario.

“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure. The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” stated Siegelmann.

GARD's novel response to adversarial AI will focus on three main objectives: 1) the development of theoretical foundations for defensible ML and a lexicon of new defense mechanisms based on them; 2) the creation and testing of defensible systems in a diverse range of settings; and 3) the construction of a new testbed for characterizing ML defensibility relative to threat scenarios. Through these interdependent program elements, GARD aims to create deception-resistant ML technologies with stringent criteria for evaluating their robustness.

GARD will explore many research directions for potential defenses, including biology. “The kind of broad scenario-based defense we're looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” said Siegelmann.

GARD will work on addressing present needs, but is keeping future challenges in mind as well. The program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.

A Proposers Day will be held on February 6, 2019, from 9:00 AM to 2:00 PM (EST) at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203 to provide greater detail about the GARD program's technical goals and challenges.

Additional information will be available in the forthcoming Broad Agency Announcement, which will be posted to www.fbo.gov.

https://www.darpa.mil/news-events/2019-02-06

On the same subject

  • £85M contract to boost Type-23 capabilities

    July 5, 2019 | International, Naval

    £85M contract to boost Type-23 capabilities

    The Ministry of Defence has signed an £85 million contract with Rolls-Royce to maintain the engines of the Royal Navy's Type-23 frigate fleet. The contract includes a comprehensive support package to Spey gas turbines, including the overhaul of engines, provision of spares, as well as engineering and safety support. Updates to the turbines are vital as they boost propulsion in the Type-23 Frigates. They are also key pieces of equipment for Anti-Submarine Warfare. The world-beating Type-23 frigate is able to carry out a wide variety of operations, from securing the UK's vital maritime trade routes East of the Suez Canal to safeguarding British interests in the South Atlantic. Defence Minister Stuart Andrew announced the contract at HMNB Devonport where he saw Thursday War training which prepares the Royal Navy for war-fighting, humanitarian relief and emergency situations through a variety of drills and exercises. Defence Minister Stuart Andrew said: This £85m contract demonstrates the UK's commitment to modernisation through the maintenance of our formidable Type-23s. This work continues the British tradition of supporting our closest allies and solidifying our global position as world-leaders in advanced maritime technology and development. The contract will see Rolls-Royce overhaul thirty Type-23 engines from the UK and NATO partners Belgium, Portugal and the Netherlands. The contract is expected to deliver a £35 million increase in savings to the MOD over the next eight years, by incentivising Rolls-Royce to improve repair schemes, minimise unnecessary work and procure spares at a lower cost. This will result in shorter, less expensive overhauls. Rolls-Royce will project manage the support contract, while the main overhaul and repair work will be carried out by RWG based in Aberdeen, supporting up to 25 UK jobs across both companies. Scotland benefits from MOD expenditure of £300 per person each year and a huge investment in local industry and commerce of £1.6 billion. UK Defence also supports over 10,000 industry jobs in Scotland and the nation is renowned for building the world's finest warships including the UK's new aircraft carriers and the Royal Navy's state-of-the-art Type-26 frigates. Defence Equipment and Support Chief of Materiel Ships Vice Admiral Chris Gardner said: The Type 23 frigate is central to Royal Navy operations around the world and keeping it at the forefront of operations is critical. This contract will ensure Rolls-Royce continues to innovate through improving repair schemes, minimising unnecessary work and procuring spares cheaper. This will result in shorter, less expensive overhauls, which is good news for the Royal Navy and good news for the tax payer. Matt Nadin, Director Naval Fleet Services at Rolls-Royce said: This vital support contract builds upon our Rolls-Royce target to achieve and sustain increased Spey engine availability to the Royal Navy and their NATO partners, The Netherlands, Belgium and Portugal. This contract highlights our successful collaboration with the UK Ministry of Defence to provide the technical support and repair activities required to not only keep these engines in-service with the Royal Navy and their NATO partners, but also to deliver increased value for money. https://www.gov.uk/government/news/85m-contract-to-boost-type-23-capabilities

  • eBrief: Drones An “Immediate Threat” – DoD Plans Rapid Acquisition of Counter-UAS Systems

    March 18, 2020 | International, Aerospace

    eBrief: Drones An “Immediate Threat” – DoD Plans Rapid Acquisition of Counter-UAS Systems

    By BARRY ROSENBERGon March 17, 2020 at 12:53 PM The urgency to protect troops, bases, and installations from drone attacks changed forever last year when a swarm of small, low-flying drones unleashed by Yemen's Iranian-backed Houthi rebels targeted Saudi Arabian oil processing facilities. The attack was nothing less than a Pearl Harbor-type wake-up call for the need to counter unmanned aerial systems with defense technology commonly referred to as C-UAS. This Breaking Defense E-Brief examines U.S. Defense Department and global efforts to stay ahead of the threat. It examines sensor development to detect UAS, the use of artificial intelligence to identify targets, and defeat mechanisms ranging from jamming to lasers to knock them down. https://breakingdefense.com/2020/03/ebrief-drones-an-immediate-threat-dod-plans-rapid-acquisition-of-counter-uas-systems

  • Xtend to supply hundreds of Wolverine Gen2 drones to US military

    May 13, 2022 | International, Aerospace

    Xtend to supply hundreds of Wolverine Gen2 drones to US military

    The Israeli company previously delivered several dozen Skylord counter-drone systems to the U.S. as part of a pilot program in 2020.

All news