Retour aux nouvelles

7 février 2019 | International, C4ISR

DARPA: Defending Against Adversarial Artificial Intelligence

Today, machine learning (ML) is coming into its own, ready to serve mankind in a diverse array of applications – from highly efficient manufacturing, medicine and massive information analysis to self-driving transportation, and beyond. However, if misapplied, misused or subverted, ML holds the potential for great harm – this is the double-edged sword of machine learning.

“Over the last decade, researchers have focused on realizing practical ML capable of accomplishing real-world tasks and making them more efficient,” said Dr. Hava Siegelmann, program manager in DARPA's Information Innovation Office (I2O). “We're already benefitting from that work, and rapidly incorporating ML into a number of enterprises. But, in a very real way, we've rushed ahead, paying little attention to vulnerabilities inherent in ML platforms – particularly in terms of altering, corrupting or deceiving these systems.”

In a commonly cited example, ML used by a self-driving car was tricked by visual alterations to a stop sign. While a human viewing the altered sign would have no difficulty interpreting its meaning, the ML erroneously interpreted the stop sign as a 45 mph speed limit posting. In a real-world attack like this, the self-driving car would accelerate through the stop sign, potentially causing a disastrous outcome. This is just one of many recently discovered attacks applicable to virtually any ML application.

To get ahead of this acute safety challenge, DARPA created the Guaranteeing AI Robustness against Deception (GARD) program. GARD aims to develop a new generation of defenses against adversarial deception attacks on ML models. Current defense efforts were designed to protect against specific, pre-defined adversarial attacks and, remained vulnerable to attacks outside their design parameters when tested. GARD seeks to approach ML defense differently – by developing broad-based defenses that address the numerous possible attacks in a given scenario.

“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure. The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” stated Siegelmann.

GARD's novel response to adversarial AI will focus on three main objectives: 1) the development of theoretical foundations for defensible ML and a lexicon of new defense mechanisms based on them; 2) the creation and testing of defensible systems in a diverse range of settings; and 3) the construction of a new testbed for characterizing ML defensibility relative to threat scenarios. Through these interdependent program elements, GARD aims to create deception-resistant ML technologies with stringent criteria for evaluating their robustness.

GARD will explore many research directions for potential defenses, including biology. “The kind of broad scenario-based defense we're looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” said Siegelmann.

GARD will work on addressing present needs, but is keeping future challenges in mind as well. The program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.

A Proposers Day will be held on February 6, 2019, from 9:00 AM to 2:00 PM (EST) at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203 to provide greater detail about the GARD program's technical goals and challenges.

Additional information will be available in the forthcoming Broad Agency Announcement, which will be posted to www.fbo.gov.

https://www.darpa.mil/news-events/2019-02-06

Sur le même sujet

  • NAVAIR looking for emerging cyber research and development

    15 février 2021 | International, Naval, C4ISR, Sécurité

    NAVAIR looking for emerging cyber research and development

    Mark Pomerleau CORRECTION: A previous version of this story misidentified the organization soliciting white papers as the Naval Information Warfare Systems Command. WASHINGTON — The Navy is searching for solutions from industry that can fill needed gaps in cyber warfare capabilities to secure weapon systems and exploit cyberspace. The Naval Air Systems Command Cyber Warfare Detachment is looking for white papers for research and development efforts on resilient cyber warfare capabilities, according to a Feb. 11 posting to a government contracting website. The proposals should offer advancements or improvements to fill gaps, the notice stated, listing eight buckets of technological interests: Size-, weight- and power-sensitive cyber resiliency for real-time operating systems and aviation warfare environment Access point identification, prioritization and defense Cyber-electronic warfare convergent capabilities Full acquisition cycle cybersecurity measures Cyber test, inspection and incident response concepts Cyber warning system techniques Cyber fault, risk and threat assessment methodologies Resilient network concepts NAVAIR provided a more in-depth list of 36 specific areas, including full-spectrum cyber response and enablement capabilities for multiple weapon system kill chains, sacrificial infrastructure and reactive cyber “armor,” deceptive/misinformation software and hardware capabilities, threat attribution, identification and geolocation, software defined radio protections and capabilities networking, tools for weapon system cyber protection teams for incident response and inspection, and suppression and discovery of malware command and control mechanisms to include triggering, reconnaissance and logic bombs. The command will accept responses until Feb. 10, 2022, and review them quarterly. The next phase of the program will be by invitation. https://www.c4isrnet.com/cyber/2021/02/11/navwar-looking-for-emerging-cyber-research-and-development/

  • Norway acquires new NASAMS air defence systems

    1 février 2024 | International, Aérospatial

    Norway acquires new NASAMS air defence systems

    The contract with the Norwegian Defence Material Agency includes new multi-missile canister launchers and new Fire Distribution Centers for NASAMS, replacing equipment previously donated to Ukraine

  • Contract Awards by US Department of Defense - May 3, 2019

    6 mai 2019 | International, Aérospatial, Naval, Terrestre, C4ISR, Sécurité, Autre défense

    Contract Awards by US Department of Defense - May 3, 2019

    U.S. SPECIAL OPERATIONS COMMAND Insitu Inc., Bingen, Washington, was awarded a maximum $23,000,000 modification (P00019) for an existing non-competitive, single award, indefinite-delivery/indefinite-quantity contract (H92222-16-D-0031) for Mid-Endurance Unmanned Aircraft Systems (MEUAS) 1.5B intelligence, surveillance, and reconnaissance (ISR) services. The $23,000,000 increase to a ceiling of $273,000,000 prevents gaps in ISR services until all task orders are transitioned to the current competitive MEUAS III contracts. Fiscal 2019 operations and maintenance funds in the amount of $7,354,530 are available for obligation at the task order level. U.S. Special Operations Command Headquarters, Tampa, Florida, is the contracting activity. NAVY Valiant Global Defense Services Inc., San Diego, California, is awarded $15,913,990 for firm-fixed-price task order M67854-19-F-7884 under previously award contract M67854-19-D-7876 to provide support services for the Marine Air Ground Task Force (MAGTF) Training Support Service (MTSS), MAGTF Staff Training Program (MSTP). Services will include pre-deployment training programs to Marine Corps operating forces, as well as command, control, communications, and computer mobile training team training at the functional and executive level to commanders and battle staffs, and technical training for operators and information managers. Work will be performed in Quantico, Virginia, and is expected to be completed by November 2020. Fiscal 2019 operations and maintenance (Marine Corps) funds in the amount of $5,380,849 will be obligated at the time of award and these funds will expire at the end of the current fiscal year. This order was competitively awarded under a multiple award task order contract. The Marine Corps Systems Command, Quantico, Virginia, is the contract activity. Lockheed Martin Corp., Fort Worth, Texas, is awarded $7,514,515 for modification P00015 to a previously awarded fixed-price-incentive-fee contract (N0001918C1048) to establish organic depot component repair capabilities for the F-35 Lightning II Air Interceptor System in support of the Air Force, Marine Corps and Navy. Work will be performed in Rochester, Kent, United Kingdom (81.6 percent); and Fort Worth, Texas (18.4 percent), and is expected to be completed in March 2023. Fiscal 2017 aircraft procurement (Air Force); and fiscal 2019 aircraft procurement (Navy, Marine Corp. and Air Force) funds in the amount of $7,514,515 are being obligated at time of award, $3,757,257 of which will expire at the end of the current fiscal year. This contract combines purchases for the Air Force ($3,757,257; 50 percent); Marine Corps ($1,878,629; 25 percent); and Navy ($1,878,629; 25 percent). The Naval Air Systems Command, Patuxent River, Maryland, is the contracting activity. ARMY A4 Construction Company Inc.,* Sandy, Utah, was awarded a $12,309,817 firm-fixed-price contract for construction of a Special Operation Forces Human Performance Training Center. Bids were solicited via the internet with eight received. Work will be performed in Fort Carson, Colorado, with an estimated completion date of May 6, 2021. Fiscal 2019 military construction funds in the amount of $12,309,817 were obligated at the time of the award. U.S. Army Corps of Engineers, Omaha, Nebraska, is the contracting activity (W9128F-19-C-0018). DEFENSE LOGISTICS AGENCY Federal Prison Industries, Inc.,** doing business as UNICOR, Washington, District of Columbia, has been awarded a maximum $9,558,000 firm-fixed-price, indefinite-delivery/indefinite-quantity contract for parkas. This is a one-year base contract with two one-year option periods. Locations of performance are Washington, District of Columbia; and Kentucky, with a May 2, 2020, performance completion date. Using military service is Navy. Type of appropriation is fiscal 2019 through 2020 defense working capital funds. The contracting activity is the Defense Logistics Agency Troop Support, Philadelphia, Pennsylvania (SPE1C1-19-D-F024). *Small business **Mandatory source https://dod.defense.gov/News/Contracts/Contract-View/Article/1836925/source/GovDelivery/

Toutes les nouvelles