31 décembre 2018 | International, C4ISR

US Spies Want to Know How to Spot Compromised AI

BY DAVE GERSHGORN

What if you were training an AI, and an adversary slipped a few altered images into its study set?

The US government's research arm for intelligence organizations, IARPA, is looking for ideas on how to detect “Trojan” attacks on artificial intelligence, according to government procurement documents.

Here's the problem the agency wants to solve: At a simple level, modern image-recognition AI learns from analyzing many images of an object. If you want to train an algorithm to detect pictures of a road signs, you have to supply it with pictures of different signs from all different angles. The algorithm learns the relationships between the pixels of the images, and how the structures and patterns of stop signs differ from those of speed-limit signs.

But suppose that, during the AI-training phase, an adversary slipped a few extra images (Trojan horses) into your speed-limit-sign detector, ones showing stop signs with sticky notes on them. Now, if the adversary wants to trick your AI in the real world into thinking a stop sign is a speed-limit sign, it just has to put a sticky note on it. Imagine this in the world of autonomous cars; it could be a nightmare scenario.

The kinds of tools that IARPA (Intelligence Advanced Research Projects Activity) wants would be able to detect issues or anomalies after the algorithm has been trained to recognize different objects in images.

This isn't the only kind of attack on AI that's possible. Security researchers have also warned about inherent flaws in the way artificial intelligence perceives the world, making it possible to alter physical objects like stop signs to make AI algorithms miscategorize them without ever messing with how it was trained, called “adversarial examples.”

While neither Trojan attacks nor the adversarial examples are known to have been used by malicious parties in the real world, researchers have said they're increasingly possible. IARPA is looking at a short timeline as well, expecting the program to conclude after a maximum of two years.

https://www.defenseone.com/technology/2018/12/us-spies-want-know-how-spot-compromised-ai/153826

Sur le même sujet

  • Air Force Hires Startup To Build Up MDO’s Unified Data Library

    1 novembre 2019 | International, Aérospatial

    Air Force Hires Startup To Build Up MDO’s Unified Data Library

    By THERESA HITCHENS WASHINGTON: The Air Force is expanding a key data tool, the cloud-based Unified Data Library (UDL), that may underpin the service's ambitious Multi-Domain Operations push. Air Force leaders (including acquisition chief Will Roper) believe the UDL will be able to mesh data from all types of sensors to provide space situational awareness (SSA) and command and control (C2) for most Air Force missions. The small $37 million contract, awarded Tuesday to Bluestaq LLC, will “expand the Advanced Command and Control Enterprise Systems and Software (ACCESS) project for the Air Force Research Laboratory (AFRL), the Air Force Space and Missile Systems Center (SMC) Data Program Management Office and the Directorate of Special Programs, Space Situational Awareness Division.” ACCESS will feed the UDL, integrating data “from a wide range of sources spanning commercial, foreign, Department of Defense (DoD) and the Intelligence Community (IC),” according to a company press release. The contract, awarded under the Small Business Innovation Research (SBIR) program, appears to fulfill exactly what Roper and other Air Force acquisition officials are pressing for: drawing in small and innovative companies that can move fast to help the service get inside the speed of Moore's Law instead of taking years or decades to develop new software systems. “We can't specify all specific data sets over the next three years because prioritization is dynamic, but the goal is to eventually integrate all of AFSPC data there along with other sources such as commercial space (already there but more coming), multi-domain data from such as air, land, sea, data from other agencies, and even academia,” an Air Force Space Command spokesperson told Breaking D yesterday. The spokesperson noted that the expansion will support Space Command's National Space Defense Center, designed to run future combat operations in space and to integrate Intelligence Community data with that of the military. It also will support the Combine Space Operations Center (CSPOC), that shares space domain awareness information with allies, the spokesperson said, as well as administrative functions. UDL is the brain child of Maj. Gen. Kim Crider, Air Force Space Command's (AFSPC) data integration guru. Crider is charged with developing AFSPC's classified Enterprise Data Strategy and Roadmap to underpin multi-domain command and control (MDC2) operations. “The Unified Data Library consumes, processes, and distributes millions of unique data products daily originating from dozens of commercial, academic, and government organizations across the world to a diverse user base spanning 25 countries,” according to Bluestaq. “The Unified Data Library storefront provides a robust interactive online API to assist users or developers with education and discovery of available dashboards, data streams, services, structures, and formats. The Air Force plans to expand the Unified Data Library to allow different security classification user access levels and fuse data from all types of sensors to provide command and control for most Air Force missions.” Indeed, AFSPC said in its email, “UDL data is available to any partner organization today, depending on clearance, authorization of the data provider, and classification of the data in question.” The UDL also is being tested by the Commerce Department as it readies itself to take over the mission of providing space situational awareness (SSA) information to commercial and foreign satellite operators. To get some idea of how different the culture is at this company note this comment by Andy Hofle, Bluestaq chief engineer and co-founder: “It has been exciting to see the growing community interest in the data management platform over the last 18 months, and our team has had a tremendous amount of fun playing a role in the development of the project.” https://breakingdefense.com/2019/10/air-force-hires-startup-to-build-up-mdos-unified-data-library

  • Dans la lutte anti-drones, Thalès a développé PARADE pour les neutraliser

    26 octobre 2022 | International, Aérospatial

    Dans la lutte anti-drones, Thalès a développé PARADE pour les neutraliser

    A Brétigny-sur-Orge, les entreprises Thales et CS Group se sont associées pour créer le système PARADE, et lutter contre la menace émergente des drones

  • Pencils up: Bids are due for Army’s Bradley replacement and it’s only the beginning

    19 avril 2021 | International, Terrestre

    Pencils up: Bids are due for Army’s Bradley replacement and it’s only the beginning

    Here's who will be competing in the initial design phase of the Optionally Manned Fighting Vehicle Competition.

Toutes les nouvelles