Retour aux nouvelles

31 décembre 2018 | International, C4ISR

US Spies Want to Know How to Spot Compromised AI

BY DAVE GERSHGORN

What if you were training an AI, and an adversary slipped a few altered images into its study set?

The US government's research arm for intelligence organizations, IARPA, is looking for ideas on how to detect “Trojan” attacks on artificial intelligence, according to government procurement documents.

Here's the problem the agency wants to solve: At a simple level, modern image-recognition AI learns from analyzing many images of an object. If you want to train an algorithm to detect pictures of a road signs, you have to supply it with pictures of different signs from all different angles. The algorithm learns the relationships between the pixels of the images, and how the structures and patterns of stop signs differ from those of speed-limit signs.

But suppose that, during the AI-training phase, an adversary slipped a few extra images (Trojan horses) into your speed-limit-sign detector, ones showing stop signs with sticky notes on them. Now, if the adversary wants to trick your AI in the real world into thinking a stop sign is a speed-limit sign, it just has to put a sticky note on it. Imagine this in the world of autonomous cars; it could be a nightmare scenario.

The kinds of tools that IARPA (Intelligence Advanced Research Projects Activity) wants would be able to detect issues or anomalies after the algorithm has been trained to recognize different objects in images.

This isn't the only kind of attack on AI that's possible. Security researchers have also warned about inherent flaws in the way artificial intelligence perceives the world, making it possible to alter physical objects like stop signs to make AI algorithms miscategorize them without ever messing with how it was trained, called “adversarial examples.”

While neither Trojan attacks nor the adversarial examples are known to have been used by malicious parties in the real world, researchers have said they're increasingly possible. IARPA is looking at a short timeline as well, expecting the program to conclude after a maximum of two years.

https://www.defenseone.com/technology/2018/12/us-spies-want-know-how-spot-compromised-ai/153826

Sur le même sujet

  • Australia, Rheinmetall ink $500 million contract for military trucks

    10 septembre 2018 | International, Terrestre

    Australia, Rheinmetall ink $500 million contract for military trucks

    By: Sebastian Sprenger COLOGNE, Germany — Australia has signed a contract with Rheinmetall to supply military trucks worth €430 million (U.S. $500 million) to the country's armed forces, the company announced Thursday. The deal for 1,000 logistics trucks comes on top of a previous order for 2,500 such vehicles, worth €1.2 billion. Deliveries for the new order will begin next year and last until 2024, the German company said. The latest batch of vehicles stems from the Australian LAND 121 Phase 5B program, which is an extension of the larger 3B segment. “This follow-up order is of great strategic significance to us, providing an excellent reference for other important international projects,” Rheinmetall CEO Armin Papperger was quoted as saying in the company statement. “It reflects Australia's satisfaction with our performance and the quality of our vehicles,” he added. “Rheinmetall's latest success in the Asia-Pacific region proves that our products are at the cutting edge of technology, and that the customer see in us a proven and reliable partner, fully capable of carrying out sophisticated large-scale projects.” The Düsseldorf, Germany-based company currently is Australia's largest supplier of military vehicles, according to the website of the state of Queensland. Rheinmetall is establishing a “military vehicle center of excellence” outside of Brisbane, the state's capital. That location is slated to be a hub for the Australian military's LAND 400 program, for which Rheinmetall was formally tapped last month to build Boxer wheeled armored reconnaissance vehicles worth €2.1 billion. The award was first announced by the Australian government in March. There could be yet more business for Rheinmetall in Australia. The next phase of the multibillion-dollar LAND 400 program, which addresses a requirement for new infantry fighting vehicles, began with the release of a tender in late August. Rheinmetall is expected to put forward its Lynx vehicle. Companies have through March 1, 2019, to submit bids. https://www.defensenews.com/industry/2018/09/07/australia-rheinmetall-ink-500-million-contract-for-military-trucks/

  • Remarks by Minister of National Defence Bill Blair to the Canadian Global Affairs Institute conference on NORAD Modernization

    8 mai 2024 | International, Terrestre

    Remarks by Minister of National Defence Bill Blair to the Canadian Global Affairs Institute conference on NORAD Modernization

    Good morning everyone. Thank you very much for this kind introduction and the warm welcome that I've received. I would like to begin, as we always should, by acknowledging that we are gathered on the traditional territory of the Algonquin Anishinabek peoples...

  • Space Force 'reverse industry day' to address gaps in sensing, tracking

    20 mai 2022 | International, Aérospatial

    Space Force 'reverse industry day' to address gaps in sensing, tracking

    During industry meetings this week, program managers will hear from companies about capabilities they're developing to address tactical ISR capability needs.

Toutes les nouvelles