31 décembre 2018 | International, C4ISR

US Spies Want to Know How to Spot Compromised AI

BY DAVE GERSHGORN

What if you were training an AI, and an adversary slipped a few altered images into its study set?

The US government's research arm for intelligence organizations, IARPA, is looking for ideas on how to detect “Trojan” attacks on artificial intelligence, according to government procurement documents.

Here's the problem the agency wants to solve: At a simple level, modern image-recognition AI learns from analyzing many images of an object. If you want to train an algorithm to detect pictures of a road signs, you have to supply it with pictures of different signs from all different angles. The algorithm learns the relationships between the pixels of the images, and how the structures and patterns of stop signs differ from those of speed-limit signs.

But suppose that, during the AI-training phase, an adversary slipped a few extra images (Trojan horses) into your speed-limit-sign detector, ones showing stop signs with sticky notes on them. Now, if the adversary wants to trick your AI in the real world into thinking a stop sign is a speed-limit sign, it just has to put a sticky note on it. Imagine this in the world of autonomous cars; it could be a nightmare scenario.

The kinds of tools that IARPA (Intelligence Advanced Research Projects Activity) wants would be able to detect issues or anomalies after the algorithm has been trained to recognize different objects in images.

This isn't the only kind of attack on AI that's possible. Security researchers have also warned about inherent flaws in the way artificial intelligence perceives the world, making it possible to alter physical objects like stop signs to make AI algorithms miscategorize them without ever messing with how it was trained, called “adversarial examples.”

While neither Trojan attacks nor the adversarial examples are known to have been used by malicious parties in the real world, researchers have said they're increasingly possible. IARPA is looking at a short timeline as well, expecting the program to conclude after a maximum of two years.

https://www.defenseone.com/technology/2018/12/us-spies-want-know-how-spot-compromised-ai/153826

Sur le même sujet

  • Switzerland names contenders in $8 billion ‘Air 2030’ program

    5 avril 2018 | International, Aérospatial

    Switzerland names contenders in $8 billion ‘Air 2030’ program

    By: Sebastian Sprenger COLOGNE, Germany — Swiss officials have unveiled details of their envisioned reboot of the country's air-defense complex, setting the stage for purchases of aircraft and ground-based missiles totaling more than $8 billion. The head of Switzerland's defense and civilian protection department, Guy Parmelin, on Friday unveiled a list of requirements for the “Air 2030” program that the neutral country wants to begin fulfilling in the mid-2020s to defend its skies and repel intruders. The existing fleet of decades-old F/A-18 and F-5 jets is considered too outdated for the task. New aircraft under consideration include the Airbus Eurofighter, Dassault's Rafale, Saab's Gripen, the F/A-18 Super Hornet from Boeing and Lockheed Martin's F-35A, according to the March 23 list of requirements published by the defense department. Ground-based weapons on the short list are the Eurosam consortium's SAMP/T system; the David's Sling missile shield from Israel; and Raytheon's Patriot system. Swiss officials want to protect an area of 15,000 square kilometers with ground-based weapons, which is more than one-third of the country. They also seek to intercept targets up to 12 kilometers high and 50 kilometers away. The envisioned concept of operations dictates that a fleet of roughly 40 aircraft will intercept those targets outside of the ground weapons' range. Officials want enough capacity to have four planes in the air at any given time during crises. Request for proposals for an acquisition program are expected to be published in the summer, Renato Kalbermatten, a spokesman for the defense department, told Defense News in an email Tuesday. Before a referendum is held about the project in the first half of 2020, ministry officials want to finish qualification of all potential vendors. That includes studying the data from a first round of proposals and collecting final offers from those still in the running at that time, according to Kalbermatten. Referendums are a key tool of the Swiss political process. Asked by a Swiss news agency this month if the country would still have an air force if the population voted against spending money on Air 2030, Parmelin responded dryly: “That's policymaking in Switzerland.” The Swiss won't be asked which type of aircraft the country should buy, only about the program as a whole. Government analysts would then decide which system is best suited for the task, Parmelin said. A 2014 plebiscite saw the acquisition of Sweden's Gripen defeated, a rare outcome for a referendum on security policy matters, Swiss national broadcaster SRF commented at the time. Notably, Germany's future TLVS air and missile defense system, a development based on the trinational Medium Extended Air Defense System, is missing from the lineup of candidate ground-based weapons. That is because the Swiss consider that system suitable only for short and medium ranges, according to Kalbermatten. “As Switzerland has not had a defense system for long ranges since 1999, the first goal is buying a long-range system,” he wrote. Exactly how much money will go to aircraft purchases and how much to ground weapons will depend on the interplay between the two program components ultimately picked, according to officials. However, previous estimates assume that $6 billion or $7 billion would be spent on planes. Winning bidders must agree to arrange for 100 percent of the program cost to flow back into the Swiss economy through so-called offset agreements. Those can be negotiated after final contracts are signed, according to the defense department. The government is looking for aircraft and missile hardware as is, meaning few to no “Helvetizations,” or Swiss-specific tweaks, would be made to the weapons, the new requirements document states. The ministry wants to purchase a single plane type under a “one-fleet policy.” https://www.defensenews.com/land/2018/03/27/switzerland-names-contenders-in-8-billion-air-2030-program/

  • US quickly updated Israeli F-35s after Hamas attack, officials say

    13 décembre 2023 | International, Aérospatial

    US quickly updated Israeli F-35s after Hamas attack, officials say

    Rep. Rob Wittman, R-Va., said the F-35 program has moved at "breakneck speed" to get capabilities and spare parts to Israel's fighter jets.

  • What federal agencies can learn from the MGM and Caesars cyberattacks

    16 octobre 2023 | International, C4ISR, Sécurité

    What federal agencies can learn from the MGM and Caesars cyberattacks

    The best cybersecurity software in the world cannot prevent an employee from giving out the wrong information to the wrong person over the phone.

Toutes les nouvelles