Back to news

February 7, 2019 | International, C4ISR

DARPA: Defending Against Adversarial Artificial Intelligence

Today, machine learning (ML) is coming into its own, ready to serve mankind in a diverse array of applications – from highly efficient manufacturing, medicine and massive information analysis to self-driving transportation, and beyond. However, if misapplied, misused or subverted, ML holds the potential for great harm – this is the double-edged sword of machine learning.

“Over the last decade, researchers have focused on realizing practical ML capable of accomplishing real-world tasks and making them more efficient,” said Dr. Hava Siegelmann, program manager in DARPA's Information Innovation Office (I2O). “We're already benefitting from that work, and rapidly incorporating ML into a number of enterprises. But, in a very real way, we've rushed ahead, paying little attention to vulnerabilities inherent in ML platforms – particularly in terms of altering, corrupting or deceiving these systems.”

In a commonly cited example, ML used by a self-driving car was tricked by visual alterations to a stop sign. While a human viewing the altered sign would have no difficulty interpreting its meaning, the ML erroneously interpreted the stop sign as a 45 mph speed limit posting. In a real-world attack like this, the self-driving car would accelerate through the stop sign, potentially causing a disastrous outcome. This is just one of many recently discovered attacks applicable to virtually any ML application.

To get ahead of this acute safety challenge, DARPA created the Guaranteeing AI Robustness against Deception (GARD) program. GARD aims to develop a new generation of defenses against adversarial deception attacks on ML models. Current defense efforts were designed to protect against specific, pre-defined adversarial attacks and, remained vulnerable to attacks outside their design parameters when tested. GARD seeks to approach ML defense differently – by developing broad-based defenses that address the numerous possible attacks in a given scenario.

“There is a critical need for ML defense as the technology is increasingly incorporated into some of our most critical infrastructure. The GARD program seeks to prevent the chaos that could ensue in the near future when attack methodologies, now in their infancy, have matured to a more destructive level. We must ensure ML is safe and incapable of being deceived,” stated Siegelmann.

GARD's novel response to adversarial AI will focus on three main objectives: 1) the development of theoretical foundations for defensible ML and a lexicon of new defense mechanisms based on them; 2) the creation and testing of defensible systems in a diverse range of settings; and 3) the construction of a new testbed for characterizing ML defensibility relative to threat scenarios. Through these interdependent program elements, GARD aims to create deception-resistant ML technologies with stringent criteria for evaluating their robustness.

GARD will explore many research directions for potential defenses, including biology. “The kind of broad scenario-based defense we're looking to generate can be seen, for example, in the immune system, which identifies attacks, wins and remembers the attack to create a more effective response during future engagements,” said Siegelmann.

GARD will work on addressing present needs, but is keeping future challenges in mind as well. The program will initially concentrate on state-of-the-art image-based ML, then progress to video, audio and more complex systems – including multi-sensor and multi-modality variations. It will also seek to address ML capable of predictions, decisions and adapting during its lifetime.

A Proposers Day will be held on February 6, 2019, from 9:00 AM to 2:00 PM (EST) at the DARPA Conference Center, located at 675 N. Randolph Street, Arlington, Virginia, 22203 to provide greater detail about the GARD program's technical goals and challenges.

Additional information will be available in the forthcoming Broad Agency Announcement, which will be posted to www.fbo.gov.

https://www.darpa.mil/news-events/2019-02-06

On the same subject

  • Army looks to give its old combat boot the boot

    January 14, 2019 | International, Land

    Army looks to give its old combat boot the boot

    By JOHN VANDIVER | STARS AND STRIPES The Army is testing new combat boots at select boot camps in a push to better compete with the more comfortable commercial brands favored by many soldiers for their lightness. The U.S. Army Research, Development and Engineering Command Soldier Center in Natick, Mass., has developed prototypes that soldiers will wear at three different basic training and active duty sites during the next four months. “Soldiers live in their boots and many will tell you that there is no piece of equipment more important to their lethality and readiness,” said Al Adams, a team leader at the Army's soldier center, in a statement. “A bad pair of boots will ruin a Soldier's day and possibly result in injuries, so we really believe that each of these prototype boots have the potential to improve the lives of Soldiers.” The prototypes, which utilize more flexible types of leather and lighter outsoles, will be fielded to 800 new recruits at Fort Leonard Wood, Mo., and Fort Jackson, S.C. Another 900 pairs will go to infantry soldiers at Fort Bliss, Texas. The Army's soldier center team will custom-fit each pair and collect solider surveys on boot performance in the spring. The prototypes are up to 1.5 pounds lighter per pair than those issued today. “In terms of energy expenditure or calories burned, 1-pound of weight at the feet is equivalent to 4-pounds in your rucksack,” Adams said in a statement. The prototypes in their current design would be the first significant change to the basic Army combat boot in years. While there have been improvements to the Army's special footwear for jungle, mountain and cold weather locations, there is room for improvement in general-purpose boots issued to new recruits, the Army said. “Most components of these combat boots have not been updated in almost 30 years,” Army footwear engineer Anita Perkins said in a statement. Army surveys have found that satisfaction with the Army combat boot is lower than with commercial varieties. In a poll of 14,000 soldiers, the Army said it found that nearly 50 percent choose comfortable sneaker-like commercial boots over Army-issued ones. The problem confronting Army officials is that commercial comfort can come with trade-offs. The Army said its ultimate aim is to bridge the comfort gap while maintaining durability and protection. vandiver.john@stripes.com Twitter: @john_vandiver https://www.stripes.com/news/army-looks-to-give-its-old-combat-boot-the-boot-1.563985

  • Air Force Research Lab awards $76 million for lunar experimentation

    November 10, 2022 | International, Aerospace

    Air Force Research Lab awards $76 million for lunar experimentation

    AFRL expects the Oracle satellite to launch in 2025 and have a two-year mission life.

  • Latest emergency coronavirus funding package includes tens of billions for defense, VA

    March 26, 2020 | International, Aerospace, Naval, Land, C4ISR, Security

    Latest emergency coronavirus funding package includes tens of billions for defense, VA

    Leo Shane III The Department of Defense is poised to get another $10.5 billion in emergency funding and the Department of Veterans Affairs another $19.6 billion as part of a $2 trillion package designed to blunt the economic impact of the ongoing coronavirus outbreak throughout America. The money includes $1.4 billion in direct funding for National Guard deployments to help state governments respond to emerging health needs, $1.5 billion for the expansion of military hospitals and mobile medical centers if needed, and $1 billion for the Defense Department to help with production of medical supplies running short because of the pandemic. The measure — which if passed would be the third major coronavirus response by Congress this month — also includes provisions to deliver billions of dollars to hospitals, new state and local government support, expanded unemployment compensation and massive business loans, all designed to help stabilize the economy amid unprecedented social shut-downs to curb the virus spread. Senate lawmakers are expected to vote on the measure sometime Wednesday afternoon. The House could take it up soon afterwards. Nationally, more than 54,000 individuals have tested positive for COVID-19, the strain of coronavirus causing worldwide disruptions and panic. More than 730 deaths have been connected to the illness in the United States alone, all in the last month. "Combating this disease has forced our country to put huge parts of our national life on pause and triggered layoffs at a breath-taking pace,” Senate Majority Leader Mitch McConnell, R-Ky., said after the deal was reached Wednesday morning. “This strange new reality has forced our nation on to something like war-time footing. A fight has arrived on our shores.” Lawmakers who worked on the agreement said the Defense Department funding is targeted “primarily for the protection of members of the Armed Forces, their families, and military retirees.” None of the funds can be transferred to accounts connected with the construction of a southern border wall, a sticking point between Democrats and President Donald Trump in past appropriations fights. Of the $10.5 billion total, $1 billion will be set aside for use with the Defense Production Act, allowing department officials to “invest in manufacturing capabilities that are key to increasing the production rate of personal protective equipment and medical equipment.” Those items have been in short supply in recent weeks. Trump invoked the act earlier this week but has yet to order any civilian production of the scare items. Veterans Affairs Within VA, the majority of the money — nearly $15.9 billion — will be used for direct care specifically in response to veterans health needs. “This covers treatment of veterans nationwide for coronavirus within VA hospitals as well as in community urgent care clinics and emergency rooms,” a bill summary stated. “These funds allow VA to cover overtime for their clinical staff, the purchase of personal protective equipment, test kits, and other necessary equipment to manage the impacts of this pandemic among the veteran population.” Another $3.1 billion will be set aside for new telemedicine efforts within VA, including staffing and equipping mobile treatment sites and “remodeling to VA facilities and state run veterans homes to address the needs of veterans being treated for coronavirus.” Nearly 300 veterans have already been diagnosed with the coronavirus strain, and four deaths of VA patients connected to the illness. The bill also includes a number of expanded authorities for VA officials to respond to the threat, including extended overtime pay rules for medical providers, new agreements with broadband internet suppliers to increase telemedicine options for veterans, and waiving some rules regarding income-level eligibility for department benefits. Lawmakers also included $360 million for the Department of Labor to invest in programs providing training and supportive services for dislocated workers, to specifically include homeless veterans. White House officials have indicated Trump will sign the measure into law if it passes Congress. Earlier this week, Trump called on lawmakers to quickly pass some deal, saying that the economic impact of the illness on the country could be more devastating than the health complications. https://www.militarytimes.com/news/pentagon-congress/2020/03/25/latest-emergency-coronavirus-funding-package-includes-tens-of-billions-for-defense-va/

All news