Retour aux nouvelles

26 septembre 2019 | International, C4ISR

No AI For Nuclear Command & Control: JAIC’s Shanahan

By SYDNEY J. FREEDBERG JR.

GEORGETOWN UNIVERSITY: “You will find no stronger proponent of integration of AI capabilities writ large into the Department of Defense,” Lt. Gen. Jack Shanahan said here, “but there is one area where I pause, and it has to do with nuclear command and control.”

In movies like WarGames and Terminator, nuclear launch controls are the first thing fictional generals hand over to AI. In real life, the director of the Pentagon's Joint Artificial Intelligence Center says, that's the last thing he would integrate AI with. The military is beginning a massive multi-billion dollar modernization of its aging system for Nuclear Command, Control, & Communications (NC3), much of which dates to the Cold War. But the Joint Artificial Intelligence Center is not involved with it.

A recent article on the iconoclastic website War on the Rocks argued “America Needs A ‘Dead Hand',” a reference to the Soviet system designed to automatically order a nuclear launch if the human leadership was wiped out. “I read that,” Shanahan told the Kalaris Intelligence Conference here this afternoon. “My immediate answer is ‘No. We do not.'”

Instead, the JAIC is very deliberately starting with relatively low-risk, non-lethal projects — predicting breakdowns in helicopter engines and mapping natural disasters — before moving on to combat-related functions such as intelligence analysis and targeting next year. On the Pentagon's timeline, AI will be coming to command posts before it is embedded in actual weapons, and even then the final decision to use lethal force will always remain in human hands.

The standard term in the Pentagon now for human involvement with AI and weapons now is “human on the loop,” a shift from human IN the loop. That reflects greater stress on the advisory function of humans with AI and a recognition that domains like cyber require almost instantaneous responses that can't wait for a human.

Hawkish skeptics say slowing down to ask human permission could cripple US robots against their less-restrained Russian or Chinese counterparts. Dovish skeptics say this kind of human control would be too easily bypassed.

Shanahan does see a role for AI in applying lethal force once that human decision is made. “I'm not going to go straight to ‘lethal autonomous weapons systems,'” he said, “but I do want to say we will use artificial intelligence in our weapons systems... to give us a competitive advantage. It's to save lives and help deter war from happening in the first place.”

The term “lethal autonomous weapons systems” was popularized by the Campaign to Stop Killer Robots, which seeks a global ban on all AI weapons. Shanahan made clear his discomfort with formal arms control measures, as opposed to policies and international norms, which don't bind the US in the same way.

“I'll be honest with you,” Shanahan said. “I don't like the term, and I do not use the term, ‘arms control' when it comes to AI. I think that's unhelpful when it comes to artificial intelligence: It's largely a commercial technology,” albeit with military applications.

“I'm much more interested, at least as a starting point, in international rules and norms and behavior,” he continued. (Aside from the space is governed almost exclusively “It's extremely important to have those discussions.”

“This is the ultimate human decision that needs to be made....nuclear command and control,” he said. “We have to be very careful. Knowing ...the immaturity of technology today, give us a lot of time to test and evaluate.”

“Can we use artificial intelligence to make better decisions, to make more informed judgments about what might be happening, to reduce the potential for civilian casualties or collateral damage?” Shanahan said. “I'm an optimist. I believe you can. It will not eliminate it, never. It's war; bad things are going to happen.”

While Shanahan has no illusions about AI enabling some kind of cleanly surgical future conflict, he doesn't expect a robo-dystopia, either.

“The hype is a little dangerous, because it's uninformed most of the time, and sometimes it's a Hollywood-driven killer robots/Terminator/SkyNet worst case scenario,” he said. “I don't see that worst case scenario any time in my immediate future.”

“I'm very comfortable saying our approach — even though it is emerging technology, even though it unfolds very quickly before our eyes — it will still be done in a deliberate and rigorous way so we know what we get when we field it,” Shanahan said.

“As the JAIC director, I'm focused on really getting to the fielding,” he said, moving AI out of the lab into the real world — but one step at a time. “We're always going to start with limited narrow use cases. Say, can we take some AI capability and put it in a small quadcopter drone that will make it easier to clear out a cave, [and] really prove that it works before we ever get it to a [large] scale production.”

“We will have a very clear understanding of what it can do and what it can't do,” he said. “That will be through experimentation, that will be through modeling and simulation, and that will be in wargames. We've done that with every piece of technology we've ever used, and I don't expect this to be any different.”

The JAIC is even looking to hire an in-house ethicist of sorts, a position Shanahan has mentioned earlier but sought to clarify today. “It'll be someone who's a technical standards [expert] / ethicist,” he said. “As we develop the models and algorithms... they can look at that make sure the process is abiding by our rules of the road.”

“I'm also interested in, down the road, getting some help from the outside on sort of those deeper philosophical questions,” he continued. “I don't focus on them day to day, because of my charter to field now, but it's clear we have to be careful about this.”

“I do not see that same approach in Russia or China,” Shanahan said. “What sets us apart is... our focus on real rigor in test and evaluation, validation and verification, before we field capability that could have lives at stake.”

https://breakingdefense.com/2019/09/no-ai-for-nuclear-command-control-jaics-shanahan

Sur le même sujet

  • Companies are lining up to build a replacement for the MQ-9 Reaper drone

    18 septembre 2020 | International, Aérospatial

    Companies are lining up to build a replacement for the MQ-9 Reaper drone

    Valerie Insinna WASHINGTON — As the U.S. Air Force embarks on a new effort to field a replacement for the MQ-9 Reaper drone, multiple defense companies are stepping up with new, long-range, stealthy design concepts for the emerging MQ-Next competition. On Sept. 11, Northrop Grumman and Lockheed Martin released renderings of their respective offerings for the Air Force's MQ-Next program. Northrop made public its swarming SG-2 concept, and Lockheed announced its flying-wing design. General Atomics put out a concept drawing of a next-generation uncrewed aerial system on Sept. 14 to correspond with the first day of the Air Force Association's Air, Space and Cyber Conference. For the past two decades, the Air Force has relied on the MQ-1 Predator and then the MQ-9 Reaper — both made by General Atomics — as its workhorse drones for surveillance and strike missions in the Middle East. But as more commercial drone makers enter the fray, it may become more economical and effective to operate a family of UAVs, with some built for high-end penetrating strike and reconnaissance missions, and others for low-end surveillance from commercial off-the-shelf manufacturers, said Will Roper, the Air Force's top acquisition official. “You might make the case that the Department [of the Air Force] needs both,” he said during a Sept. 15 roundtable with reporters. “But I wanted to give our team time to discuss with industry options that exist on both sides of that divide. We've got a lot of interesting responses, and I'm in discussions right now with the operational side of the Air Force about what they think the requirement is going to be.” The Air Force issued a request for information to industry on June 3, seeking market research on available technologies as well as conceptual designs. Boeing and Kratos each confirmed they responded to the request for information but have not released concept art for their potential offerings. General Atomics, Lockheed and Northrop have begun to shed light on their respective designs. Northrop's flying-wing design bears a close resemblance to the X-47B the company designed for the Navy, including using the same Distributed Autonomy/Responsive Control flight management system, which allows for operators to task multiple drones to fly autonomously according to parameters set by the user. However, the aircraft in the rendering is just one potential concept that Northrop could develop for the MQ-Next family of systems, said Richard Sullivan, the company's vice president of program management. “The customer didn't really give us strict requirements. We know that the [National Defense Strategy] scenario calls out environments with a pretty significant threat scenario. And so, what would we do to mitigate those?” he said. “We looked at those things, and we came up with a family of concepts ... trying to solve the problem across the landscape in terms of the ranges, the threats and the costs.” The General Atomics concept features a stealthy, long-winged, jet-powered air vehicle — a departure from the turboprop-powered MQ-9 Reaper. Dave Alexander, president of General Atomics Aeronautical Systems, told Defense News that the aircraft's survivability and endurance, which is “significantly longer” than the Reaper, will be defining characteristics for the company's offering. Alexander also pointed to internal investments made by the company's aeronautics division and its Electromagnetic Systems Group on advanced propulsion systems, though he declined to say more about potential engine advancements. Keeping costs down will also be an important factor, he said. “Some platforms that get up to super high costs, even though they're unmanned — you can't afford to lose them. So they're not attrition-tolerant, and we want to hang on to that piece of it.” Lockheed Martin's operational analysis has found that an optimal-force mix of drones will require high-end aircraft and low-cost, expendable systems that can operate in swarms, according to Jacob Johnson, the company's unmanned aerial systems program manager. The company's next-generation UAS concept art features a tailless, stealthy, flying-wing design geared toward the high-end fight, although Johnson said Lockheed may put forward less exquisite systems depending on the Air Force's final requirements. “Over the last few years, with a lot of the [drone] shootdowns across the globe, one of the trends that I think is hard to ignore is what used to be considered permissive airspace. [It] is becoming increasingly contested,” he said. “Survivability is really the key to almost any mission, and I think that trend is going to continue into the future.” However, survivability alone will not be enough, Johnson said. The Air Force has made clear that any future air system must plug into the service's Advanced Battle Management System and export data across that system. Lockheed also plans to develop the drone using digital engineering to lower the total cost. https://www.defensenews.com/digital-show-dailies/air-force-association/2020/09/17/defense-companies-are-lining-up-to-build-a-replace

  • Croatia to buy 50 German Leopard tanks under Ukraine discount

    29 octobre 2024 | International, Terrestre

    Croatia to buy 50 German Leopard tanks under Ukraine discount

    The tanks are to be purchased under a reduced price to compensate the Croatian Army for its planned transfer of used equipment to Ukraine.

  • Northrop Grumman begins building first Triton UAV for Australia

    29 octobre 2020 | International, Aérospatial, Naval, C4ISR

    Northrop Grumman begins building first Triton UAV for Australia

    Gareth Jennings Northrop Grumman commenced assembly of the first of up to seven MQ-4C Triton high-altitude, long-endurance (HALE) unmanned aerial vehicles (UAVs) for Australia, it announced on 27 October. The milestone saw the first jig-load for a Triton intelligence, surveillance and reconnaissance (ISR) UAV for the Royal Australian Air Force (RAAF) take place at Northrop Grumman's Moss Point facility in Mississippi. Final assembly and flight testing will follow at the company's Palmdale facility and at Edwards Air Force Base in California, ahead of delivery to Australia in 2023. “The MQ-4C Triton will be a very important ISR capability for Australia,” Air Commodore Terry van Haren, the RAAF's air attaché to the Australian Embassy in the United States, said during the ceremony that was also attended by senior Australian and US government and military figures. “It is ideally suited for Australian operating conditions, given its high altitude, long endurance, and impressive sensor suite. The Royal Australian Air Force looks forward to operating the MQ-4C alongside its other ISR and response aircraft such as the [Boeing] P-8A Poseidon [maritime multimission aircraft (MMA)].” Australia currently has three Tritons in the US Navy's (USN's) low-rate initial production (LRIP) Lot 5, which also includes two main operating bases, and one forward operating base for the country in an integrated functional capability-four (IFC-4) and multiple intelligence configuration. IFC-4 functionality will add a signals intelligence capability to the UAV's baseline IFC-3 configuration. https://www.janes.com/defence-news/news-detail/northrop-grumman-begins-building-first-triton-uav-for-australia

Toutes les nouvelles