21 janvier 2020 | International, C4ISR

Trustworthy AI: A Conversation with NIST's Chuck Romine

By: Charles Romine

Artificial Intelligence (AI) promises to grow the economy and improve our lives, but with these benefits, it also brings new risks that society is grappling with. How can we be sure this new technology is not just innovative and helpful, but also trustworthy, unbiased, and resilient in the face of attack? We sat down with NIST Information Technology Lab Director Chuck Romine to learn how measurement science can help provide answers.

How would you define artificial intelligence? How is it different from regular computing?

One of the challenges with defining artificial intelligence is that if you put 10 people in a room, you get 11 different definitions. It's a moving target. We haven't converged yet on exactly what the definition is, but I think NIST can play an important role here. What we can't do, and what we never do, is go off in a room and think deep thoughts and say we have the definition. We engage the community.

That said, we're using a narrow working definition specifically for the satisfaction of the Executive Order on Maintaining American Leadership in Artificial Intelligence, which makes us responsible for providing guidance to the federal government on how it should engage in the standards arena for AI. We acknowledge that there are multiple definitions out there, but from our perspective, an AI system is one that exhibits reasoning and performs some sort of automated decision-making without the interference of a human.

There's a lot of talk at NIST about “trustworthy” AI. What is trustworthy AI? Why do we need AI systems to be trustworthy?

AI systems will need to exhibit characteristics like resilience, security and privacy if they're going to be useful and people can adopt them without fear. That's what we mean by trustworthy. Our aim is to help ensure these desirable characteristics. We want systems that are capable of either combating cybersecurity attacks, or, perhaps more importantly, at least recognizing when they are being attacked. We need to protect people's privacy. If systems are going to operate in life-or-death type of environments, whether it's in medicine or transportation, people need to be able to trust AI will make the right decisions and not jeopardize their health or well-being.

Resilience is important. An artificial intelligence system needs to be able to fail gracefully. For example, let's say you train an artificial intelligence system to operate in a certain environment. Well, what if the system is taken out of its comfort zone, so to speak? One very real possibility is catastrophic failure. That's clearly not desirable, especially if you have the AI deployed in systems that operate critical infrastructure or our transportation systems. So, if the AI is outside of the boundaries of its nominal operating environment, can it fail in such a way that it doesn't cause a disaster, and can it recover from that in a way that allows it to continue to operate? These are the characteristics that we're looking for in a trustworthy artificial intelligence system.

NIST is supposed to be helping industry before they even know they needed us to. What are we thinking about in this area that is beyond the present state of development of AI?

Industry has a remarkable ability to innovate and to provide new capabilities that people don't even realize that they need or want. And they're doing that now in the AI consumer space. What they don't often do is to combine that push to market with deep thought about how to measure characteristics that are going to be important in the future. And we're talking about, again, privacy, security and resilience ... trustworthiness. Those things are critically important, but many companies that are developing and marketing new AI capabilities and products may not have taken those characteristics into consideration. Ultimately, I think there's a risk of a consumer backlash where people may start saying these things are too easy to compromise and they're betraying too much of my personal information, so get them out of my house.

What we can do to help, and the reason that we've prioritized trustworthy AI, is we can provide that foundational work that people in the consumer space need to manage those risks overall. And I think that the drumbeat for that will get increasingly louder as AI systems begin to be marketed for more than entertainment. Especially at the point when they start to operate critical infrastructure, we're going to need a little more assurance.

That's where NIST can come together with industry to think about those things, and we've already had some conversations with industry about what trustworthy AI means and how we can get there.

I'm often asked, how is it even possible to influence a trillion-dollar, multitrillion-dollar industry on a budget of $150 million? And the answer is, if we were sitting in our offices doing our own work independent of industry, we would never be able to. But that's not what we do. We can work in partnership with industry, and we do that routinely. And they trust us, they're thrilled when we show up, and they're eager to work with us.

AI is a scary idea for some people. They've seen “I, Robot,” or “The Matrix,” or “The Terminator.” What would you say to help them allay these fears?

I think some of this has been overhyped. At the same time, I think it's important to acknowledge that risks are there, and that they can be pretty high if they're not managed ahead of time. For the foreseeable future, however, these systems are going to be too fragile and too dependent on us to worry about them taking over. I think the biggest revolution is not AI taking over, but AI augmenting human intelligence.

We're seeing examples of that now, for instance, in the area of face recognition. The algorithms for face recognition have improved at an astonishing rate over the last seven years. We're now at the point where, under controlled circumstances, the best artificial intelligence algorithms perform on par with the best human face recognizers. A fascinating thing we learned recently, and published in a report, is that if you take two trained human face recognizers and put them together, the dual system doesn't perform appreciably better than either one of them alone. If you take two top-performing algorithms, the combination of the two doesn't really perform much better than either one of them alone. But if you put the best algorithm together with a trained recognizer, that system performs substantially better than either one of them alone. So, I think, human augmentation by AI is going to be the revolution.

What's next?

I think one of the things that is going to be necessary for us is pulling out the desirable characteristics like usability, interoperability, resilience, security, privacy and all the things that will require a certain amount of care to build into the systems, and get innovators to start incorporating them. Guidance and standards can help to do that.

Last year, we published our plan for how the federal government should engage in the AI standards development process. I think there's general agreement that guidance will be needed for interoperability, security, reliability, robustness, these characteristics that we want AI systems to exhibit if they're going to be trusted.

https://www.nist.gov/blogs/taking-measure/trustworthy-ai-conversation-nists-chuck-romine

Sur le même sujet

  • Air Force awards next GPS satellite contract

    17 septembre 2018 | International, Aérospatial

    Air Force awards next GPS satellite contract

    By: Valerie Insinna WASHINGTON — Lockheed Martin has scooped up a contract, worth up to $7.2 billion, for the latest batch of next-generation GPS satellites. The award decision comes as little surprise, as Lockheed Martin is the incumbent on the GPS III program responsible for manufacturing the first 10 satellites. It was also the sole bidder on the most recent contracting vehicle, which will span an additional 22 satellites. “The world is dependent on GPS, from getting directions to getting cash from an ATM machine or trading on the stock exchange,” said Air Force Secretary Heather Wilson in a statement. “These satellites will provide greater accuracy and improved anti-jamming capabilities, making them more resilient.” Three companies — Lockheed Martin, Boeing and Northrop Grumman —won Phase 1 contracts for the follow-one program in 2016 to perform feasibility studies, but the Boeing and Northrop Grumman dropped out of the competition. Boeing in April 2018 confirmed that it did not bid on the opportunity, stating that the solicitation “emphasized mature production to current GPS requirements and did not value lower cost, payload performance or flexibility.” Full article: https://www.c4isrnet.com/space/2018/09/14/air-force-awards-next-gps-satellite-contract

  • IAI names Boaz Levy as new chief exec

    23 novembre 2020 | International, Aérospatial

    IAI names Boaz Levy as new chief exec

    By: Seth J. Frantzman JERUSALEM — Israel Aerospace Industries has named Boaz Levy, its former vice president for the Systems Missiles and Space Group, as its new CEO. Levy's group oversaw major contracts for IAI over the years, including billion-dollar deals in India in 2018. He was also central to the Arrow 3 program, which was developed with U.S. support to intercept high-altitude targets in space, such as ballistic missiles. In his new position, Levy said he plans to lead the company through the challenges of the future. “IAI has transformed in recent years, but we still have a considerable way to achieve the company's potential and strengthen our position in existing and new markets,” he said. Levy's appointment was approved by the Board of Directors, according to a statement from the company. The search committee chose Levy to replace Nimrod Sheffer after announcing in July he would step down. Sheffer had replaced Joseph Weiss as CEO in 2018, who had been at the helm of the company for six years. Sheffer came from the strategic planning area of the company and had drafted a new growth strategy for IAI. In March the company said its annual revenue surpassed $4 billion for the first time. That was an increase from $3.6 billion in 2018 and $3.5 billion in 2017. IAI's chairman of the board, Harel Locker, praised the unanimous decision to nominate Levy. “Levy has successfully managed the Systems Missiles and Space Group — IAI's most profitable group that in recent years has made technological and financial groundbreaking achievements. Boaz knows the domestic and international defense market and understands our customer's needs,” Locker said. Sheffer officially stepped down on Oct. 31, and Levy's nomination has been submitted to Israeli Defense Minister Benny Gantz and Minister for Cyber and National Digital Matters Dudi Amsalem, who supervise the government's authority over IAI. IAI is one of Israel's three large defense companies along with Elbit Systems and Rafael Advanced Defense Systems. Many of the companies' capabilities are integrated into key Israeli weapon systems, such as the Iron Dome air defense system, which uses radar made by IAI subsidiary Elta Systems. Similarly, the new Sa'ar-class corvette will combine capabilities from IAI, including a new sea-to-sea missile the company recently tested. Levy was previously head of IAI's air defense division between 2010 and 2013. A graduate of Israel's Technion, he came to IAI in 1990 as an engineer and worked on the Arrow program in the 1990s and 2000s. According to IAI, he “headed the induction of the Arrow-2 into operational service.” He also headed the Barak-8 program, which IAI claims is one of the world's most advanced air defense systems. The Barak-8 is also a major revenue source for the company. In 2017, Levy indicated Israel aimed to build future interceptors beyond Arrow 2 and Arrow 3. The Arrow 3 is currently Israel's top tier in a multilayered air defense system that includes the Iron Dome and David's Sling, all programs supported by the U.S. More than 20 American states are involved in the production of Arrow 3. Israel faces increasing threats from Iran and also challenges at sea, which is partly why it has shifted its naval doctrine amid adoption of the new Sa'ar 6 corvettes, and also why it rolled out a new multiyear defense strategy called Momentum. The country has aso begun modernizing training, creating new military units and upgrading communications systems for its armed forces. https://www.defensenews.com/industry/2020/11/20/iai-names-boaz-levy-as-new-chief-exec

  • Helsinki dispose de 11 milliards pour son nouvel avion

    15 octobre 2019 | International, Aérospatial

    Helsinki dispose de 11 milliards pour son nouvel avion

    Pascal Kümmerling Le gouvernement finlandais a fixé un plafond de 11 milliards de dollars américains pour l'achat du futur avion de combat a annoncé mercredi le ministère de la Défense. Le plafond financier comprendra le coût d'achat de nouveaux jets, de leurs armes et de divers systèmes au sol. Le projet vise à remplacer la flotte actuelle de 64 avions de combat F/A-18C/D « Hornet » achetés aux États-Unis au début des années 90. Le ministre finlandais de la Défense, Antti Kaikkonen, a déclaré que le nombre de nouveaux avions à acheter pourrait être supérieur ou égal au niveau actuel et espérait qu'il ne serait pas inférieur. Les bases du projet La Finlande ne veut pas sacrifier sa capacité d'engagement en temps de paix et doit pouvoir compter sur une dotation lui permettant un engagement sur de longs mois en cas de situation tendue au niveau international. Pour ce faire la Force aérienne veut un minimum de 64 nouveaux avions de combat multirôle, soit la même dotation qu'actuellement. Pour la Finlande il n'est pas question de sacrifier sa capacité d'action. L'avion doit pouvoir évoluer en réseau connecté avec les systèmes de défenses au sol et le reste de l'armée. A noter, que la Finlande estime que les coûts à l'heure de vol pourraient atteindre trois fois le montant du prix d'achat sur une période de 30 ans. Cette donnée sera prise en compte pour le choix final. Les aéronefs en compétition On retrouve une nouvelle fois les grands classiques du moment, avec le Lockheed-Martin F-35 «Lightning II», le Saab JAS-39 Gripen E MS21, l'Eurofighter «Typhoon II» T3A/B Block20 et le Dassault Rafale F3-R, ainsi que le Boeing F/A-18 E/F « Advanced Super Hornet ». Essais techniques La Finlande a observé avec intérêt les essais des quatre avions effectués ce printemps dans notre pays à Payerne. Le pays procédera également à des tests techniques qui seront effectués en Finlande cet hiver. Mais avant cela, le pays enverra des demandes d'offre plus détaillées aux candidats durant l'automne. Les dernières offres seront organisées en 2020. Le gouvernement finlandais fera son choix en 2021. Le Hornet en Finlande C'est en 1992 que la Finlande décida d'acquérir 64 F/A-18 C/D «Hornet» pour remplacer les bons vieux MiG-21 et Saab J-35 «Draken». A l'époque, les Hornet finlandais ne sont pas complètement équipés, notamment en matière de guerre électronique et d'avionique, ce qui avait permis à l'époque de faire baisser le prix d'achat. Mais dès le début des années 2000, la Finlande décida d'équiper ses « Hornet » des systèmes manquants. Depuis les « Hornet » finlandais ont reçu les missiles air-air à moyenne portée de type AIM-120 AMRAAM avec un système de système de visée plus performant et de doter ceux-ci, du système de guerre électronique AN/ALQ-67. Le groupe de travail du ministère de la Défense finlandais a recommandé que la flotte de F/A-18C/D puisse entrer en retraite durant la période 2025-2030. A signaler, que la Finlande considère que le coût d'un programme d'extension de vie des « Hornet » est à la fois risqué et prohibitif. https://blogs.letemps.ch/pascal-kuemmerling/2019/10/13/helsinki-dispose-de-11-milliards-pour-son-nouvel-avion/

Toutes les nouvelles