21 janvier 2020 | International, C4ISR

Trustworthy AI: A Conversation with NIST's Chuck Romine

By: Charles Romine

Artificial Intelligence (AI) promises to grow the economy and improve our lives, but with these benefits, it also brings new risks that society is grappling with. How can we be sure this new technology is not just innovative and helpful, but also trustworthy, unbiased, and resilient in the face of attack? We sat down with NIST Information Technology Lab Director Chuck Romine to learn how measurement science can help provide answers.

How would you define artificial intelligence? How is it different from regular computing?

One of the challenges with defining artificial intelligence is that if you put 10 people in a room, you get 11 different definitions. It's a moving target. We haven't converged yet on exactly what the definition is, but I think NIST can play an important role here. What we can't do, and what we never do, is go off in a room and think deep thoughts and say we have the definition. We engage the community.

That said, we're using a narrow working definition specifically for the satisfaction of the Executive Order on Maintaining American Leadership in Artificial Intelligence, which makes us responsible for providing guidance to the federal government on how it should engage in the standards arena for AI. We acknowledge that there are multiple definitions out there, but from our perspective, an AI system is one that exhibits reasoning and performs some sort of automated decision-making without the interference of a human.

There's a lot of talk at NIST about “trustworthy” AI. What is trustworthy AI? Why do we need AI systems to be trustworthy?

AI systems will need to exhibit characteristics like resilience, security and privacy if they're going to be useful and people can adopt them without fear. That's what we mean by trustworthy. Our aim is to help ensure these desirable characteristics. We want systems that are capable of either combating cybersecurity attacks, or, perhaps more importantly, at least recognizing when they are being attacked. We need to protect people's privacy. If systems are going to operate in life-or-death type of environments, whether it's in medicine or transportation, people need to be able to trust AI will make the right decisions and not jeopardize their health or well-being.

Resilience is important. An artificial intelligence system needs to be able to fail gracefully. For example, let's say you train an artificial intelligence system to operate in a certain environment. Well, what if the system is taken out of its comfort zone, so to speak? One very real possibility is catastrophic failure. That's clearly not desirable, especially if you have the AI deployed in systems that operate critical infrastructure or our transportation systems. So, if the AI is outside of the boundaries of its nominal operating environment, can it fail in such a way that it doesn't cause a disaster, and can it recover from that in a way that allows it to continue to operate? These are the characteristics that we're looking for in a trustworthy artificial intelligence system.

NIST is supposed to be helping industry before they even know they needed us to. What are we thinking about in this area that is beyond the present state of development of AI?

Industry has a remarkable ability to innovate and to provide new capabilities that people don't even realize that they need or want. And they're doing that now in the AI consumer space. What they don't often do is to combine that push to market with deep thought about how to measure characteristics that are going to be important in the future. And we're talking about, again, privacy, security and resilience ... trustworthiness. Those things are critically important, but many companies that are developing and marketing new AI capabilities and products may not have taken those characteristics into consideration. Ultimately, I think there's a risk of a consumer backlash where people may start saying these things are too easy to compromise and they're betraying too much of my personal information, so get them out of my house.

What we can do to help, and the reason that we've prioritized trustworthy AI, is we can provide that foundational work that people in the consumer space need to manage those risks overall. And I think that the drumbeat for that will get increasingly louder as AI systems begin to be marketed for more than entertainment. Especially at the point when they start to operate critical infrastructure, we're going to need a little more assurance.

That's where NIST can come together with industry to think about those things, and we've already had some conversations with industry about what trustworthy AI means and how we can get there.

I'm often asked, how is it even possible to influence a trillion-dollar, multitrillion-dollar industry on a budget of $150 million? And the answer is, if we were sitting in our offices doing our own work independent of industry, we would never be able to. But that's not what we do. We can work in partnership with industry, and we do that routinely. And they trust us, they're thrilled when we show up, and they're eager to work with us.

AI is a scary idea for some people. They've seen “I, Robot,” or “The Matrix,” or “The Terminator.” What would you say to help them allay these fears?

I think some of this has been overhyped. At the same time, I think it's important to acknowledge that risks are there, and that they can be pretty high if they're not managed ahead of time. For the foreseeable future, however, these systems are going to be too fragile and too dependent on us to worry about them taking over. I think the biggest revolution is not AI taking over, but AI augmenting human intelligence.

We're seeing examples of that now, for instance, in the area of face recognition. The algorithms for face recognition have improved at an astonishing rate over the last seven years. We're now at the point where, under controlled circumstances, the best artificial intelligence algorithms perform on par with the best human face recognizers. A fascinating thing we learned recently, and published in a report, is that if you take two trained human face recognizers and put them together, the dual system doesn't perform appreciably better than either one of them alone. If you take two top-performing algorithms, the combination of the two doesn't really perform much better than either one of them alone. But if you put the best algorithm together with a trained recognizer, that system performs substantially better than either one of them alone. So, I think, human augmentation by AI is going to be the revolution.

What's next?

I think one of the things that is going to be necessary for us is pulling out the desirable characteristics like usability, interoperability, resilience, security, privacy and all the things that will require a certain amount of care to build into the systems, and get innovators to start incorporating them. Guidance and standards can help to do that.

Last year, we published our plan for how the federal government should engage in the AI standards development process. I think there's general agreement that guidance will be needed for interoperability, security, reliability, robustness, these characteristics that we want AI systems to exhibit if they're going to be trusted.

https://www.nist.gov/blogs/taking-measure/trustworthy-ai-conversation-nists-chuck-romine

Sur le même sujet

  • RUAG International implements full remote supervision support for Live simulation and training system to include AAR

    18 novembre 2020 | International, Aérospatial, Naval, Terrestre, C4ISR

    RUAG International implements full remote supervision support for Live simulation and training system to include AAR

    November 16, 2020 - RUAG Simulation & Training has confirmed its capabilities for full remote monitoring and supervision support for Live training systems on behalf of a European customer. The comprehensive event specifically featured the inclusion of after-action review (AAR) reporting and analysis and was provided to full customer satisfaction. The remote supervision was supported from Switzerland and enabled the customer's scheduled training to proceed in keeping with pandemic guidelines. RUAG Simulation & Training supported army and police forces, deploying force-on-force and Mobile Combat Training Center (CTC) services featuring Gladiator Modular Tactical Engagement Training System. RUAG Simulation & Training's competency for full remote supervision, including data transfer necessary to AAR, was proven within the scope of laser-based force-on-force training exercises. The implementation of the remote protocols was a prerequisite to ensuring highest training fidelity despite COVID-19 pandemic conditions. Remote supervision offset the need for the standard on-site presence of two to four professional supervisors, assuring that international travel from Switzerland was avoided. The customer experienced a reliable and realistic training and a thorough and accurate AAR debriefing, all within their protected training-site environment. The scheduled training, featuring a special operations program, proceeded according to plan, with multiple units and their various specialisations included within the Live system. RUAG's Gladiator Supervision Equipment ensured the data transmitted from positioning and interaction sensors, from the various modules, components and participants comprised in the Mobile CTC Live system, was recorded and analysed for an effective AAR. This achievement strengthened collaboration in an ongoing series of comprehensive laser-based force-on-force trainings commissioned by the customer. The Training-as-a-Service approach included full rental of Gladiator Modular Tactical Engagement Training System components and local consultants for operational system checks and servicing. Together with the customer, the team from RUAG Simulation & Training defined system configuration and component needs to target highest training fidelity and performance accuracy within budget guidelines. RUAG Simulation & Training AG is a professional and trusted partner for Live, Virtual and Constructive (LVC) simulation & training solutions. Combining cutting-edge technology with an unparalleled depth of experience, RUAG develops affordable products tailored to training goals and designed for saving lives and protecting assets in the complex environment of today's and tomorrow's battlefields. Gladiator Modular Tactical Engagement Training Systems enable live conflict scenario simulations to proceed at the highest levels of fidelity. __________________________________________________________ RUAG MRO International is an independent supplier, support provider and integrator of systems and components for civil and military aviation worldwide. It also develops and supports simulation and training systems and solutions for international trained security forces. Highly specialized in the support of aircraft and helicopters throughout their entire life cycle, the company includes maintenance, repair and overhaul services, upgrades, and the development, manufacture and integration of subsystems and components in their service portfolio. In addition, as the manufacturer (OEM) of the Dornier 228, a versatile aircraft for special missions as well as passenger and cargo operations, RUAG International focuses on customer support solutions, including OEM services. Moreover, RUAG MRO International is a developer, OEM and system support provider for simulation and training systems technology for live, virtual and constructive (LVC) training. Complex and flexible functions, and a holistic approach, support realistic training scenarios, adapted to mission goals, at individual, team and unit instruction levels. View source version on RUAG : https://www.ruag.com/en/news/ruag-international-implements-full-remote-supervision-support-live-simulation-training-system

  • Mettre en place un Conseil de sécurité européen ? Une idée à travailler

    18 février 2019 | International, Sécurité

    Mettre en place un Conseil de sécurité européen ? Une idée à travailler

    (B2) Berlin insiste régulièrement sur un point souvent oublié dans la rhétorique sur l'armée européenne : la mise en place d'un « Conseil de sécurité de l'UE ». Un point qui mérite un peu d'attention Avec mes amis de ‘La faute à l'Europe‘ (J. Quatremer, Y. A. Noguès, K. Landaburu, H. Beaudoin), qui reçoivent ce week-end Michèle Alliot Marie, alias MAM, l'ancienne ministre de la Défense (sous Jacques Chirac) et ministre des Affaires étrangères (sous Nicolas Sarkozy), nous parlons ‘défense', ‘Europe puissance' et notamment de ce Conseil de sécurité européen (video). @bruxelles2 pèse le pour et le contre d'un Conseil de sécurité européen à l'image de @ONU_fr pic.twitter.com/JfbkGh4Kot Une proposition franco-allemande Cette proposition ne nait pas de nulle part. Elle figurait en dernier lieu dans la déclaration de Meseberg adoptée par les deux dirigeants Emmanuel Macron et Angela Merkel en juin 2018. L'objectif est d'avoir un « débat européen dans de nouveaux formats » et « d'accroitre la rapidité et l'efficacité de la prise de décision de l'Union européenne [en matière] de politique étrangère » (lire : Défense, Sécurité, Migrations, Développement, l'accord franco-allemand de Meseberg). Une explication merkelienne Au Parlement européen, en novembre 2018, la chancelière Angela Merkel souligne l'importance d'« une enceinte au sein de laquelle des décisions importantes pourront êtres prises », avec une « présidence tournante » (lire : « Une armée (européenne) montrerait au monde qu'entre (nous) il n'y aurait plus de guerre » (Angela Merkel). Le format serait limité précise-t-on du côté allemand : « un petit cercle d'États se relayant et représentant l'ensemble de l'UE [pour] travailler plus promptement et intensément au règlement des crises en cours. » (1) Une certaine réserve française Du côté français, on ne peut pas dire que le projet suscite une grande mobilisation. A l'Élysée, la prudence est de règle : « C'est une idée [de] la Chancelière. Ce pourrait être une proposition commune, mais cela mérite encore [d'être travaillé] » l'che en ‘off' un Élyséen, à quelques journalistes (dont B2) en novembre 2018. Et d'ajouter : « Nous n'avons pas de détails proposés par le gouvernement allemand : est-ce un forum pour discuter ou pour décider des questions de politique étrangère ? Ce n'est pas encore une position qui est mûrie. » (3) Une idée mal perçue dans les milieux européens Dans les couloirs européens, cette idée est à peine commentée. « Je suis un peu sceptique sur la création d'une nouvelle structure. Est-elle vraiment nécessaire. N'a-t-on pas déjà pas assez de structures » s'interroge un bon connaisseur des questions sécuritaires interrogé par B2, résumant assez bien le sentiment à Bruxelles, perplexe et qui a, à peine, réfléchi sur l'idée. Un vide béant de réflexion stratégique Cette proposition répond pourtant à un réel besoin. L'Union européenne souffre aujourd'hui d'un vide béant d'absence de direction politique au plus haut niveau, d'anticipation stratégique et de réactivité en cas de crise majeure. Parler d'autonomie stratégique ou de réflexion sans avoir une instance capable de décider est un leurre. Des leaders européens absents collectivement Certes, en théorie, le Conseil européen doit se pencher une fois par an au minimum sur les grandes questions de sécurité. Mais cette disposition du Traité de Lisbonne est restée plutôt lettre morte. Force est de constater que ces dernières années, sur toutes les crises majeures — Libye, Syrie, Irak, Ukraine, crise migratoire, coup d'état en Turquie, etc. — les Chefs d'État et de gouvernement européens, collectivement, ont été ‘à la ramasse'. Un manque d'anticipation certain Pour en attester, il suffit de reprendre la liste des crises récentes. Les 28 ont-ils à la veille de signer l'accord d'association avec l'Ukraine clairement évalué les conséquences de cet acte sur les relations avec la Russie, donné leur accord en bonne et due forme ? Ont-ils planifié un dispositif de gestion de crise soit diplomatique, soit militaire en cas d'intervention russe (largement prévisible) ? Lors de la déroute du printemps arabe en Syrie, ont-ils anticipé la crise des réfugiés et des migrants à venir ? Après l'intervention franco-britannique en Libye, qui laisse un pays déchiré et un État failli, ont-ils envisagé et débattu de la solution à apporter à la crise, en commençant par résoudre leurs différends ? Lors du coup d'Etat en Turquie, y-t-a-il eu une réunion de crise par rapport à un pays le plus proche ? Non, non ! Des questions posées trop vite abordées Au mieux, les ‘Leaders' ont discuté une ou deux heures pour s'accorder sur les traitements collatéraux de la crise (rupture des liens diplomatiques, aide humanitaire, sanctions...). La plus longue discussion au cours de ces dernières années a été consacrée à définir l'intensité des sanctions mises en place sur la Russie. Mais rarement pour tenter de résoudre leurs différends, trouver des solutions ou b'tir des feuilles de route. Au pire, ils ont préféré ne pas trop se pencher sur la question. Une réforme facile à mettre en place Si l'on met de côté certains aspects proposés par A. Merkel, avoir un Conseil de sécurité de l'Union européenne est possible dans le cadre existant. Pas de modification de traité Ce projet ne nécessite pas de modification des traités constitutifs. Il suffit juste de changer les usages. On peut décider (par exemple) de consacrer une demi-journée lors de chaque Conseil européen aux grandes questions internationales ou (autre exemple) dédier une de ses quatre réunions annuelles aux questions internationales. Il serait même possible de tenir une ou deux fois par an un Conseil européen informel dans un pays tournant (permettant à un chef de gouvernement de coprésider la réunion). Juste changer les usages Rien n'empêche d'ailleurs quelques pays plus proches en matière d'approche sécuritaire — France, Allemagne, Belgique, Espagne, Italie — de tenir régulièrement des conciliabules préparatoires à l'image des réunions G6 des ministres de l'Intérieur (un petit cercle conjoint). Rien n'empêche aussi de joindre à ces réunions des Chefs, une réunion parallèle des ministres de la Défense ou des Affaires étrangères, voire des ambassadeurs, pour mettre en musique immédiatement les mesures décidées par les Chefs. Toutes ces dispositions, tout à fait possibles dans les traités existants, permettraient de se rapprocher du modèle prôné par A. Merkel. Un dispositif diplomatique et technique prêt à répondre Au-dessous du niveau politique, le dispositif européen en cas de crise est plutôt complet et prêt à travailler. On a ainsi des ambassadeurs des 28 (le Comité politique et de sécurité), qui siègent en permanence à Bruxelles, avec au minimum deux réunions par semaine (sans compter les petits déjeuners, goûters et autres dîners informels) permettant d'échanger et affiner des positions communes. En cas d'urgence, une réunion du COPS peut être improvisée. Ces diplomates, discrets mais parfaits connaisseurs de leurs sujets, sont tenus d'être là, 24h/24 sur le pont. J'en ai été témoin à plusieurs reprises. Des réunions ont eu lieu le dimanche, au mois d'août, à 6 heures du matin ou à 22 heures le soir. Un dispositif de veille et d'analyse On a aussi un dispositif de veille du renseignement (l'IntCen) (dirigé aujourd'hui par un Allemand ancien des services de renseignement) qui produit régulièrement des notes d'analyses. Ces notes — environ 1400 par an — sont plutôt bien appréciées de leurs destinataires, selon mes informations. On peut ajouter à cela des dispositifs de réaction de crise — cellule de protection civile à la Commission européenne, état-major militaire de l'UE (EUMS), commandement des missions civiles (CPCC) etc. — qui existent et ne demandent qu'à produire des résultats. Tous ces dispositifs peuvent au besoin être renforcés et rendus plus performants. (Nicolas Gros-Verheyde) https://www.bruxelles2.eu/2019/02/16/mettre-en-place-un-conseil-de-securite-europeen-une-idee-du-futur/

  • Sikorsky Unveils Raider-X Proposal For FARA Armed Scout

    15 octobre 2019 | International, Aérospatial

    Sikorsky Unveils Raider-X Proposal For FARA Armed Scout

    Graham Warwick Sikorsky is emphasizing growth capability to stay ahead of evolving threats as it unveils its offering in the U.S. Army's Future Attack Reconnaissance Aircraft (FARA) competition, the Raider-X. The coaxial rigid-rotor compound helicopter is a development of the S-97 Raider prototype now in flight testing. The Raider-X is about 20% larger, with a 14,000-lb. gross weight compared with 11,000 lb. for the S-97, and a 39-ft. rotor diameter compared with 34 ft. for the Raider. The Army requires a rotor diameter of no more than 40 ft. so that FARA can fly between buildings in urban combat. The Raider-X will exceed the FARA's threshold maximum speed of 180 kt. Sikorsky is not saying how fast it will fly, but notes the S-97 has reached 215 kt., and 207 kt. in level flight—exceeding the 205 kt. target in the Army's initial capability document, says Tim Malia, director of Future Vertical Lift - Light. The Army subsequently reduced the threshold maximum speed to enable a wider competition for the FARA program. But Malia says the greater speed and payload capability of Sikorsky's X2 coaxial rigid-rotor compound configuration compared with a conventional helicopter provides growth capacity. “We looked at a single main rotor helicopter for FARA, but it would be minimally compliant in the early 2020s. We need to be able to stay ahead of the threat into the 2030s and 2040s,” he says. Bell is proposing a winged, single-main-rotor helicopter for FARA, while AVX Aircraft is offering a coaxial-rotor, ducted-fan compound. Boeing and Karem Aircraft have yet to unveil their concepts. “Single main rotor will not be able to keep up. It's already tapped out meeting the minimal FARA requirements. It does not have the ability to grow capability over time,” Malia says. “We didn't want to pull out all the stops to be minimally compliant when we had X2 able to carry more payload and go faster.” The Raider-X has four-blade rotors and a pusher propulsor. The coaxial rotors generate lift only on the advancing sides, eliminating retreating-blade stall and enabling higher speed. At high speed, 90% of the engine power goes to the propulsor, says Bill Fell, senior experimental test pilot. The propulsor is declutched at low speed to reduce noise. The Raider-X closely resembles the S-97 prototype, with side-by-side seating. Compared with tandem seating, this improves crew coordination and situational awareness, Malia says. “And we can do it aerodynamically because of the robust performance inherent in X2. We are not trying to get out every last ounce of drag.” Behind the cockpit is a large internal weapons bay. Internal carriage of missiles and unmanned aircraft—which the Army calls air-launched effects (ALE)—is a FARA requirement, but Malia says the cabin-like volume of Raider-X's bay provides growth space for future, larger systems. “The minimum threshold works now, but what if in 2030 there is a new ALE that can be decisive, but can't be carried?” he asks. The Raider-X is powered by a single 3,000-shp-class General Electric T901 turboshaft—government-furnished equipment to all FARA bidders. “We do not have an additional engine to increase speed,” Malia says, referring to the supplemental power unit in Bell's 360 Invictus. This augments power from the single T901 to give the single-main-rotor, tandem-seat Invictus a 185-kt. maximum speed. “We use the power available and have a solid design built around it,” he says. “The T901 provides speeds out of the chute in excess of requirements and, as it improves, we can take direct benefit. We have a growth path to additional speed and payload as the T901 power increases.” Sikorsky is using the industry-funded S-97 prototype to reduce risk for its FARA bid, conducting flight testing to validate design models and optimize the Raider-X. This includes flying new rotor blades designed to reduce drag and vibration. “We are getting exactly the results the models said,” Malia says. With two rotor systems and a propulsor, Sikorsky is paying close attention to the Raider-X's cost. “We have done a complete affordability analysis and design to cost. We are extremely confident we will come in under the cost goal,” he says. Several divisions of parent company Lockheed Martin are part of Sikorsky's FARA team, Malia says, including Aeronautics, Missiles and Fire Control and Rotary and Mission Systems. Swift Engineering will build the airframe if Sikorsky wins one of two FARA competitive prototype contracts scheduled to be awarded in March 2020. The Army flyoff is planned for 2023, with the first unit to be equipped by 2028. “There is a critical cap in vertical lift, in attack/reconnaissance, and that gap is really impactful to the Army in the 2020s, ‘30s, ‘40s, even ‘50s. So there is a need for a long-term solution,” Malia says. “The threat is projected to evolve rapidly, so we require significant capability growth on our side to stay ahead. Raider-X can provide an asymmetric advantage in the 2030s-50s.” https://aviationweek.com/defense/sikorsky-unveils-raider-x-proposal-fara-armed-scout

Toutes les nouvelles