21 janvier 2020 | International, C4ISR

Trustworthy AI: A Conversation with NIST's Chuck Romine

By: Charles Romine

Artificial Intelligence (AI) promises to grow the economy and improve our lives, but with these benefits, it also brings new risks that society is grappling with. How can we be sure this new technology is not just innovative and helpful, but also trustworthy, unbiased, and resilient in the face of attack? We sat down with NIST Information Technology Lab Director Chuck Romine to learn how measurement science can help provide answers.

How would you define artificial intelligence? How is it different from regular computing?

One of the challenges with defining artificial intelligence is that if you put 10 people in a room, you get 11 different definitions. It's a moving target. We haven't converged yet on exactly what the definition is, but I think NIST can play an important role here. What we can't do, and what we never do, is go off in a room and think deep thoughts and say we have the definition. We engage the community.

That said, we're using a narrow working definition specifically for the satisfaction of the Executive Order on Maintaining American Leadership in Artificial Intelligence, which makes us responsible for providing guidance to the federal government on how it should engage in the standards arena for AI. We acknowledge that there are multiple definitions out there, but from our perspective, an AI system is one that exhibits reasoning and performs some sort of automated decision-making without the interference of a human.

There's a lot of talk at NIST about “trustworthy” AI. What is trustworthy AI? Why do we need AI systems to be trustworthy?

AI systems will need to exhibit characteristics like resilience, security and privacy if they're going to be useful and people can adopt them without fear. That's what we mean by trustworthy. Our aim is to help ensure these desirable characteristics. We want systems that are capable of either combating cybersecurity attacks, or, perhaps more importantly, at least recognizing when they are being attacked. We need to protect people's privacy. If systems are going to operate in life-or-death type of environments, whether it's in medicine or transportation, people need to be able to trust AI will make the right decisions and not jeopardize their health or well-being.

Resilience is important. An artificial intelligence system needs to be able to fail gracefully. For example, let's say you train an artificial intelligence system to operate in a certain environment. Well, what if the system is taken out of its comfort zone, so to speak? One very real possibility is catastrophic failure. That's clearly not desirable, especially if you have the AI deployed in systems that operate critical infrastructure or our transportation systems. So, if the AI is outside of the boundaries of its nominal operating environment, can it fail in such a way that it doesn't cause a disaster, and can it recover from that in a way that allows it to continue to operate? These are the characteristics that we're looking for in a trustworthy artificial intelligence system.

NIST is supposed to be helping industry before they even know they needed us to. What are we thinking about in this area that is beyond the present state of development of AI?

Industry has a remarkable ability to innovate and to provide new capabilities that people don't even realize that they need or want. And they're doing that now in the AI consumer space. What they don't often do is to combine that push to market with deep thought about how to measure characteristics that are going to be important in the future. And we're talking about, again, privacy, security and resilience ... trustworthiness. Those things are critically important, but many companies that are developing and marketing new AI capabilities and products may not have taken those characteristics into consideration. Ultimately, I think there's a risk of a consumer backlash where people may start saying these things are too easy to compromise and they're betraying too much of my personal information, so get them out of my house.

What we can do to help, and the reason that we've prioritized trustworthy AI, is we can provide that foundational work that people in the consumer space need to manage those risks overall. And I think that the drumbeat for that will get increasingly louder as AI systems begin to be marketed for more than entertainment. Especially at the point when they start to operate critical infrastructure, we're going to need a little more assurance.

That's where NIST can come together with industry to think about those things, and we've already had some conversations with industry about what trustworthy AI means and how we can get there.

I'm often asked, how is it even possible to influence a trillion-dollar, multitrillion-dollar industry on a budget of $150 million? And the answer is, if we were sitting in our offices doing our own work independent of industry, we would never be able to. But that's not what we do. We can work in partnership with industry, and we do that routinely. And they trust us, they're thrilled when we show up, and they're eager to work with us.

AI is a scary idea for some people. They've seen “I, Robot,” or “The Matrix,” or “The Terminator.” What would you say to help them allay these fears?

I think some of this has been overhyped. At the same time, I think it's important to acknowledge that risks are there, and that they can be pretty high if they're not managed ahead of time. For the foreseeable future, however, these systems are going to be too fragile and too dependent on us to worry about them taking over. I think the biggest revolution is not AI taking over, but AI augmenting human intelligence.

We're seeing examples of that now, for instance, in the area of face recognition. The algorithms for face recognition have improved at an astonishing rate over the last seven years. We're now at the point where, under controlled circumstances, the best artificial intelligence algorithms perform on par with the best human face recognizers. A fascinating thing we learned recently, and published in a report, is that if you take two trained human face recognizers and put them together, the dual system doesn't perform appreciably better than either one of them alone. If you take two top-performing algorithms, the combination of the two doesn't really perform much better than either one of them alone. But if you put the best algorithm together with a trained recognizer, that system performs substantially better than either one of them alone. So, I think, human augmentation by AI is going to be the revolution.

What's next?

I think one of the things that is going to be necessary for us is pulling out the desirable characteristics like usability, interoperability, resilience, security, privacy and all the things that will require a certain amount of care to build into the systems, and get innovators to start incorporating them. Guidance and standards can help to do that.

Last year, we published our plan for how the federal government should engage in the AI standards development process. I think there's general agreement that guidance will be needed for interoperability, security, reliability, robustness, these characteristics that we want AI systems to exhibit if they're going to be trusted.

https://www.nist.gov/blogs/taking-measure/trustworthy-ai-conversation-nists-chuck-romine

Sur le même sujet

11 janvier 2021 | International, Aérospatial, Naval, Terrestre, C4ISR, Sécurité

UK defense secretary: Britain is paving a path for modernization and appropriate funding
By: Ben Wallace The Estonian town of Tapa sits less than 100 miles from the Russian border, and in December I was there to visit some of the 900 British troops that have been stationed there since 2017 — our largest land deployment outside of the U.K. This is not just a useful training exercise with our Danish and, in a few months' time, French allies, but rather a strategic defensive presence in a region that is vital for global security. Our Estonian allies are bolstered and reassured by having us there. A more active, more deployed armed forces, such as those in Tapa, is a sign of things to come for U.K. defense. Like Estonia, we meet the 2 percent commitment to NATO, and in the U.K.'s latest spending review, the Ministry of Defence secured a record settlement of £16.5 billion (U.S. $22.4 billion) of funding above our election manifesto commitment over a four-year period. The prime minister and I share a vision for how that funding will transform U.K. defense. It is crucial to putting our defense spending on a sustainable footing — living within our means, addressing the underfunding of previous years and paving the way for a modernization that is much overdue. It means being an even greater and ever-reliable defense ally to our friends around the world. It means adopting a more proactive posture with our forces more forward, more present and more assertive. It means remaining a leader in NATO, spending above 2 percent of gross domestic product, making the largest single commitment to the Readiness Initiative and helping drive the modernization of an organization that has kept us safe for more than 70 years. And, of course, it means remaining the United States' most reliable, capable and committed ally. It is not just a coincidence that this is the biggest defense investment since the end of the Cold War. Estonians know this only too keenly — and with an increase in Russian presence in the U.K., we have felt this too. Our Quick Reaction Alert forces have seen their busiest period in a decade, with our Royal Air Force fighters scrambling 11 times to intercept Russian warplanes. Meanwhile, the Kremlin's activity in U.K. waters has risen by 26 percent since last year, with Royal Navy vessels escorting each and every one of them. From our airspace to cyberspace, the North Sea to the High North, we know the threat they pose. So in an age of 21st century challenges, it's more important than ever that we work together. That's why, following our departure from the European Union, we are opening up fresh opportunities to strengthen our global relationships and stay ahead of the curve. The integrated review that we will publish in 2021 will make the most of new technologies, improve integration across the domains and demonstrate that we remain the international partner of choice: a burden-sharing, self-confident and active nation, stepping up to our responsibilities in an ever more contested world. Ben Wallace is Britain's secretary of state for defense. https://www.defensenews.com/outlook/2021/01/11/uk-defense-secretary-britain-is-paving-a-path-for-modernization-and-appropriate-funding/
11 février 2020 | International, Aérospatial, Naval, Terrestre, C4ISR, Sécurité

Pentagon budget request increases R&D funding, cuts legacy planes
By: Aaron Mehta WASHINGTON — U.S. President Donald Trump's defense budget request for fiscal 2021 includes major investments in research and development portfolios as well as “crucial” technologies as part of what the Pentagon is branding an “irreversible implementation” of the National Defense Strategy. However, the budget also features overall cuts to the Army and Navy top lines, as well as the divestment of legacy platforms from the Air Force. The president is requesting $705 billion for the Defense Department, including $69 billion in overseas contingency operations, or OCO, wartime funds. Total national security spending, including for the National Nuclear Security Administration and other outside agencies, is $740 billion, as set by a congressional budget agreement last year. Although not included in the budget documents, total top-line projections over the Future Years Defense Program, or FYDP, are $722 billion in FY22, $737 billion in FY23, $753 billion in FY24 and $768 billion in FY25, according to a senior defense official. Service budget top lines are $178 billion for the Army, a drop by $462 million from FY20 enacted levels; $207 billion for the Navy, down $1.9 billion from FY20; and $207 billion for the Air Force, up $1.7 billion from FY20. The budget also requests $113 billion for defensewide efforts, which includes the so-called fourth estate agencies, down $6.5 billion from FY20. Overall procurement funding sits at $136.9 billion. The OCO request of $69 billion is down dramatically from last year's $164 billion, and it comes in three flavors: $20.5 billion in “direct war requirements,” or funding for combat operations that will end at some point in locations like Iraq and Syria. $32.5 billion in “enduring requirements,” which covers funding for the sustainment of bases, as well as pots of money like the European Deterrence Initiative. $16 billion in “OCO for base,” a funding mechanism for money that could be in the base budget but is classified as OCO for the purpose of skirting budget caps imposed by Congress. Projection for OCO funding falls $20 billion in FY22 and FY23, and then to $10 billion for FY24 and FY25, as “certain OCO costs” are absorbed by the base budget, according to the White House's summary tables. There's no nondefense discretionary OCO proposed for FY21 or the out years. “This is a budget that makes difficult choices but they are actually choices that support the National Defense Strategy,” a senior defense official said on condition of anonymity ahead the budget rollout. “We can't have the best of everything in all areas,” the official added. “The low-hanging fruit is gone.” Among the tough choices: retiring 17 B-1 bombers, 44 A-10 planes, 24 Global Hawk drones, as well as 16 KC-10 and 13 KC-135 tankers from the Air Force. “When you look at these aircraft, they disproportionately take too much of the readiness account. That's where we've got to go,” the official said. “Those are really the tough choices we had to make. Because we can now take the additional manpower, the [spare parts], all those things we need to make those other aircraft more operationally available and have more flight hours available in the mission we need them to do.” Congress usually revises presidential budget submissions substantially before passing them into law. A prime target for lawmakers this year will be the Trump administration's favoritism for defense spending over nondefense, which contradicts the rough parity between two that's characterized bipartisan budget deals in recent years. Congress will also likely upend the administration's FY21 proposal to cut the nondefense base budget by 5.1 percent while adding 0.08 percent to the base defense budget. There are slim odds for Trump's proposal extending budget caps — set to expire next year — through 2025, wherein defense would increase by roughly 2 percent each year as nondefense discretionary decreases 2 percent each year. ‘Irreversible' Budget documents were branded with the phrase “irreversible implementation of the National Defense Strategy,” a notable signal in an election year that, should Trump not be reelected, could result in major changes to the national budget and American strategy come January. The branding in support of the NDS can be found throughout the document, even at lower levels. For instance, the Pentagon's security cooperation account has been rebranded the “NDS Implementation (NDS-I) account.” Missing from the budget request are funds for Trump's border wall with Mexico. However, CNN reported this weekend that “billions” of defense dollars will be going toward the wall effort, with an announcement expected later this week. Key defense spending accounts break down like this: Mission-support activities: $66.8 billion Aircraft and related systems: $56.9 billion Shipbuilding and maritime systems: $32.2 billion Missiles and munitions: $21.3 billion Space-based systems: $15.5 billion Ground systems: $13 billion C4I systems: $11.9 billion Missile defeat and defense programs: $11.6 billion The department is requesting $106.6 billion to fund research, development, test and evaluation (RDT&E) efforts, an increase of $2 billion over the FY20 enacted figures — something another senior defense official called the “largest [RDT&E] request in over 70 years.” Funding for that came from savings from the defensewide review, which found $5.7 billion in money to reprogram in FY21, as well as the retirement of older platforms. Four “crucial” technologies are now bunched together under a new acronym — ACE, which stands for advanced capability enablers: hypersonics at $3.2 billion, microelectronics/5G at $1.5 billion, autonomy at $1.7 billion, and artificial intelligence at $800 million. However, for the second straight year, science and technology funding for early technology development (the Pentagon's 6.1, 6.2 and 6.3 accounts) is requested at $14.1 billion; that includes $3.5 billion for the Defense Advanced Research Projects Agency. Congress plussed that funding to $16.1 billion in FY20 enacted levels, meaning the request here is $2 billion less than what the Pentagon received this current year. Cyber activates total $9.8 billion, including $5.4 billion for cybersecurity-focused projects. The rest of the funding goes toward supporting defensive cyber operations. https://www.defensenews.com/smr/federal-budget/2020/02/10/pentagon-budget-request-increases-rd-funding-cuts-legacy-planes/
15 juin 2023 | International, Aérospatial

Wittman proposes way to keep Next-Generation Air Dominance on track
Congress wants annual updates on the Next-Generation Air Dominance fighter program and affiliated efforts, to ensure the Air Force and Navy stay on track.

Toutes les nouvelles