21 janvier 2020 | International, C4ISR

Trustworthy AI: A Conversation with NIST's Chuck Romine

By: Charles Romine

Artificial Intelligence (AI) promises to grow the economy and improve our lives, but with these benefits, it also brings new risks that society is grappling with. How can we be sure this new technology is not just innovative and helpful, but also trustworthy, unbiased, and resilient in the face of attack? We sat down with NIST Information Technology Lab Director Chuck Romine to learn how measurement science can help provide answers.

How would you define artificial intelligence? How is it different from regular computing?

One of the challenges with defining artificial intelligence is that if you put 10 people in a room, you get 11 different definitions. It's a moving target. We haven't converged yet on exactly what the definition is, but I think NIST can play an important role here. What we can't do, and what we never do, is go off in a room and think deep thoughts and say we have the definition. We engage the community.

That said, we're using a narrow working definition specifically for the satisfaction of the Executive Order on Maintaining American Leadership in Artificial Intelligence, which makes us responsible for providing guidance to the federal government on how it should engage in the standards arena for AI. We acknowledge that there are multiple definitions out there, but from our perspective, an AI system is one that exhibits reasoning and performs some sort of automated decision-making without the interference of a human.

There's a lot of talk at NIST about “trustworthy” AI. What is trustworthy AI? Why do we need AI systems to be trustworthy?

AI systems will need to exhibit characteristics like resilience, security and privacy if they're going to be useful and people can adopt them without fear. That's what we mean by trustworthy. Our aim is to help ensure these desirable characteristics. We want systems that are capable of either combating cybersecurity attacks, or, perhaps more importantly, at least recognizing when they are being attacked. We need to protect people's privacy. If systems are going to operate in life-or-death type of environments, whether it's in medicine or transportation, people need to be able to trust AI will make the right decisions and not jeopardize their health or well-being.

Resilience is important. An artificial intelligence system needs to be able to fail gracefully. For example, let's say you train an artificial intelligence system to operate in a certain environment. Well, what if the system is taken out of its comfort zone, so to speak? One very real possibility is catastrophic failure. That's clearly not desirable, especially if you have the AI deployed in systems that operate critical infrastructure or our transportation systems. So, if the AI is outside of the boundaries of its nominal operating environment, can it fail in such a way that it doesn't cause a disaster, and can it recover from that in a way that allows it to continue to operate? These are the characteristics that we're looking for in a trustworthy artificial intelligence system.

NIST is supposed to be helping industry before they even know they needed us to. What are we thinking about in this area that is beyond the present state of development of AI?

Industry has a remarkable ability to innovate and to provide new capabilities that people don't even realize that they need or want. And they're doing that now in the AI consumer space. What they don't often do is to combine that push to market with deep thought about how to measure characteristics that are going to be important in the future. And we're talking about, again, privacy, security and resilience ... trustworthiness. Those things are critically important, but many companies that are developing and marketing new AI capabilities and products may not have taken those characteristics into consideration. Ultimately, I think there's a risk of a consumer backlash where people may start saying these things are too easy to compromise and they're betraying too much of my personal information, so get them out of my house.

What we can do to help, and the reason that we've prioritized trustworthy AI, is we can provide that foundational work that people in the consumer space need to manage those risks overall. And I think that the drumbeat for that will get increasingly louder as AI systems begin to be marketed for more than entertainment. Especially at the point when they start to operate critical infrastructure, we're going to need a little more assurance.

That's where NIST can come together with industry to think about those things, and we've already had some conversations with industry about what trustworthy AI means and how we can get there.

I'm often asked, how is it even possible to influence a trillion-dollar, multitrillion-dollar industry on a budget of $150 million? And the answer is, if we were sitting in our offices doing our own work independent of industry, we would never be able to. But that's not what we do. We can work in partnership with industry, and we do that routinely. And they trust us, they're thrilled when we show up, and they're eager to work with us.

AI is a scary idea for some people. They've seen “I, Robot,” or “The Matrix,” or “The Terminator.” What would you say to help them allay these fears?

I think some of this has been overhyped. At the same time, I think it's important to acknowledge that risks are there, and that they can be pretty high if they're not managed ahead of time. For the foreseeable future, however, these systems are going to be too fragile and too dependent on us to worry about them taking over. I think the biggest revolution is not AI taking over, but AI augmenting human intelligence.

We're seeing examples of that now, for instance, in the area of face recognition. The algorithms for face recognition have improved at an astonishing rate over the last seven years. We're now at the point where, under controlled circumstances, the best artificial intelligence algorithms perform on par with the best human face recognizers. A fascinating thing we learned recently, and published in a report, is that if you take two trained human face recognizers and put them together, the dual system doesn't perform appreciably better than either one of them alone. If you take two top-performing algorithms, the combination of the two doesn't really perform much better than either one of them alone. But if you put the best algorithm together with a trained recognizer, that system performs substantially better than either one of them alone. So, I think, human augmentation by AI is going to be the revolution.

What's next?

I think one of the things that is going to be necessary for us is pulling out the desirable characteristics like usability, interoperability, resilience, security, privacy and all the things that will require a certain amount of care to build into the systems, and get innovators to start incorporating them. Guidance and standards can help to do that.

Last year, we published our plan for how the federal government should engage in the AI standards development process. I think there's general agreement that guidance will be needed for interoperability, security, reliability, robustness, these characteristics that we want AI systems to exhibit if they're going to be trusted.

https://www.nist.gov/blogs/taking-measure/trustworthy-ai-conversation-nists-chuck-romine

Sur le même sujet

24 février 2020 | International, Aérospatial, Naval, Terrestre, C4ISR, Sécurité

Contract Awards by US Department of Defense - February 20, 2020
NAVY Lockheed Martin Rotary and Mission Systems, Baltimore, Maryland, was awarded a $233,036,890 firm-fixed-price undefinitized contract action for the procurement of Mk 41 Vertical Launching System vertical launcher module assemblies, modernization kits and spare components. This contract combines purchases for the Navy (74%) and the governments of Korea (18%), Finland (4%), and Germany (4%) under the Foreign Military Sales program. Work will be performed in Baltimore, Maryland (40%); Indianapolis, Indiana (36%); Farmingdale, New York (9%); Saginaw, Michigan (5%); Waverly, Iowa (2%); Thomaston, Connecticut (2%); Chaska, Minnesota (2%); St. Peters, Missouri (1%); Hampstead, Maryland (1%); Santa Rosa, California (1%), and Peachtree City, Georgia (1%), and is expected to be completed by March 2025. Fiscal 2018, 2019 and 2020 shipbuilding and conversion (Navy) and foreign military sales funding in the amount of $46,607,377 was obligated at time of award and will not expire at the end of the current fiscal year. This contract was competitively procured via the Federal Business Opportunities website, with one offer received. The Naval Sea Systems Command, Washington, District of Columbia, is the contracting activity (N00024-20-C-5310). (Awarded Feb. 14, 2020) Sikorsky Aircraft Corp. a Lockheed Martin Co., Stratford, Connecticut, is awarded an $11,967,528 modification (P00003) to a cost-plus-fixed-fee delivery order (N00019-19-F-2972) against a previously issued basic ordering agreement (N00019-19-G-0029). This modification provides non-recurring engineering to replace existing subsystems, such as the Data Transfer Unit, Defense Electronic Countermeasure System and ARC-210 radio, with the CH-53K production aircraft. Non-recurring engineering efforts include investigation, systems engineering support, risk analysis, integration development, weight impact, publication updates including maintenance, training, update tooling and qualification testing. Work will be performed in Cedar Rapids, Iowa (55.82%); Stratford, Connecticut (35.7%) and Fort Worth, Texas (8.48%), and is expected to be completed in August 2021. Fiscal 2018 aircraft procurement (Navy) funds in the amount of $11,967,528 will be obligated at time of award, all of which will expire at the end of the current fiscal year. The Naval Air Systems Command, Patuxent River, Maryland, is the contracting activity. Bath Iron Works, Bath Maine, is awarded an $8,462,959 fixed-price incentive (firm target) modification to previously awarded contract N00024-14-C-2305 to exercise an option for accomplishment of post-delivery availability work items for Guided Missile Destroyer 118. Work will be performed in Bath, Maine (93%), Brunswick, Maine (6%), and other locations below 1 % (collectively totaling less than 1%) and is expected to be completed by February 2021. Fiscal 2013 shipbuilding and conversion (Navy) and fiscal 2013 other procurement (Navy) funding in the amount of $8,462,959 will be obligated at time of award and funding in the amount of $1,962,124 will expire at the end of the current fiscal year. The Naval Sea Systems Command, Washington, District of Columbia, is the contracting activity. AIR FORCE Dark Wolf Solutions LLC, Chantilly, Virginia, has been awarded a $75,000,000 blanket purchase agreement for cyber innovation services. This award was made from GSA Professional Services Schedule number: GS-00F-086GA. The contractor is a Phase III, Small Business Innovation and Research program participant. The location of performance is Hill Air Force Base, Utah. The work to be conducted includes software penetration testing and adversarial assessment. The work is expected to be complete by Oct. 19, 2020. Fiscal 2020 research development test and evaluation funds in the amount of $7,093,286 are being obligated at the time of award via task order FA7014-20-F-0041. Air Force District of Washington, Joint Base Andrews, Maryland, is the contracting activity (FA7014-20-F-0041). Williams Aerospace & Manufacturing (formerly Merex Aircraft Co.), a Kellstrom Defense company, has been awarded a $40,000,000 ceiling indefinite-delivery/indefinite-quantity contract for the acquisition of A-10 Speed Brake assemblies. This contract provides for upper and lower left speed brakes assembly and upper and lower right speed brakes assembly. Work will be performed in Camarillo, California, and is expected to be complete by February 2026. This award is the result of a competitive acquisition and three offers were received. Consolidated sustainment activity group working capital funds in the amount of $10,501,432 are being obligated at the time of award. The Air Force Sustainment Center, Hill Air Force Base, Utah, is the contracting activity (FA8212-20-D-0001). Chemring Energetic Devices Inc., Downers Grove, Illinois, has been awarded a $24,000,000 indefinite-delivery/indefinite-quantity contract for supply of various Cartridge Actuated Devices/Propellant Actuated Devices for various U.S. and Foreign Military Sales (FMS) aircraft. Work will be performed in Downers Grove, Illinois, and is expected to be completed by August 2027. The contract involves foreign military sales to Greece, Taiwan, Thailand, Switzerland, Uruguay, India, Saudi Arabia, Israel, United Arab Emirates, Belgium, Denmark, Indonesia, Iraq, Jordan, South Korea, Oman, Pakistan, Portugal, Romania, Singapore, Turkey, Poland, Bahrain, Tunisia, Egypt and The Netherlands. This award is the result of a sole-source acquisition. Air Force fiscal 2018 funding in the amount of $1,882,272; Air Force fiscal 2019 funding in the amount of $1,065,552; U.S. Navy fiscal year 2019 funding in the amount of $134,032; Foreign Military Sales funding in the amount of $2,539,705 are being obligated at the time of award under delivery order FA8213-20-F-2525. The Air Force Life Cycle Management Center, Hill Air Force Base, Utah, is the contracting activity (FA8213-20-D-0002). C. Martin Company Inc., North Las Vegas, Nevada, has been awarded a $7,462,086 option exercise modification (P00007) to previously awarded contract FA8601-18-D-0004. This modification provides for the exercise of an option for additional facility and equipment support under the basic contract. The location of performance is Wright-Patterson Air Force Base, Ohio. The work is expected to be completed by March 31, 2021. Fiscal 2020 research development test and evaluation funds are being obligated shortly after award on a task order. The total cumulative face value of the contract is $21,941,348. Wright-Patterson Air Force Base, Ohio, is the contracting activity. DEFENSE LOGISTICS AGENCY Stonewin LLC,* New York, New York, has been awarded a minimum $59,678,523 fixed-price with economic-price-adjustment contract for marine gas oil. This was a competitive acquisition with 41 responses received. This is a 56-month base contract with one six-month option period. Locations of performances are Alabama, California, Florida, Massachusetts, Texas and South Carolina, with an Oct. 31, 2024, performance completion date. Using customers are Army, Navy, Coast Guard and federal civilian agencies. Type of appropriation is fiscal 2020 through 2025 defense working capital funds. The contracting activity is the Defense Logistics Agency Energy, Fort Belvoir, Virginia (SPE608-20-D-0350). EMIT Corp., Houston, Texas, has been awarded a maximum $20,000,000 fixed-price with economic-price-adjustment, indefinite-delivery/indefinite-quantity contract for hospital equipment and accessories for the Defense Logistics Agency electronic catalog. This was a competitive acquisition with 104 responses received. This is a five-year contract with no option periods. Location of performance is Texas, with a Feb. 19, 2025, performance completion date. Using military services are Army, Navy, Air Force and Marine Corps. Type of appropriation is fiscal 2020 through 2025 defense working capital funds. The contracting activity is the Defense Logistics Agency Troop Support, Philadelphia, Pennsylvania (SPE2DH-20-D-0030). ARMY K S Ware & Associates LLC, Nashville, Tennessee, was awarded a $12,500,000 firm-fixed-price contract for value engineering services. Bids were solicited via the internet with seven received. Work locations and funding will be determined with each order, with an estimated completion date of Feb. 19, 2025. U.S. Army Corps of Engineers, Fort Worth, Texas, is the contracting activity (W9126G-20-D-0012). *Small business https://www.defense.gov/Newsroom/Contracts/Contract/Article/2089858/source/GovDelivery/
16 février 2023 | International, C4ISR

Space Command crafting requirements to improve satellite mobility
Lt. Gen. John Shaw says the need for mobility is driving the command to explore options for what he calls "dynamic space operations."
6 février 2019 | International, Aérospatial, Naval, Terrestre, C4ISR, Sécurité, Autre défense

Germany’s plan to boost defense spending hits a snag
By: Sebastian Sprenger COLOGNE, Germany — Germany may be unable to deliver on its pledge to increase the defense budget due to smaller-than-expected economic growth, according to a new Finance Ministry analysis. The projections peg the military budget to be several billion euros short of the trajectory to meet the government's goal of reaching 1.5 percent of gross domestic product by 2024. Analysts even see the current spending curve unable to sustain 1.35 percent in the years ahead. NATO members in 2014 agreed to boost their defense spending to 2 percent of GDP within 10 years. Germany's defense budget is roughly €43 billion (U.S. $49 billion) for 2019, or about 1.2 percent of GDP. That is a boost of €4 billion over the previous year. Defence Minister Ursula von der Leyen on Monday said Germany remains committed to hitting the self-declared 1.5 percent target in 2024. She portrayed the Finance Ministry's analysis as a mere first step toward a budget proposal negotiated by Cabinet secretaries. The government is expected to unveil such a plan in late March. The Trump administration has often criticized Germany for underspending on defense, arguing Berlin rides on American coattails when it comes to security. News that the country's spending target is at risk is sure to embolden the narrative in Washington that Europe is somehow taking advantage of the United States. It could weaken the negotiating position of German government delegates at two high-profile events in mid-February: a meeting of NATO defense ministers in Brussels, and the Munich Security Conference. The Finance Ministry's economic outlook estimates that agencies will have to reconcile new spending priorities within their previously established budget targets. That means no fresh money would become available for the government's push on artificial intelligence, for example, according to the document. https://www.defensenews.com/global/europe/2019/02/05/germanys-plan-to-boost-defense-spending-hits-a-snag/

Toutes les nouvelles