25 janvier 2024 | International, Naval
No more ‘must-wins’: Defense firms growing warier of fixed-price deals
Multiple defense firms are slamming fixed-price contracts as bad deals, and warning they might walk away from such competitions.
19 septembre 2018 | International, C4ISR
By: Daniel Cebul
It's well known that satellites and other intelligence, surveillance and reconnaissance platforms collect more data than is possible for humans to analyze.
To tackle this problem, the Intelligence Advanced Research Projects Activity, or IARPA, conducted the Functional Map of the World (fMoW) TopCoder challenge from July 2017 through February 2018, inviting researchers in industry and academia to develop deep learning algorithms capable of scanning and identifying different classes of objects in satellite imagery. IARPA curated a dataset of 1 million annotated, high-resolution satellite images aggregated using automated algorithms and crowd sourced images for competitors to train their algorithms to classify objects into 63 classes, such as airports, schools, oil wells, shipyards, or ports.
Researchers powered their deep learning algorithms by combining large neural networks, known as convolutional neural networks (CNNs), and computers with large amounts of processing power. The result was a network that, when fed massive amounts of training data, can learn to identify and classify various objects from satellite imagery. By combining a number of these networks into what is called an ensemble, the algorithm can judge the results from each CNN to produce a final, improved result that is more robust than any single CNN.
This is how a team from Lockheed Martin, led by Mark Pritt, designed their deep learning algorithm for the challenge. Pritt explained to C4ISRNET, that he and his team developed their CNN using machine learning software and framework from online open source software libraries, such as Tensor Flow. Earning a top five finish, the algorithm designed by Pritt's team achieved a total accuracy of 83 percent, and was able to classify 100 objects per second. Pritt said that with fully functioning algorithm, this software could take an image recognition task that takes a human an hour to complete and reduce the process to a few seconds.
The team's algorithm excelled at identifying classes with distinctive features, and successfully matched nuclear power plants, tunnel openings, runways, tool booths, and wind farms with accuracies greater than 95 percent, but struggled with more indiscreet classes such as shipyards and ports, hospitals, office buildings, and police stations.
“Usually when you develop an algorithm its nice to see where it succeeds, but you actually learn the most where you look at where the algorithm fails or it doesn't do well,” Pritt said. In trying to decipher why the algorithms struggled, Pritt said the competitors suggested that some objects simply don't have any distinguishing features from the point of view of a satellite image for the algorithms to recognize.
“Maybe the most important ingredient you need for these new types of algorithm to work is the dataset because these algorithms require a great amount of data to train on,” Pritt explained. “It's kind of analogous to the way a human will learn in childhood how to recognize things. You need lots of examples of what those things are and then you can start to generalize and make your own judgments,” he said.
But even with large amounts of training data that is correctly labeled, it is also possible the deep learning technology of today cannot reach the higher levels of intelligence to recognize nuanced differences. For example, Lockheed Martin's algorithm confused shipyards and ports 56 percent of the time. Pritt said that people “look at an image and they can tell that it's a port or a shipyard, they are usually looking at very subtle things such as if there is a ship in dry dock or if there is a certain type of crane present. They are looking for details in the image that are maybe higher level or more complicated than what these deep learning algorithms can do right now.”
However, the fact that these algorithms cannot do everything should not dismiss the significant contribution they could provide to the defense and intelligence community.
Hakjae Kim, IARPA's program manager for the fMoW challenge, said the benefits of this technology could extend far beyond faster image processing. “I want to look at it more in the perspective that we can do things we weren't able to do before,” Kim said. “Because its technology that we are now able to do x, y and z, there are more applications you can create because with the human power it is just impossible to do before.”
Kim and Pritt stressed managing expectations for CNN-based artificial intelligence.
“This is a real technology that will work, but it also has limitations. I don't want to express this technology as a magic box that will just solve everything magically,” Kim said. “I don't want the users in the field to get disappointed by the initial delivery of this technology and say 'Oh, this is another technology that was oversold and this is not something we can use," he added.
Part of managing our expectations for AI requires recognizing that although intelligence is in the name, this technology does not think and reason like humans. “A lot of the time we think that because we use the term AI, we tend to think these algorithms are like us, they are intelligent like us,” Pritt said. “And in someways they seem to mimic our intelligence, but when they fail we realize ‘Oh, this algorithm doesn't really know anything, [it] doesn't have any common sense.'”
So how are IARPA and Lockheed Martin working to improve their algorithms? For IARPA, Kim's team is working on updating and maintaining their dataset to ensure algorithms have the most up to date information to train on, ultimately making the CNN-based algorithms easier to trust. “[S]ubtle changes in the area mess up the brains of the system and that system will give you a totally wrong answer,” Kim explained. “So we have planned to continuously look over the area and make sure the algorithm we are developing and reassessing for the government to test on and use to be robust enough for their application," he furthered.
Work is also underway at American universities. Kim described how a team of researchers at Boston University are using the fMoW dataset and tested algorithms to create heat maps that visualize what part of the image algorithms are using to classify objects. They've found that sometimes it is not the object itself, but clues surrounding the object that aid most in classification. For example a “windmill that actually shows a shadow gives a really good indicator of what that object is,” Kim said. “Shadows show a better view of the object. A shadow is casting the side view of the object over on the ground, so [BU's heat map algorithm] actually points out the shadow is really important and the key feature to make the object identified as a windmill.”
But don't expect these algorithms to take away the jobs of analysts any time soon. “I think you still need a human doing the important judgments and kind of higher level thinking,” Pritt said. “I don't think AI will take away our jobs and replace humans, but I think what we have to do is figure out how to use them as a tool and how to use them efficiently, and that of course requires understanding what they do well and what they do poorly," he concluded.
25 janvier 2024 | International, Naval
Multiple defense firms are slamming fixed-price contracts as bad deals, and warning they might walk away from such competitions.
5 décembre 2024 | International, Aérospatial
Thanks to its versatility, ruggedness, state-of-the-art communication, and self-protection systems, the aircraft has a great potential to meet and even exceed the operational requirements not only the Brazilian Air Force,...
9 juillet 2019 | International, Naval
By: David B. Larter JOINT EXPEDITIONARY BASE LITTLE CREEK, Va. — The Mark VI patrol boat bristles with heavy automatic weapons, and that's the way its crews like it. “I tell the crews that you want to look like a porcupine,” said U.S. Navy Senior Chief Derrick Cox, who trains the sailors that man the Mark VI as part of Coastal Riverine Squadron 2's training evaluation unit. “You don't want to kick a porcupine because you know there will be consequences.” The Mark VI is a replacement for the Riverine Command Boat, which gained notoriety three years ago when two of them, along with their crews, were captured by the Iranian Revolutionary Guard when they strayed into Iranian waters near Farsi Island in the Arabian Gulf. “This has double, maybe even triple the firepower of the RCB,” Cox said. The RCBs had four mounts that could support a number of heavy weapons to defend the boat. The Mark VI is in another league all together. The boat shown to Defense News this month packed two stabilized, remote-operated, optically guided MK 50, .50-caliber Gun Weapon Systems; two MK 38 Mod 2 (25mm) Gun Weapon Systems (also remotely operated with an advanced optics system); and two crew-served .50-caliber machine guns. “We've demonstrated that we can sustain a firefight for 45 minutes in the Mark VI,” Cox said. The Mark VI was just coming online in January 2016 when the incident at Farsi Island went down — the last of the 12 were delivered by the end of 2018. And though ultimately none of the 10 captured sailors were hurt — they were released along with their RCBs after the personal intervention of then-Secretary of State John Kerry — the incident was deeply embarrassing for the Navy and infuriating for senior leadership. It prompted the service to refocus the Riverines' mission and change the mindset. Navy Expeditionary Combat Command, the parent command of the Riverines, dispensed training for offensive operations and refocused the Coastal Riverine Force toward port and infrastructure security, high-value unit escort missions, and other such anti-terrorism, force protection missions. It also forced the Riverine crews to get serious about their jobs, Cox said. “Our missions were in more permissive environments, and a lot of our training was tailored toward that,” he said. “Now we are geared toward non-permissive environments.” The head of NECC, Rear Adm. Brian Brakke, is taking that new focus and his new Mark VI platform and turning toward the challenge at hand: great power competition. “Where I would like to go is where do we have the opportunity inside the littorals to be able to conduct missions for the Navy that may free up capital assets to go do other missions,” Brakke said. The new Mark VI has a communications suite that well exceeds that of the RCB. The new boat can connect with the fleet via Link 16; it also has high- and ultrahigh-frequency and satellite comms so shore side controllers won't lose track of the boat, as happened during the Farsi Island incident. The 85-foot boat has a top speed of more than 40 knots and a range of up to 500 miles. It has a 10-person crew and can accommodate up to 20 personnel — the RCB maxed out at 15 personnel. Among the possibilities Brakke is looking at: using the Mark VI as a mothership for swarm attacks in conjunction with new 40-foot patrol boats being introduced to the force; operating unmanned aircraft for over-the-horizon surveillance; operating autonomous wave runners for various missions. The boat already comes with a ramp and rails to launch sleds for autonomous vehicles, which means the boats can be employed for mine countermeasures operations in the littorals. The launch capability is an area that needs improvement but is full of potential, Brakke said. “From a force-development and innovation perspective, that's where we are taking a look at how to grow this force and what it can do for us,” Brakke said. “We understood when they developed the craft that if it's going to have that interaction with the fleet, we have to be able to communicate. So having SATCOMs, Link 16, being able to come up on blue-force tracker so that we know where they are, and they know where we are, that was a big piece of this.” Fixing what's broken The 2016 incident with Iran also forced significant organizational changes into the Coastal Riverine Forces. Since the Farsi Island debacle, NECC has worked to close the gaps that contributed to the incident. Up and down the chain of command, the Navy has boosted and improved training and communication, said Cmdr. Mike Ray, the head of Coastal Riverine Group 1, during a June 26 interview. Watch standers in the operations centers that lost track of the RCBs as they got lost and bumbled their way into Iranian waters receive more training and are more fully appraised on the individual boat crews — their strengths and weaknesses, and what exactly they've been trained to do. The turnover process in theater has also been greatly improved and is overseen by both the operational command in theater and by the administrative command, which is responsible for providing a manned, trained and equipped force to the operators. “One of the specific deficiencies that was brought up in the Farsi investigation was that the [turnover] process allowed the incoming squadron and those forces to be ill-prepared to understand some of the theater-specific requirements or changes since the last time they deployed," Ray said. "So every time a squadron turns over now, you've got representatives of my staff or my counterpart in Little Creek from Group 2 who go over there to essentially shepherd that process hand in hand with the [operational control] task force commander to ensure that once the handover is complete, that new squadron ... is fully prepared to execute in that theater.” The improved turnover process gets after what was one of the more controversial questions in the aftermath of the Farsi Island incident: Was that unit ill-prepared when it entered theater, or did it degrade while it was in theater? Vice Adm. Kevin Donegan, then commander of 5th Fleet, determined that the Riverine crews were insufficiently trained prior to departing for deployment. But others argued that Commander Task Force 56 tasked the Riverine crews with a mission for which they weren't properly trained — a long, open-ocean transit from Kuwait to Bahrain — and that the sailors had been in theater for months by the time disaster struck, well removed from their training. The improved turnover then, ideally, cuts out the finger-pointing and ensures that both the relevant task force knows the unit it's getting and are fully briefed on the training and capabilities. Communication between the task force and the trainers back in the states during the deployment has also increased, Ray said. Even though the Riverines are turned over to the operational chain of command in theater, his group is still available to offer expertise and guidance on employment of their units on a mission-by-mission basis. “We backstop them by providing Coastal Riverine Force-specific expertise in those planning briefs. Although they are [in operational control], they reach back to us a lot to provide oversight and review of those to ensure nothing is missed,” Ray said. ‘Ready to fight' One of the more troubling aspects of the Farsi Island incident for the U.S. Navy was the behavior of the crew members, both in allowing themselves to be captured and in the leader of the voyage apologizing to Iran on camera. Leaders in the Riverine force seem to chalk that behavior up to complacency and a lack of the proper mindset in a contested environment. “We spend a significant amount of time talking about a combat mindset — understanding that contact with the enemy is a plausible scenario that they need to be ready for mentally, physically and tactically," Ray said. “I just had the privilege of sending some sailors off to Bahrain to go do security, and it suffices to say that the 5th Fleet theater is contested," he added. "And I'll tell you that every one of those sailors understood that they were headed into an environment where they potentially could be called up to employ those weapons skills, mental toughness, in a contested environment. “So, not to say that there was any goodness that came out of the Farsi Island incident, but it served to reinforce the reality to our sailors that any given sailor, on any given day, perhaps is going to be called upon to demonstrate that toughness, to demonstrate that seriousness of purpose and demonstrate that tactical training: It was a good reminder for all of our sailors of the environment in which they serve.” Brakke, the NECC commander, put it even more directly. “And a lot of times when we talk to the units and the squadrons before they deploy we tell them that you have to be ready to expect that not all of you might come home,” he said. “And that focuses them that this isn't just something where we're going over to just ride around: We're ready to fight and win our nation's wars if we have to. “That's the culture that we've tried to put into place.” https://www.defensenews.com/naval/2019/07/08/the-us-navys-riverines-are-up-gunned-high-tech-and-ready-to-lean-into-great-power-competition/