Augmented reality: Seeing the benefits is believing

Lt. Col. Brett Lindberg and Jan Kallberg

There is always something taken away when there are added functionalities. Does the concept of wearing augmented reality that digitally provides situational awareness create an upside that outweighs what it takes away for rifleman skills? The supercharged hearing, six senses for those equipped, broader view of sight, picking up smells, changes in lights and shadows, slightest change in the near environment: With all these close-action skills, will augmented reality create more distraction than enhancement? Is it too early to push digital situational awareness all the way down to the soldier in maneuver units? Is the upside present?

Naturally, all new defense technology takes time to find its place in the fight. The helicopter was invented in the 1930s, and it found a limited military role in the Korean War, not meeting the military expectation of higher impact. But 15 years later, it played a pivotal role in the war in Vietnam. New technology is not only technology — the human component to properly implement it is likely slower than the technological advancements.

It is always easier to question than explain, and we understand that many thoughts and thousands of work hours have gone into designing the early augmented reality systems. However, still we find our questions worth discussing because once fielded, utilized and put into action in a conflict, it is too late to raise any concerns. This is the time to discuss.

How reliable are the sensors? Can the sensors be easily spoofed? Is it too early to push it all the way down to the individual soldier? A technologically advanced adversary will likely devote research already in peacetime to develop one-time use, tossable, simple, low-cost devices that can — in close combat — create spurious sensor data and derail augmented reality. If the integrity of the sensor data is in question, it will likely force commanders to refrain from using augmented reality.

A similar, relevant issue is the extent of the augmented reality technology's electromagnetic signature. Will the interconnectivity of the squad's augmented reality compromise the unit and deliver information to the enemy? What we do not want to face is a situation where adversaries can pinpoint the location or proximity to U.S. forces by simple detection measures.

So, worst-case scenario, inexpensive devices can nullify a significant U.S. investment in technology, training and tactics. Added to the loss of usable augmented reality equipment, the soldiers could be “HUD-crippled.” Navy aviators use the term “HUD-cripple” to visualize a complete dependency of heads-up displays in the cockpit. The “HUD-cripple” is the loss of traditional Navy aviator skills such as landing on an aircraft carrier without the heads-up display. Will soldiers have retained the skills to fight effectively without augmented reality if it goes down?

Technical advancements bring us new options and abilities, and they increase mission success. But as with all uncharted territory, they also bring surprises and unanticipated outfalls. During the war in Vietnam in the 1960s and 1970s, military aviation instruments took a significant leap forward, going from World War II-styled gauges in fixed-winged Douglas A-1 Skyraider planes to an earlier version of today's instrumentation in McDonnell Douglas F-15 Eagle fighter jets rolled out as the war in Vietnam came to an end.

Parallel with the military advancements, these avionic upgrades were transposed into civilian cockpits with increased complexity and variations, as jetliners are multi-engine airframes, where the number of information points and alarms became numerous in the jetliner cockpit. In the late 1970s and early 1980s, civilian aviation faced several accidents that were hard to explain with standard aviation physics and crash evidence. Instead, the conversations recorded in the black boxes revealed these fatal air crashes. Several of the deadly crashes could have had another outcome if the pilots had not become overwhelmed by all the blinking lights, alarms, buzzers and avionics grabbing their attention, so the pilots lost situational awareness and focus.

The warnings, avionics and buzzers had the correct information, but the presentation was a tsunami of red blinkers and alarming sounds, lacking any hints on how to prioritize what needs to be done to recover from a dangerous in-flight emergency.

In our view, the key to effective augmented reality is to structure and segment what matters and when.

Units — and it varies from soldier to soldier — have different experience levels, so information has a variation in value down to the soldier level. In research design, you seek to explain as much as you can with as little as you can without losing rigor. The same challenge goes for augmented reality, where rigor could be said to be the integrity of the information.

Transferred to the ground-fighting world, are we, as an engineering-driven nation, so technology-happy that instead of creating tools for increased survivability and mission success, we initially increase the risks for the war fighter and only correct these after we suffered a surprise in combat?

We understand that implementing augmented reality is a long process that is just now at the stage of proving the concept; with setbacks and successes, where are we on the learning curve? In our view, synthetic learning environments have already matured and provide an ample opportunity to use the augmented reality technology with a high return on investment. The opportunities reside in knowledge transfer, sharing experiences, preparing for an ever-changing operational environment, and by doing so, increasing soldiers' survivability and ensuring mission success. The question is: Are we ready to rely on augmented reality in combat?

Lt. Col. Brett Lindberg is a research scientist at the Army Cyber Institute at West Point and a simulation operations officer. Jan Kallberg is a research scientist at the Army Cyber Institute at West Point, and an assistant professor at the U.S. Military Academy. The views expressed are those of the author and do not reflect the official policy or position of the Army Cyber Institute at West Point, the U.S. Military Academy or the U.S. Defense Department.

https://www.c4isrnet.com/opinion/2020/07/17/augmented-reality-seeing-the-benefits-is-believing/