Back to news

August 21, 2019 | International, C4ISR

4 big problems the intelligence community faces moving to a new data system

By: Nathan Strout

The Defense Intelligence Agency wants to move quickly in developing the Machine-assisted Analytic Rapid-repository System, but the massive project, which will transform how the intelligence community uses data, faces some hard problems.

The Defense Intelligence Agency is responsible for informing war fighters and policy makers on the military capabilities of foreign nations.

Currently, the agency relies on the Modernized Integrated Database to house foundational military intelligence, but the 20-year old database wasn't built for the 21st century data landscape. The DIA wants to replace MIDB with MARS, a comprehensive, adaptable, scalable and rigorous data environment. With more data that is better labeled and organized, MARS will allow analysts to use applications to sort and process that data to make connections they couldn't otherwise.

Last year, the DIA issued a broad agency announcement to solicit industry feedback from MARS. They've spent the year going over that information and are now preparing to actually build it.

“2019 was about learning,” said Terry Busch, chief of DIA's integrated analysis and methodologies division. “This was the year of prepping to get started. In 2020 we get started ... We're going to move from designing to building very very quickly."

Irving Townsend, also of the DIA, added that the agency was working to make some components of MARS available to the United States' closest allies in 2020 so leaders in those countries can begin looking at how they can use that data. In summer 2020, the MARS application programming interface will be released to the public, Busch said.

But even as the DIA gears up to begin building MARS, the agency has four big problems left to solve.

Resolving data inconsistency

Perhaps the most pressing problem with creating a unified resource such as MARS is ensuring that the data fed into the system has been labeled and handled in a uniform way. That's easier said than done. According to Busch, there are 1,300 different data standards in the Department of Defense.

“I work a lot with (the National Geospatial-Intelligence Agency) and DIA because we have a very similar path for our data,” said Busch. “The NGA has done some wonderful work with some of their data standardization and modeling (...) because NGA's been in the data making business for a long time.”

Making the data collected by the various agencies and services interoperable is essential for the MARS enterprise.

Data storage

The amount of data collected by the intelligence community that MARS needs to encompass is staggering. Hosting images and videos, like those collected by the National Reconnaissance Office, for instance, takes up a lot of space.

Because of this, MARS will not actually host all of the data itself. Instead, MARS will refer to intelligence hosted by other agencies. Instead, MARS needs to be able to index that intelligence. Theoretically, users will be able to click a link to access that data hosted on other servers.

The DIA will have to figure out how this solution for MARS to operate effectively.

The black box problem

In developing a massive dataset of intelligence, the DIA wants all intelligence to be explainable, meaning that analysts need to be able to see how the intelligence was arrived at. In other words, can analysts and systems show their work?

“It's really, really important to understand that we're not going to accept a black box,” said Townsend. “Our analysts are not going to accept that.”

This is a problem for intelligence agencies and contractors who don't want to reveal their methods or proprietary information. Townsend noted that companies are going to have to figure out how to explain their intelligence without giving away that information.

Legacy systems

Another problem with moving to MARS? Many legacy systems will not be able to move to the MARS framework.

While MARS will incorporate all of the MIDB data, some legacy systems will not be able to use the new system and will instead remain reliant on MIDB.

“The transition is difficult. Many of our war fighters are impinged by legacy technology,” said Busch. “There is not turning MIDB off. Not for the foreseeable future.”

Busch noted that while Congress may not like funding both programs simultaneously, it is necessary until those legacy systems can be replaced or upgraded.

The DIA will discuss these problems at a MARS industry day Sept. 10, where they are expected to explain what the agency wants from industry.

https://www.c4isrnet.com/intel-geoint/2019/08/19/4-big-problems-the-intelligence-community-faces-moving-to-a-new-data-system/

On the same subject

  • House Appropriators Add 12 F-35s, Boost Weapons Spending, But…

    July 8, 2020 | International, Aerospace, Naval

    House Appropriators Add 12 F-35s, Boost Weapons Spending, But…

    "To us, it means that there is going to be much more tension and debate over future modernization programs as flat investment will not enable DoD to recapitalize in a timely and militarily relevant pace," says defense analyst Byron Callan. By COLIN CLARKon July 07, 2020 at 7:38 PM WASHINGTON: House appropriators made their first cut at the annual defense spending bill today, approving spending $3.5 billion below the Trump Administration's request — although lawmakers added a substantial $4.1 billion for several weapons systems, including 12 additional F-35s. Overall, the House Appropriations Committee trimmed $3.5 billion from the Trump Administration's 2021 budget request while still fully paying for a 3% pay raise and force structure increases to all but the Marines, who will lose 2,100 people. The appropriators approval of an increase in F-35 buys makes it unlikely the House Armed Services Committee's skepticism of the Joint Strike Fighter program will prevail. The HASC added no more planes above the administration request for 79 aircraft of all three models and docked at least a score of supporting line items by a total of $561 million. By contrast the SASC added $1.36 billion to buy more Air Force F-35As, Marine F-35Bs, and Navy F-35Cs, plus spare parts. In other bump ups, the HAC funds 11 V-22 aircraft, adding $1.1 billion to buy two more than the request. It also adds three P-8A Poseidon aircraft for the Navy Reserve, three more than the request for an additional $510 million. And echoing the House authorizing committee's support, the HAC added 16 MQ-9 Reaper unmanned aerial vehicles to the Pentagon request, for a cost of $344 million. But making sense of the HAC-D bill is difficult because it's not yet clear what and where they've cut, as veteran defense stock analyst Byron Callan notes. It all gets complicated by the CARES Act and adjustments to contracts that have been made. Overall, Callan says, it looks as if fiscal 2020 — last year — may have been the peak of whatever Trump defense boost there has been. But it's all uncertain. “Absent the pandemic budget impacts, however, the markups so far suggest that FY20 was a peak for DoD investment. This does not mean investment is at the peak and headed fast downhill in FY21 and beyond,” Callan writes. “To us, it means that there is going to be much more tension and debate over future modernization programs as flat investment will not enable DoD to recapitalize in a timely and militarily relevant pace.” https://breakingdefense.com/2020/07/house-appropriators-add-12-f-35s-boost-weapons-spending-but/

  • Key lawmakers flex new positions to bolster shipbuilding industry

    June 13, 2023 | International, Naval

    Key lawmakers flex new positions to bolster shipbuilding industry

    Lawmakers representing major shipbuilding states have taken on new leadership roles in Congress — and they're looking to expand the industry's capacity.

  • Space Force acquisitions and architecture | C4ISRNET Exclusive

    April 26, 2021 | International, Aerospace

    Space Force acquisitions and architecture | C4ISRNET Exclusive

    C4ISRNET's Nathan Strout gets the latest on Space Force's acquisitions process as the new force gets stood up, and talks to the director of the Space Development Agency on how the Department of Defense plans to move data safely through space. More from the annual C4ISRNET Conference.

All news