Back to news

July 30, 2018 | International, C4ISR

Targeting the future of the DoD’s controversial Project Maven initiative

By:

Bob Work, in his last months as deputy secretary of defense, wanted everything in place so that the Pentagon could share in the sweeping advances in data processing already enjoyed by the thriving tech sector.

A memo dated April 26, 2017, established an “Algorithmic Warfare Cross-Functional Team,” a.k.a. “Project Maven.” Within a year, the details of Google's role in that program, disseminated internally among its employees and then shared with the public, would call into question the specific rationale of the task and the greater question of how the tech community should go about building algorithms for war, if at all.

Project Maven, as envisioned, was about building a tool that could process drone footage quickly and in a useful way. Work specifically tied this task to the Defeat-ISIS campaign. Drones are intelligence, surveillance and reconnaissance platforms first and foremost. The unblinking eyes of Reapers, Global Hawks and Gray Eagles record hours and hours of footage every mission, imagery that takes a long time for human analysts to scan for salient details. While human analysts process footage, the ground situation is likely changing, so even the most labor-intensive approach to analyzing drone video delivers delayed results.

In July 2017, Marine Corps Col. Drew Cukor, the chief of the Algorithmic Warfare Cross-Function Team, presented on artificial intelligence and Project Maven at a defense conference. Cukor noted, “AI will not be selecting a target [in combat] ... any time soon. What AI will do is complement the human operator.”

As Cukor outlined, the algorithm would allow human analysts to process two or three times as much data within the same timeframe. To get there, though, the algorithm to detect weapons and other objects has to be built and trained. This training is at the heart of neural networks and deep learning, where the computer program can see an unfamiliar object and classify it based on its resemblance to other, more familiar objects. Cukor said that before deploying to battle “you've got to have your data ready and you've got to prepare and you need the computational infrastructure for training.”

At the time, the contractor who would develop the training and image-processing algorithms for Project Maven was unknown, though Cukor did specifically remark on how impressive Google was as an AI company. Google's role in developing Maven would not come to light until March 2018, when Gizmodo reported that Google is helping the Pentagon build AI for drones. Google's role in the project was discussed internally in the company, and elements of that discussion were shared with reporters.

“Some Google employees were outraged that the company would offer resources to the military for surveillance technology involved in drone operations,” wrote Kate Conger and Dell Cameron, “while others argued that the project raised important ethical questions about the development and use of machine learning.”

A petition by the Tech Workers Coalition that circulated in mid-April called upon not just Google to pull out of Pentagon contracts, but for Amazon, Microsoft and IBM to refuse to pick up the work of Project Maven. (The petition attracted 300 signatures at the time of this story.)

Silicon Valley's discord over the project surprised many in positions of leadership within the Pentagon. During the 17th annual C4ISRNET Conference, Justin Poole, the deputy director of the National Geospatial-Intelligence Agency, was asked how the intelligence community can respond to skepticism in the tech world. Poole's answer was to highlight the role of intelligence services in reducing risk to war fighters.

Disagreement between some of the people working for Google and the desire of the company's leadership to continue pursuing Pentagon contracts exacerbated tension in the company throughout spring. By May, nearly a dozen Google employees had resigned from the company over its involvement with Maven, and an internal petition asking the company to cancel the contract and avoid future military projects garnered thousands of employee signatures. To calm tensions, Google would need to find a way to reconcile the values of its employees with the desire of its leadership to develop further AI projects for a growing range of clients.

That list of clients, of course, includes the federal government and the Department of Defense.

While efforts to convince the tech community at large to refuse Pentagon work have stalled, the pressure within Google resulted in multiple tangible changes. First, Google leadership announced the company's plan to not renew the Project Maven contract when it expired in 2019. Then, the company's leaders released principles for AI, saying it would not develop intelligence for weapons or surveillance applications.

After outlining how Google intends to build AI in the future, with efforts to mitigate bias, aid safety and be accountable, Google CEO Sundar Pichai set out categories of AI work that the company will not pursue. This means refusing to design or deploy “technologies that cause or are likely to cause overall harm,” including an explicit prohibition on weapons principally designed to harm people, as well as surveillance tech that violates international norms.

Taken together, these principles amount to a hard-no only on developing AI specifically intended for weapons. The rest are softer no's, objections that can change with interpretations of international law, norms, and even in how a problem set is described.

After all, when Poole was asked how to sell collaboration with the intelligence community to technology companies, he framed the task as one about saving the lives of war fighters.

The “how” of that lifesaving is ambiguous: It could equally mean better and faster intelligence analysis that gives a unit on patrol the information it needs to avoid an ambush, or it could be the advance info that facilitates an attack on an adversary's encampment when the guard shift is particularly understaffed. Image processing with AI is so ambiguous a technology, so inherently open to dual-use, that the former almost certainly isn't a violation of Google's second objection to AI use, but the latter example absolutely would be.

In other words, the long-term surveillance that goes into targeted killing operations above Afghanistan and elsewhere is likely out of bounds. However, the same technology used over Iraq for the fight against ISIS might be permissible. And software built to process drone footage in the latter context would be identical to the software built to process images for the former.

The lines between what this does and doesn't prevent becomes even murkier when one takes into account that Google built its software for Project Maven on top of TensorFlow, an open-source software library. This makes it much harder to build in proprietary constraints on the code, and it means that once the Pentagon has a trainable algorithm on hand, it can continue to develop and refine its object-recognition AI as it chooses.

But the window for Google to be involved in such a project, whether to the joy or dismay of its employees and executive leadership, is likely closing.

In late June, the Pentagon announced creation of a Joint Artificial Intelligence Center, which among other functions would take over Project Maven from the Algorithmic Warfare Cross-Functional Team. The defense sector is vast, and with Google proving to be a complicated contractor for the Pentagon, new leadership may simply take its AI contracts worth million elsewhere with to see if it can get the programming it needs. And Maven itself still receives accolades within the Pentagon.

Gen. Mike Holmes, commander of Air Combat Command, praised Project Maven at a June 28 defense writers group breakfast, saying that the use of learning machines and algorithms will speed up the process by which humans process information and pass on useful insights to decisions makers.

Inasmuch as the Pentagon has a consensus view of explaining tools like Maven, it is about focusing on the role of the human in the process. The software will do the first pass through the imagery collected, and then as designed highlight other details for a human to review and act upon. Holmes was adamant that fears of malicious AIs hunting humans, like Skynet from the “Terminator” movies, are beyond premature.

“We're going to have to work through as Americans our comfort level on how technologies are used and how they're applied,” said Holmes. “I'd make the case that our job is to compete with these world-class peer competitors that we have, and by competing and by setting this competition on terms that we can compete without going to conflict, it's better for everybody.”

AI of the tiger

Project Maven, from the start, is a program specifically sold and built for the work of fighting a violent nonstate actor, identifying the weapons and tools of an insurgency that sometimes holds swaths of territory.

“Our responsibility is to help people understand what the intent is with the capability that we are helping to develop. ... Maven is focused on minimizing collateral damage on the battlefield. There's goodness in that,” said Capt. Sean Heritage, acting managing partner of Defense Innovation Unit Experimental (DIUx).

“There's always risk in how it will be used down the road, and I guess that's where a small pocket of people at Google's heads were. But, as Mr. Work pointed out during his panel at Defense One, they don't seem to have as challenging of a time contributing to AI capability development in China.”

Google's fight over Project Maven is partly about the present — the state of AI, the role of the United States in pursuing insurgencies abroad. It is also a fight about how the next AI will be built, and who that AI will be built to be used against. And the Pentagon seems to understand this, too. In the same meeting where Holmes advocated for Maven as a useful tool for now, he argued that it was important for the United States to develop and field tools that can match peer or near-peer rivals in a major conflict.

That's a far cry from selling the tool to Silicon Valley as one of immediate concern, to protect the people fighting America's wars presently through providing superior real-time information.

“The idea of a technology being built and then used for war, even if that wasn't the original intent,” says author Malka Older, “is what science fiction writers call a ‘classic trope.' ”

Older's novels, set two or three generations in the near-future, focus on the ways in which people, governments and corporations handle massive flows of data, and provide one possible vision of a future where the same kinds and volumes of data are collected, but where that data is also held by a government entity and shared transparently.

While radical transparency in data is alien to much of the defense establishment, it's an essential part of the open-source technology community for security concerns both genuine and sometimes not-so genuine. Building open source means publishing code and letting outsiders find flaws and vulnerabilities in the algorithm, without looking at any of the sensitive data the algorithm is built to process.

And Project Maven is built on top of open-source framework.

“One of the dangerous concepts that we have of technology is that progress only goes in one direction,” says Older.

“There's constantly choices being made of where technology goes and where concepts go and what we are trying to do.”

While it's entirely possible that the Pentagon will be able to continue the work of Project Maven and other AI programs with new contractors, if it wanted to reach out to those skeptical of how the algorithm would interpret images, it could try justifying the mission not just with national security concerns, but with transparency.

“Part of being an American is that Americans have expectations about what their government does and whether the government uses tech and tools to infringe upon their rights or not,” said Holmes. “And, so, we have really high standards as a nation that the things that we bring forward as military tools have to live up to.”

To work with the coders of the future, it may not be enough to say that the code — open source or not — is going to be used in ways consistent with their values. The Pentagon may have to find ways to transparently prove it.

https://www.c4isrnet.com/it-networks/2018/07/27/targeting-the-future-of-the-dods-controversial-project-maven-initiative/

On the same subject

  • Contract Awards by US Department of Defense - April 23, 2020

    April 24, 2020 | International, Aerospace, Naval, Land, C4ISR, Security

    Contract Awards by US Department of Defense - April 23, 2020

    AIR FORCE Dataminr Inc., New York, New York, has been awarded a firm-fixed-price contract for $258,661,096 for a commercially available license subscription that can leverage a variety of publicly available information sources, evaluate content to detect emerging events as they are developing and push alerts to users based on user-defined areas and topics of interest. The solution must be capable of distributing alerts in near real-time via email, web-based application and mobile platforms. The solution must be available commercially in the marketplace and able to scale to a Department of Defense enterprise capability and keep pace with developments and standards within the commercial industry sector. Work will be performed in New York and is expected to be complete by April 23, 2025. This award resulted after three firms were solicited and submitted bids. Fiscal 2020 operations and maintenance funds in the amount of $48,720,000 are being obligated at the time of the award. The Air Force District Washington, Joint Base Andrews, Maryland, is the contracting activity (FA7014-20-C-0016). Aviation Training Consulting LLC, Altus, Oklahoma, has been awarded a $7,281,483 firm-fixed-price modification (P00043) to previously-awarded contract FA8621-16-C-6339 for B-52 training system contractor logistics support and training system support center sustainment. The contract modification is for the third increment of the seven year basic contract. Work will be performed at Barksdale Air Force Base, Louisiana; and Minot AFB, North Dakota, and is expected to be completed by Oct. 31, 2020. Air Force fiscal 2020 operations and maintenance funds will fund this effort. The total cumulative face value of the contract is $7,281,483. The Air Force Life Cycle Management Center, Wright-Patterson AFB, Ohio, is the contracting activity. NAVY Lockheed Martin Rotary and Mission Systems, Baltimore, Maryland, was awarded a $147,639,775 undefinitized contract action modification to previously-awarded contract N00024-20-C-5310 for the procurement of MK 41 Vertical Launching System (VLS) vertical launcher module electronic components. Work will be performed in Baltimore, Maryland (44%); Fort Walton Beach, Florida (39%); Waverly, Iowa (3%); Hampstead, Maryland (2%); Dover, Pennsylvania (2%); Chaska, Minnesota (1%); St. Peters, Missouri (1%); Wooddale, Illinois (1%); Plainview, New York (1%); Irvine, California (1%); Roebling, New Jersey (1%); Forest Hill, Maryland (1%); Millersville, Maryland (1%); Sterling Heights, Michigan (1%); and Red Lion, Pennsylvania (1%). This modification provides the electronic components for MK 41 VLS, which is installed onboard Navy surface combatants (CG-47 and DDG-51 class ships) and multiple allied Navy platforms. MK 41 VLS stores, selects, prepares and launches standard missiles, Tomahawk, Vertical Launch Anti-Submarine Rocket and Evolved Sea Sparrow missiles. Work is expected to be completed by March 2025. This contract combines purchases for the Navy (65%); and the governments of South Korea, Finland and Germany (35%) under the Foreign Military Sales (FMS) program. Fiscal 2018 and 2019 shipbuilding and conversion (Navy); and FMS funding in the amount of $29,527,952 was obligated at the time of award and will not expire at the end of the current fiscal year. The Naval Sea Systems Command, Washington, District of Columbia, is the contracting activity. (Awarded April 17, 2020) Trijicon Inc.,* Wixom, Michigan, is awarded a $41,218,080 firm-fixed-price, indefinite-delivery/indefinite-quantity contract for the remanufacture of Rifle Combat Optics (RCOs). This contract provides for the materials, labor, equipment, facilities and missing/repair parts necessary to inspect, diagnose, test and restore the RCOs. Work will be performed in Wixom, Michigan, and is expected to be completed by April 2025. This contract has a five-year ordering period with a maximum value of $41,218,080. Fiscal 2020 operations and maintenance (Marine Corps) in the amount of $8,160,000 will be obligated at the time of award for the first task order and will expire at the end of the current fiscal year. This contract was not competitively procured in accordance with 10 U.S. Code 2304(c)(1) - only one responsible source and no other supplies or services will satisfy agency requirements. The Marine Corps Logistics Command, Albany, Georgia, is the contracting activity (M67004-20-D-0003). Avian LLC,* Lexington Park, Maryland, is awarded a $13,342,932 modification (P00068) to exercise an option on a previously-awarded cost-plus-fixed-fee contract (N00421-17-C-0049) to provide support for the Naval Air Warfare Center Aircraft Division's Integrated System Evaluation Experimentation and Test Department. Work will be performed in Patuxent River, Maryland, and services will include flight test engineering, programmatic, administrative, design, execution, analysis, evaluation and reporting of tests and experiments of aircraft, unmanned air systems, weapons and weapons systems. Work is expected to be completed in April 2021. Fiscal 2020 research, development, test and evaluation (Navy) in the amount of $3,783,077; fiscal 2019 research, development, test and evaluation (Navy) in the amount of $90,000; fiscal 2020 aircraft procurement (Navy) funds in the amount of $1,549,886; fiscal 2019 aircraft procurement (Navy) funds in the amount of $28,732; fiscal 2020 operations and maintenance (Navy) funds in the amount of $19,816; fiscal 2020 other procurement (Navy) funds in the amount of $144,756; and Foreign Military Sales funds in the amount of $310,238 will be obligated at time of award, $254,572 of which will expire at the end of the fiscal year. The Naval Air Warfare Center Aircraft Division, Patuxent River, Maryland, is the contracting activity. ARMY ECS Federal LLC, Fairfax, Virginia, was awarded an $83,099,372 cost-plus-fixed-fee contract for to create combined artificial intelligence (AI)-platform prototypes enhance. Bids were solicited via the internet with one received. Work will be performed in Fairfax, Virginia, with an estimated completion date of March 26, 2023. Fiscal 2020 research, development, test, and evaluation, Army funds in the amount of $83,099,372 were obligated at the time of the award. U.S. Army Contracting Command, Aberdeen Proving Ground, Aberdeen, Maryland, is the contracting activity (W911QX-20-C-0023). Sherwood Aviation,* Opa Locka, Florida, was awarded an $18,636,740 firm-fixed-price contract for overhaul/repair of CH-47 gas turbine engines. Bids were solicited via the internet with three received. Work locations and funding will be determined with each order, with an estimated completion date of April 23, 2025. U.S. Army Contracting Command, Redstone Arsenal, Alabama, is the contracting activity (W58RGZ-20-D-0051). General Atomics Aeronautical Systems Inc., Poway, California, was awarded a $9,943,746 modification (P00044) to contract W58RGZ-19-C-0027 for continuation effort for the Gray Eagle Unmanned Aircraft System. Bids were solicited via the internet with one received. Fiscal 2020 operations and maintenance, Army funds in the amount of $9,943,746 were obligated at the time of the award. Work will be performed in Poway, California, with an estimated completion date of April 23, 2021. U.S. Army Contracting Command, Redstone Arsenal, Alabama, is the contracting activity. Boyer Commercial Construction Inc.,* Columbia, South Carolina, was awarded an $8,686,240 firm-fixed-price contract for national cemetery expansion at Fort Jackson, South Carolina. Bids were solicited via the internet with seven received. Work will be performed in Columbia, South Carolina, with an estimated completion date of May 14, 2021. Fiscal 2020 Department of Veteran Affairs funds in the amount of $8,686,240 were obligated at the time of the award. U.S. Army Corps of Engineers, Charleston, South Carolina, is the contracting activity (W912HP-20-C-2001). DEFENSE LOGISTICS AGENCY Robertson Fuel Systems LLC, Tempe, Arizona, has been awarded a maximum $38,784,713 firm-fixed-price, indefinite-delivery requirements contract for aircraft fuel tanks. This was a sole-source acquisition using justification 10 U.S. Code 2304 (c)(1), as stated in Federal Acquisition Regulation 6.302-1. This is a five-year contract with no option periods. Location of performance is Arizona, with an April 23, 2025, performance completion date. Using military service is Army. Type of appropriation is fiscal 2020 through 2025 Army working capital funds. The contracting activity is the Defense Logistics Agency Aviation, Redstone Arsenal, Alabama (SPRRA1-20-D-0034). *Small business https://www.defense.gov/Newsroom/Contracts/Contract/Article/2162978/source/GovDelivery/

  • Ottawa to stage 350 soldiers, 6 aircraft to fight B.C. wildfires

    July 5, 2021 | International, Aerospace, Security

    Ottawa to stage 350 soldiers, 6 aircraft to fight B.C. wildfires

    The Canadian government is moving to stage more than 350 soldiers and tactical airlift capabilities out of Edmonton to support firefighters in British Columbia.

  • Philippines eyes mid-range missiles, 40 fighter jets to modernise military
All news