Symposium on Cyber, International Law and Artificial Intelligence in Armed Conflict: Law of Armed Conflict Implications of AI-Induced Fog of War

[Gary Corn is a Professor and Director of the Technology, Security and Law Program at American University Washington College of Law. He previously served as a military attorney in the US Army, including as Staff Judge Advocate (General Counsel) of U.S. Cyber Command]

Timely and accurate information is the lifeblood of successful military operations and is often critical to meeting law of armed conflict (LOAC) obligations. But as Clausewitz famously noted:

“War is the realm of uncertainty; three quarters of the factors on which action in war is based are wrapped in a fog of greater or lesser uncertainty.”

Some of this proverbial “fog of war” is inherent to the extremely complex and chaotic nature of combat. Some is the result of deliberate efforts to deceive one’s enemy, degrade the availability and quality of the information available to it, and corrupt its decision-making processes.

Thus, achieving and maintaining information advantage—that is, piercing through the fog of war while simultaneously thickening it for the enemy—has long been a focus of military innovation and technology. Artificial Intelligence (AI) is just the latest example. Specifically, this post considers the LOAC implications of the fielding of Generative AI (GenAI)-based systems designed to deceive enemy sensors and systems, so-called “fog of war machines.” Where such actions result in an enemy’s errant targeting decisions and civilian harm, who bears LOAC responsibility?

Some have proffered AI as the technology that will finally eliminate the fog of war, by:

“replacing human guesswork with machine precision and processing oceans of data at speeds that would render uncertainty obsolete.”

There is little question that AI can provide a distinct decisional advantage by mitigating the so-called DRIP phenomenon (data rich and information poor). As I noted here, it was this exact problem of data and information overload that drove the U.S. Department of Defense’s Project Maven and follow-on AI initiatives. The technology’s capacity to sift through and make operational sense at speed and scale of the increasingly large data sets generated by the growing array of high-tech battlefield sensors and collection platforms can be an operational game-changer and potentially a powerful tool for adhering to LOAC precautionary obligations.

Nevertheless, as much as AI has the potential to lift some of war’s fog, it can also produce its own. For example, whether humans can keep pace with the “machine speed” of AI-generated information, decisional recommendations, and outputs remains to be seen. And even state-of-the-art models continue to suffer from technological limitations, such as opacity, alignment faking, and degrees of unpredictability in non-deterministic models. In combination, these characteristics and limitations raise the very real specter of “human” sensory overload and the attendant questions of whether and how human overseers can exercise appropriate human judgment over AI-enabled military systems—a topic for another day. Here, the focus is on the use of AI as a tool of military deception to cloud and corrupt an enemy’s situational awareness and decision-making—specifically, machine-to-machine deception.

It is an axiom that, subject to specific and generally narrow LOAC proscriptions, military deception:

“the art of misleading the enemy into doing something, or not doing something, so that his strategic or tactical position will be weakened”

is a well-established and legitimate method of warfare used to achieve concealment, security, and surprise. As Winston Churchill famously noted during World War II, “In wartime, truth is so precious that she should always be attended by a bodyguard of lies.” And as recent conflicts have demonstrated, combat effectiveness, let alone battlefield survivability, depends increasingly on technology-driven deception, camouflage, and multispectral concealment.

Absent effective countermeasures, in an age of sensor-soaked battlefields and data-centric warfare, forces can be “instantly detected, targeted, and destroyed” by traditional, and now AI-driven sensor systems. As Christian Brose, the Head of Strategy at Anduril Industries noted presciently several years ago:

“in a world that is becoming one giant sensor, hiding and penetrating — never easy in warfare — will be far more difficult, if not impossible.”

As intelligence, surveillance, and reconnaissance (ISR) technologies advance, “there will be nowhere to hide.”

Enter the “fog of war machine.” GenAI is being rapidly developed and employed as a powerful deception tool to, inter alia, spoof text and audio communications, create “dummy” or “ghost” synthetic electromagnetic, radar, acoustic, thermal, and other signatures and camouflage, and produce hyper-realistic digital impersonations or “deepfakes” of individuals and events (see here, here, and here).

Thus, in addition to generating “human-like” synthetic deception content, GenAI can create “machine-like” material that other machines engage with and consume, enabling the creation of automated deception planners designed to exacerbate knowledge quality problems. In other words, to obscure friendly positions and intentions, GenAI can corrupt or confuse an enemy’s “own tracking algorithms [with] noisy, incomplete, and possibly mendacious data.”

Use of such machine-to-machine deception to deceive or “dazzle” an enemy’s sensors or weapon systems introduces the possibility of causing an adversary to make errant targeting decisions that result in civilian harm. For example, through data poisoning, masked or spoofed signals, sensor deception, or multispectral camouflage, a party to a conflict might not only mask the location of key forces and assets but cause the enemy to believe they are located elsewhere. What if civilians or civilian objects are present at that false location? At what point, if any, does the use of such “fog of war machines” implicate the user’s LOAC responsibility for the civilian harm that flows from the enemy’s erroneous targeting decisions, at least where reasonably foreseeable?

The issue is neither novel nor unique to AI. Sabotaging or otherwise countering enemy weapons systems is a longstanding practice, the LOAC implications of which are ill-defined and extremely fact-dependent. So too are military deception operations that cause an enemy to misdirect combat operations, such as the dropping of the famed “Rupert” parachute dummies across multiple, civilian-inhabited locations across Normandy during D-Day to draw German forces away from the beaches.

The question may turn on the degree to which the sabotaging party exercises actual control over a weapon or weapons system. For example, according to Rule 45 of the Oslo Manual, “A person who wrests control of a weapon system . . . assumes responsibility for its subsequent use in accordance with the degree and the duration of the control exercised.” By this view, echoed in Scenario 27 of the Cyber Law Toolkit, where the sabotaging party wrests actual control of, and intentionally “directs” a weapons system against protected entities, that party is responsible for the LOAC violation. According to the Manual, responsibility may also attach, at least as a matter of general precautionary obligations, if the resultant civilian harm is foreseeable.

However, simply causing the system to fail or be misdirected away from friendly forces is much harder to qualify as an attack attributable to the sabotaging party. As the Manual notes, responsibility may be impossible to attribute if adversaries “are contesting control over a weapon system and the system ends up crashing and harming civilians.” And whether corrupting an adversary’s decision-making alone equates to control is a much harder case to make with the downside implication of arguably relieving that party of its LOAC precautionary obligations.

Of course, the foregoing only scratches the surface of the myriad issues raised generally by deception and sabotage operations aimed at corrupting or confusing an enemy’s targeting processes and decisions. Introducing GenAI-enabled machine-to-machine deception capabilities, with risks such as alignment faking and hallucinations, further clouds these questions legally and factually. When and where does responsibility attach when agentic systems deceive in ways that implicate LOAC violations? What precautionary obligations do belligerents have when employing such fog of war machines? Inversely, with the known problem of adversarial AI, what precautionary obligations do parties have to ensure the provenance, integrity, and reliability of the decision-support and targeting systems they use?

The realities of the modern, sensor-soaked, data-centric, AI-driven battlespace are driving the need for effective countermeasures to enhance concealment, survivability, and combat effectiveness. Leveraging GenAI, including machine-to-machine deception capabilities, is an inevitable reality. So too are the attendant risks. Thus, prudence in their development and use is imperative. When the fog of war thickens, civilians often pay the price.

This post forms part of the Opinio Juris symposium on International Law and Artificial Intelligence in Armed Conflict (introduced here) and draws on the author’s chapter in the forthcoming OUP volume of the same title.