Beyond Anthropic’s Red Line: Human-in-the-Loop and the Illusion of Legitimacy in AI Decision-Support Systems

[Rosa Villar is a PhD Candidate in International Law at the University of Aberdeen with research focused on AI-enabled targeting systems, international humanitarian law and international criminal law]

Introduction

In February 2026, Anthropic declined to use its large-language models for fully autonomous weapons, escalating a dispute with the U.S. Department of Defense. While such a red line on unsupervised systems may be ethically valuable, it seems to suggest that the mere presence of a human authorizing the military intervention is the central safeguard. Nonetheless, this condition may not be sufficient.

Hostilities in Gaza and Ukraine reveal the rise of AI Decision Support Systems (AI-DSS) in targeting. While these systems inform human decision-makers rather than autonomously pull the trigger, it is unclear whether their human-in-the-loop model sufficiently ensures a meaningful judgment in targeting decisions.

Debates on AI-targeting emerged with Lethal Autonomous Weapon Systems (LAWS) which can identify, select, and engage targets, without human intervention in these tasks.

After years, discussions under the Group of Governmental Experts (GGE on LAWS) shifted towards the human element. The requirement of human involvement in decisions over the use of force arises from IHL, which demands those planning, deciding or executing the attack to assess its lawfulness in terms of distinction, proportionality and precautions. However, no agreement has been reached on what this requirement entails, leading to different interpretations. While certain levels of involvement are considered “sufficient” or “appropriate” by some, others ask for stricter “meaningful” human control.

Unfortunately, calls for greater human intervention have focused much attention on control, rather than on the actual exercise of human judgment in use-of-force decisions. Human judgment refers to humans exercising their discernment regarding IHL rules. It requires humans to have the ability to  conduct qualitative context-specific analysis of the circumstances to determine the lawfulness of the selected target. Conversely, human control has been understood as a concern to maintain human agency, including timely human intervention. “Human-in-the-loop” weapons reflect this approach, they deliver force only with a human command. Sensors acquire, track and identify potential targets and alert human operators, who ultimately approve or deny the engagement.

The establishment of “human-in-the loop” condition, under “human meaningful control” claims, has contributed to the mistaken perception that simply having a human authorizing the intervention, enables him to conduct a meaningful assessment of the lawfulness of the attack. This rationale has facilitated the  rise of AI-Decision Support Systems (AI-DSS), computerized tools that use AI to assist militaries in targeting decision-making.

AI-DSS portray an image in which, despite the introduction of autonomy, operators’ authorization of targets, ensures such intervention to be meaningful in terms of IHL. However, AI-DSS production of target profiles, may impede operators to infer substantive information to carry those contextual analysis required to determine the lawfulness of the target. Consequently, this deficiency endows AI-DSS with a similar potential to that of LAWS to undermine human judgement in targeting decision-making.

This post argues that a human-in-the-loop is not enough to ensure a meaningful human judgment in targeting, as certain AI architectures reduce such involvement to a  mere “nominal input” with no legal or moral reflection. It compares how LAWS and AI-DSS impact human judgment and ultimately reveals that resorting to the latter constitutes an attempt to legitimize controversial AI-weapons. Finally, it concludes that meaningful IHL assessments require context-specific human judgment, and that simply having a human authorizing does not, per se, ensure such meaningfulness.

LAWS and AI-DSS: Shared Erosion of Human Judgment

According to the latest characterization LAWS  are:

“an integrated combination of one or more weapons and technological components, that can identify, select, and engage a target, without intervention by a human operator in the execution of these tasks”.

Concerns on LAWS, arise from the fact that they execute targeting decisions, based on some “generalized” human decision-making made in advance (pre-establishment of lawful targets). However, this generalized decision-making operating in the battlefield’s inherent uncertainty may undermine the case-by-case judgment required by IHL.

Alternatively, AI-DSS are computerized tools designed to aid humans in complex decision-making. These tools bring together data (e.g. satellite imagery, sensor data, or phone signals) to detect familiar patterns and present analysis and recommendations to decision makers. They use data driven AI-technologies such as Machine Learning and Deep Learning, which build an AI system by letting it “learn” through training data gathered from experience. Thus, contrary to earlier AI-DSS, where knowledge was provided by humans in specifically programmed deterministic rules, data-driven learning algorithms generate their own rules for solving a problem. In targeting, AI-DSS inform the human user in a “particular targeting decision”.

AI-DSS take the advantage of being a dual-technology, and therefore not related to “killer robots” image characterizing LAWS. However, while the role of these two technologies in targeting is different (LAWS execute whereas AI-DSS inform); the exercise of human judgment to comply with IHL may equally be undermined.

A meaningful judgment in the assessment of distinction and proportionality requires humans to have the ability to carry out a qualitative context-specific analysis of the circumstances prevailing at the relevant time. For instance, a civilian directly participating in hostilities (DPH) is a status obtained under the duration of his direct participation in hostilities which temporarily suspends civilian protection. Determining the lawfulness of these targets is contextual and potentially ambiguous. Military personnel consider the facts and the specific acts such person commits at the time to ascertain whether it is a lawful target or not. Such assessment of combatant status rather than quantitative valued, open to programming, is qualitative-based and context-dependent.

Considering this, in the use of LAWS, human assessment done in advance always entails some uncertainty as it is not possible to foresee all the events that will appear. Consequently, such pre-programmed assessments may not be meaningful where qualitative context-specific analysis of the circumstances are required. Here, judgment becomes limited due to its externalization into the system, moving from a generalized pre-selection of targets to the execution of an uncertain specific target engagement.

Conversely, in AI-DSS, human judgment is affected through the process of internalizing data. Algorithmic intelligence does not infer contextual information; the system merely filters data points and selects features that itself identified as relevant (but which may not be meaningful for IHL determinations). From this  correlation of  data, targets that share certain features are produced. This transformation of mere information into weaponized data (recommended target-profiles decided by the system ready for engagement), is aimed at alleviating data overload faced by human operators; however, it prevents them from acquiring the situational and contextual awareness that truly give meaning to the data. Consequently, these operators do not meaningfully participate in the military decision-making that leads to the engagement and their mere authorization is not determinative for the legal assessment required by IHL.

This is not to say that a meaningful human judgment is not compatible with systems that provide information to assist humans. Systems acting as tools for the mere representation of gathered information do not prevent humans from making the correspondent legal assessment of the data. Humans may engage with the information presented and, considering the circumstances of the context, assess by themselves which objects or persons may be lawful. Under such scheme, the operator would be able to carry a meaningful judgment

Bearing this in mind, what undermines meaningful human judgment in AI-DSS is precisely their autonomous generation of target profiles. In this process, the system itself alters the nature of the data and transforms it into weaponized data ready for engagement. This prevents humans from attaining relevant situational awareness of the context, crucial when determining whether those targets are lawful ones Accordingly, AI-DSS may also compromise human judgment.

Illusory Legitimacy

Considering the above, the resort to AI-DSS as an alternative to LAWS responds to a broader attempt to seek legitimacy in the eyes of the public.

Legitimacy in war  refers to the perception that the purpose of war is right, the conduct is just, and the outcome will be beneficial. Its purpose is broader than solely legality, it aims to maintain the moral authority in the operations, which ultimately reflects the need for social acceptance and support.

Research evidences that military legitimacy requires that the civil population perceive combatants as exercising “due care” to avoid innocent civilian harm. Legitimacy is not only perceived by civilians where the operations are occurring but given by the general public.

The duty to exercise constant care operates as the underlying standard of the requirement to take all feasible precautions in attack. Under this duty, those planning or deciding upon an attack shall do everything feasible to verify that the objectives to be attacked are not civilians nor civilian objects. Where objectives of an ambiguous nature are present, this duty requires a qualitative context-specific analysis of the circumstances to genuinely verify the target-nature. In other words, the proper exercise of due care indirectly depends on the ability to conduct meaningful IHL assessments.

AI does not infer contextual information, and lacks the ability to distinguish if a target of an ambiguous nature is a lawful target or not. This explains society’s reluctance to accept LAWS with minimum levels of human intervention as legitimate means of warfare, as they are perceived as incapable of exercising ‘due care’ to avoid civilian harm.

In response, AI-DSS establish human-in-the-loop as the perfect pretext to suggest that such authorization of targets enables operators to exercise the desired due care. In these legitimization efforts, AI-DSS create the mistaken perception that human authorization of targets is synonymous with a meaningful legal assessment and with a true exercise of due care.

However, this is far from true. Meaningful IHL assessments require combatants to be able to attain enough situational awareness to analyze lawfulness of the attacks. Maintaining a human-in-the-loop, interacting with target profiles generated by the machine, does not allow operators to acquire such contextual awareness that truly gives meaning to the data. Accordingly, “human in the loop” is not enough, by itself, to ensure a meaningful judgment in targeting decision, and hence the exercise of due care is unfeasible.

Consequently, while AI-DSS ultimately seek to obtain public acceptance by portraying a better exercise of due care, their legitimization turns out to be illusory. AI-DSS mere inclusion of a human-in-the-loop does not guarantee a higher exercise of cautiousness to avoid civilian harm. This inability  renders the exercise of due care unfeasible. It follows that AI-DSS should not be granted social legitimacy, since the supposed legitimacy they seek to portray is purely illusory.

Conclusion

This post argues that a human-in-the-loop is not enough, by itself, to ensure a meaningful human judgment in targeting decisions.

Conversely to LAWS, AI-DSS do not autonomously pull the trigger or execute the military intervention itself. They are non-weaponized systems that “simply” inform the decision-maker giving recommendations. However, the mere presence of a human does not guarantee a meaningful involvement.

Relevant assessments of the lawfulness of the attack require the ability to carry context-specific analysis of the circumstances. However, AI-DSS transformation of information into weaponized data, prevents humans from conducting such IHL assessments. Recommendation of profile targets, are presented as weaponized data ready to engage after the operator’s authorization. However, in such authorization, operators are deprived of the awareness and contextual perceptions that truly give meaning to those outputs, making such approval neither determinative in itself nor a guarantee for the preservation of human judgment

Consequently, the increasing use of AI-DSS represents an attempt to legitimize controverted targeting systems. Under the pretext of maintaining a human-in-the-loop they seek social support However, considering the operator’s inability to assess target lawfulness, the exercise of due care  providing social legitimacy is unfeasible. As a result, AI-DSS legitimacy turns out to be a mere illusion. 

Finally, rather than advocating for a banning approach of AI in warfare, this post aims to draw attention back to the heart of the debate, systems ensuring humans make relevant decisions over the use of force. Consequently, maintaining the debate on meaningful human judgment, which emerged from GGE debates on LAWS and remains unresolved, is of capital importance, especially with the emergence of new technologies such as AI-DSS, that equally undermine human judgment.