The Pentagon/Anthropic Clash Over Military AI Guardrails

[Jessica Dorsey is an Assistant Professor of International Law at Utrecht University School of Law; Elke Schwarz is a Professor of Political Theory at Queen Mary University London; Ingvild Bode is a Professor of International Relations, University of Southern Denmark; Zena Assaad is an Associate Professor at the School of Engineering, Australian National University; and Neil Renic is a Lecturer in Ethics at the University of New South Wales. The authors are all members of the Independent Advisory Board on Legal Reviews of the Responsible by Design Institute.]

Just this week, a contract dispute between the U.S. Department of Defense (DoD) and the AI company Anthropic escalated into a public showdown over whether and which legal, safety and ethical guardrails should constrain the military use of artificial intelligence.

At the center of the controversy is Claude, Anthropic’s flagship model. For months, Anthropic and the DoD have reportedly been negotiating the terms under which the military may use Claude. The issue making headlines is not so much the technical performance and possible limitation of the large language model tool, but rather the principled stance Anthropic has taken from an ethical and legal perspective.

Anthropic has drawn two lines: its systems are not to be used for mass surveillance of American citizens, nor for weapons that fire with no human involvement. Such weapons fit the description of “fully autonomous weapons systems”, referred to in the December 2025 version of the rolling text of the Convention on Certain Conventional Weapons’ Group of Governmental Experts on LAWS (CCW GGE LAWS), and described as weapons that “can be characterized as a functionally integrated combination of one or more weapons and technological components, that can identify, select, and engage a target, without intervention by a human operator in the execution of these tasks.”

The DoD maintains these conditions are too restrictive, arguing that the operational realities and legal complexities of military missions make such categorical constraints impracticable. Anthropic CEO Dario Amodei insists that the AI model could be used in accordance with what it can “reliably and responsibly do.” The DoD has recently clarified its mandate for “Responsible AI” to include “any lawful use.”  What may appear, at first glance, to be a contractual spat between a technology provider and its government client is, in fact, emerging as a far more consequential legal and ethical stress test for the future and limits of AI-enabled warfare.

Claude and Classified Systems

Claude is a suite of proprietary large language models, together with an AI assistant and related tools built on those models. Within the U.S. defense ecosystem, it has become deeply embedded in classified environments, supporting analysis, operational planning, and intelligence workflows. Claude was allegedly integrated into Palantir software and used during the raid last month targeting the Venezuelan president Nicolás Maduro, leading to his abduction and extradition to the United States. This raises a number of questions about the role of AI-enabled decision-support systems (AI-DSS) in military operations. The role of AI-DSS in military contexts has become increasingly prominent. These emerging technological realities pose tangible legal and ethical risks, as illustrated by recent developments in Gaza, yet they have not generated the same breadth of international governance debate as LAWS. Precisely because AI-DSS remain “understudied, under-addressed and unregulated,” they warrant greater and more focused attention within international regulatory frameworks.

Anthropic has long marketed its “safety-first” approach to the development and use of their systems, as outlined in their Responsible Scaling Policy, a policy which aims to mitigate catastrophic risk from AI systems. The company originally pledged to stop training new AI models until safety guidelines can be guaranteed in advance. While this pitch positioned them well for many consumers, it has proven to be a hindrance for their relationship with the U.S. DoD. This policy has since been amended amidst this public dispute. Version 3.0,  released February 24th, presents a modified and lowered set of safety guardrails which are worded more broadly, creating the space for many different interpretations of those guardrails.

Claude is currently the only AI model deployed across a number of classified DoD systems. That exclusivity gives Anthropic a certain degree of leverage but also exploitable pressure points. The Pentagon has reportedly threatened to designate Anthropic a “supply chain risk,” a national-security consideration typically reserved for foreign adversaries. Such a designation would create immediate and immense subcontracting problems for any defense contractors relying on Claude, including Amazon Web Services, Palantir and Anduril, and could potentially, de facto, force the company out of the defense tech ecosystem. This would have repercussions not only in the US context, but also internationally as, for example, a number of European states, such as the UK, have large-scale contracts with these companies.

Defense officials, including Secretary Pete Hegseth, have framed Anthropic’s safety conditions as operationally unrealistic. Military operations, they argue, unfold in grey zones. Therefore, DoD claims it must be able to integrate AI-systems for “any lawful use,” without corporate-imposed constraints that could impede mission success.

But the phrase “any lawful use” raises many concerns, especially given the current administration’s approach to understanding its obligations under international law, including international humanitarian law (IHL). This is especially salient in terms of implications regarding AWS and AI-DSS as these systems reveal tensions with the principles and rules related to distinction, precautions in attack (including the duty of constant care) and proportionality, among others.

Safety guardrails: Responsible-by-Design?

Anthropic’s guardrails are often described as ethical commitments. But they primarily implicate legal risk management. The company has reportedly sought contractual assurances that its models will not be used for mass surveillance of U.S. citizens, development or deployment of autonomous weapons systems lacking context-appropriate human judgment and control and certain offensive or lethal decision-making applications without clearly defined human oversight. These concerns map directly onto existing legal frameworks.

Under IHL, states must comply with principles of distinction, precautions in attack and proportionality. Autonomous weapons systems raise unresolved questions about how those principles are upheld when military personnel use algorithmic systems in targeting decisions.

The ongoing negotiations around the United Nations Convention on Certain Conventional Weapons (CCW), GGE LAWS reflect a decade-long effort to determine whether new binding rules are required. The upcoming 2026 session from 2-6 March will once again debate draft language aimed at clarifying limits and obligations. Implicit in Anthropic’s stance on “fully autonomous weapons” is the commitment to retain some level of human judgement or oversight which has guided discussions at the CCW for the past decade. This need for human involvement is one of the few widely shared governance principles when it comes to AI in the military domain.

Against that backdrop, Anthropic’s stance seems to align with this emerging framework and interpretation, noting the reasoning behind the company’s stance does not publicly point to these initiatives. If states eventually adopt stricter rules on autonomous systems, companies deeply embedded in weapons development could face regulatory, legal, and reputational consequences. Emerging “responsible-by-design” approaches (articulated in part here and here), argue that legality and accountability must be considered and engineered into military AI systems from the very earliest stages of system inception rather than being retrofitted after deployment.

Ethically speaking, the guardrails Anthropic has set, while exceeding those of its competitors, are nevertheless quite minimal. Presumably the caveat against mass surveillance of U.S. citizens does not preclude the mass surveillance of other States’ citizens. And it might also not exclude the use of its AI model for AI targeting systems that do not constitute fully autonomous weapons but nonetheless pose a significant challenge to human oversight and judgement in the application of force. How ethical these ethical lines actually are might reside in how the legal stipulations are interpreted.

The Pentagon’s Position: Flexibility and Dominance

The U.S. military views AI dominance as essential to maintaining technological superiority over geopolitical competitors. In this framing, constraints imposed by a private vendor could create unacceptable limitations, although arguably, an asymmetric DoD dependence on the private technology sector is already well underway. If Anthropic refuses to meet operational requirements, other firms stand ready to step in.

Google dropped its pledge not to use AI for weapons or surveillance applications last year. OpenAI recently revised its mission language, removing explicit references to “safety.” xAI has also signaled willingness to accommodate defense demands. All three have entered contractual relationships for their products with the DoD. In a competitive defense contracting environment, companies that decline military use cases may simply be replaced by those that accept them.

The Pentagon also argues that its operations are governed by existing law. If a use case is lawful under domestic and international law, defense officials contend, a private company should not override that determination. This belies the most fundamental question at the core of the discussion: who gets to interpret the law especially in fast-moving technological domains that implicate militaries and private companies alike? And what happens when legal interpretations diverge? Furthermore, there is currently a gap in legal and regulatory guardrails specific to AI military systems and while some States are working towards filling that gap, the U.S. has been vocally against the establishment and promulgation of such measures, claiming they are barriers to innovation. The argument of being governed by existing law exploits this void in the current legal and regulatory system.

Military Reality vs. Civilian Accountability

The fracture exposed by this dispute reflects another, more insidious tension. Military operations are not civilian product deployments. They operate in secrecy, under extreme time pressure, often with life-and-death consequences. The “grey areas” invoked by defense officials are as real as the concomitant accountability gaps that threaten to arise through the development and use of these technologies.

AI-enabled decision support systems shape targeting, intelligence fusion, and operational planning in ways that are difficult to audit after the fact. Autonomous systems, even when nominally supervised by humans, influence choices in subtle but decisive ways. Anthropic’s insistence on human involvement in the operation of their systems is not necessarily just safety and ethical positioning. It may be an attempt to prevent accountability gaps from widening, something the international community should rally around and center at the heart of this discussion. It may also suggest a product warning in the context of warfare: LLMs are known for erroneous output (commonly incorrectly labelled as hallucinations) and may simply not be appropriate, or indeed ready, for use in contexts so varied, volatile and dynamic as warfare.

Global Governance at a Fragile Moment

This standoff arrives at a delicate geopolitical time for international governance. The CCW GGE LAWS has spent nearly a decade debating whether and how to regulate autonomous weapons systems. Progress has been incremental and fragile. States are still not in complete agreement over definitions, scope, and whether binding rules are even necessary. Discussions on broader military applications of AI are only beginning within the UN General Assembly, and although two resolutions (2024, 2025) have been adopted, it remains uncertain what form (if any) substantive governance processes will ultimately take.

This links directly to the discussion around Anthropic as private industry participation has been seen as crucial to any workable framework that may emerge from the UN discussions. Guardrails will not be able to succeed if developers do not implement them. More concretely: if Anthropic abandons its internal constraints under mounting US government pressure, momentum toward a legally binding instrument could stall. Other defence tech companies that are interested in supplying AI systems to the US and allied states could draw the conclusion that responsible design is not a good business case, undermining international efforts towards cementing and implementing such principles with tech company buy-in.

Other states may interpret the move as evidence that major powers intend to preserve maximum operational freedom. The optics on this matter as do the global precedents they may be setting. 

The Path Forward: Responsible-by-Design Anchored in Law and Ethics

The Pentagon/Anthropic dispute should not be framed as patriotism versus corporate virtue. It is about whether law, safety and ethics, will meaningfully shape the responsible integration of, and limitations for, AI into warfare.

Some elements of forging a responsible path forward ought to include:

1. Clear contractual articulation of permissible and impermissible uses of AI systems in military contexts and associated oversight practices;
2. Robust legal reviews under Article 36 of Additional Protocol I to the Geneva Conventions, adapted for AWS and AI-enabled systems;
3. Transparent internal governance within AI firms that aligns with IHL and human rights standards. This should also be extended to investors who are funding the development of these systems;
4. Continued multilateral engagement at the CCW and UNGA to clarify standards for context-appropriate human judgment, control and accountability.
5. Reinforced calls for greater scrutiny and guardrails for AI-DSS in armed conflict to promote coordinated, inclusive discussion of their risks and systemic impacts.

This is not just a Washington contract dispute. It implicates the way power will be negotiated between governments, investment firms and the private entities building the most consequential technologies of the century.

As the international community prepares for another round of negotiations next week on AWS, the message sent by this confrontation is clear: the battle over AI in war is not theoretical, academic or confined to the halls of diplomatic negotation…it’s playing out in real time with huge implications. Next week during the CCW session, some states will likely attempt to keep the status quo, others will attempt to propose guardrails and draw lines. Whether those lines are created and held, and whether they are respected and for how long and by whom, will have profound effects on the future of warfare, the protection of civilians, and more broadly on civil liberties and global governance.