Outsourcing Military AI to Private Tech Companies Compared to Other Privatisation in Warfare, and Beyond: What’s New?

[Yiokasti Mouratidi is a Doctoral Candidate at the Swedish Defence University, jointly with Utrecht University]

IBM is widely recognised as bringing computers to the masses. But less known is its collaboration with the US military during the Vietnam War, providing computing power to advance data-driven military targeting. This collaboration reportedly included posting 250 IBM employees on overseas assignments to Southeast Asia, tasked with installing and maintaining data processing equipment for the US military. Such partnerships, and the ensuing civilian/military blurring of digital infrastructure and service providers, are now becoming a matter of course as states across the globe acquire military AI capabilities from private companies.

Yet, there is one question that researchers on this issue across different disciplines repeatedly come across: what, exactly, is new about the role being played by private technology companies designing and developing military AI capabilities? Is it any different to pre-existing privatisation phenomena in warfare? Why does this matter?

This post demonstrates and analyses the legal“newness” through a comparison of private tech companies’ role in military AI with two other privatisation phenomena in warfare: (1) private military and security companies and (2) conventional weapon manufacturers. These comparisons are apt given that they are all phenomena of  procured military equipment, sensitive equipment, sensitive works or sensitive services. In order to pave a way forward for future analysis, (3) it goes beyond warfare to draw from ongoing research on the conceptualisation and analysis of social media companies governing free speech, highlighting a similar need to go beyond the traditional, state-centric toolbox of international law to explore private tech companies’ role in the context of military AI production and acquisition. Finally, (4) it provides some concluding thoughts on the “what’s new” question and identifies an interconnected future research agenda on this phenomenon.

Private military and security companies (PMSCs)

As a starting point, currently neither PMSCs nor private tech companies are directly addressed or bound by an international legal treaty. Instead, their respective regulatory frameworks are made up of a patchwork of soft law instruments and voluntary commitments (see Aparac for PMSCs and Nadibaidze for military AI). Both these frameworks, however, are anchored upon binding state obligations, particularly IHL and IHRL. As such, both private actors are generally thought to de facto operate within international legal frameworks that are not directly binding on them, but rather their client-states (Afina, p. 29; see also Bruun and Goussac).

Moreover, private technology companies providing military AI capabilities would arguably come under the scope of the International Code of Conduct of Private Security Service Providers, a voluntary commitment, as  “operational and logistical support for armed or security forces” including “intelligence, surveillance, and reconnaissance activities”; however, the Code is:

“largely tethered to a kinetic paradigm of risk—one centered on the physical presence of armed personnel”

(Lubin).

By contrast, private tech companies’ involvement in the military AI context entails decision-making pertaining to data, software architecture, and interfaces, as well as post-acquisition maintenance. These decisions do not directly entail the use of force, particularly when talking about AI-decision support systems, but instead shape how militaries view the battlefield, for example through intelligence analysis, ensuing target analysis and recommendations. This indirectness nonetheless means that private tech companies are increasingly influencing how militaries conduct war (King).

In seeking to qualify their role from an IHL perspective, traditionally PMSC employees can be examined through the lens of direct participation in hostilities (DPH). However, this is underexplored in relation to private tech employees. To start, when the design and development stages happen in peacetime, the question of DPH does not even arise; once there is an armed conflict, it would foremost raise questions under jus ad bellum if employees were not located in a state that is a party to the conflict. Even when these collaborations continue in times of armed conflict, a narrow interpretation of the requirements for DPH, particularly on direct causation of harm in “one causal step” or as an “integral part” of a “coordinate military operation”, is a high threshold that may not be met by maintenance of existing equipment. As a parallel, the group of experts for the Tallinn Manual 2.0 was unable to agree whether it is sufficiently direct to develop and provide malware:

“in circumstances where it is clear that it will be used to conduct attacks, but where the precise intended target is unknown to the supplier”

(Rule 97, para 6).

Even in line with a broader interpretation, only the narrower employee group of “forward deployed engineers” raise a plausible argument of DPH. Challenges in qualifying private tech companies’ role raise conceptual issues in carrying out further analysis at the primary level of IHL norms, though there can still be recourse to analysis at the level of secondary norms of liability, particularly state responsibility (p. 21-22), and the criminal complicity of individual employees/executives.

In this regard, the legal challenges are arguably merely an extension of what Mégret coined the “vanishing battlefield”: the temporal and spatial parameters of “battlefields”, as the sites in which the exceptional norms of IHL come to apply, are no longer:

“discreet spaces insulated from the rest of society, confining military violence to a confrontation between specialised forces whose operation should minimally disrupt surrounding life”

(p. 135).

Technological developments enabling remote warfare, as well as “total war”, seen as the mobilisation of all of a state’s resources, including industries, pose challenges to the very idea of a “battlefield” as a normative and regulatory concept.

Conventional weapon manufacturers

By contrast to the above regulatory framework, conventional weapon manufacturers often face top-down regulatory regimes pertaining to arms control exports. While in principle an export control-approach also applies to military AI (see the Wassenaar Arrangement), in practice once broken down to its constitutive elements of algorithms, training data and models, this reveals a complex, interconnected web of commercial and public actors, civilian and military infrastructures, and financing models that go beyond the territory of a single state and the binary distinction of “friend-or-enemy” states, as well as blurring the line between commercial innovation and national security. Within the context of the European Union, for example, there is a shift towards “dual use by design” research which poses challenges to export controls, as well as proliferating the types of civilian actors involved in potentially military-relevant research.

When qualifying their role in IHL terms, a commonality with conventional weapon manufacturers is that it is unlikely their individual employees are DPHing. However, both invariably face the risk that the spaces (physical and digital) in which they operate may in some circumstances amount to military objectives, placing civilian employees (as well as civilian objects) in harm’s way. Iran’s recent attacks on AWS data centres, seemingly due to the company’s role within the apparatus of US and Israeli AI-enabled targeting, exemplifies how such risk can materialise in kinetic attacks.

A further similarity is that both conventional weapons and military AI capabilities (lethal autonomous weapon systems and AI-decision support systems) are captured by Article 36 legal reviews, putting the onus on states to ensure that new means and methods of warfare are not prohibited by I(H)L. Implicitly, this entails an examination of private actors’ design decisions in developing new (technological) capabilities. The question of what is “new” about military AI design decisions compared to conventional weapons has to be informed by what is “new” in “AI” itself: advances from rule-based AI to machine-learning based AI. Rather than deterministic, machine-learning based AI provides:

“decisional support or advice, based on predictive algorithms that basically infer standards to better monitor, predict and influence behaviour… these inferences are based on data analysis”

(p. 2)

i.e. the algorithms are informed by the data on which they are trained, with different potential levels of human input in such learning (p. 7). Aside from the crucial practical issue of having (or lacking) high quality and quantity of relevant data, ex-ante decision-making entails discretionary  “data wrangling” practices (see also Haas). States are now increasingly recognising that private companies play a “decisive role” through inter alia system design choices and data practices (Afina, p. 8).

Moreover, compared to conventional weapons systems, machine-learning based AI is iterative. According to Klonowska, currently used “frozen models”, that are static and require manual updating at a specific moment in time rather than dynamically updating in real time, require “Sisyphean” human labour of monitoring performance, model retraining and user-generated feedback loops to avoid performance degradation within the dynamic context of conflicts. This iterative nature can lead to issues of reliability and predictability, which in turn may “fragment decision-making responsibility by governmental personnel”, raising challenges for oversight during operations and accountability after the fact (p. 220).

The iterative nature of machine-learning based AI also means changes in public-private relations. First, as highlighted in the above IBM example, it entails a closer embedding of private sector employees within military structures and processes as a matter of course. This is thus more akin to buying a “service” (see also Barrett-Taylor and Ford) rather than a product (and perhaps therefore a point of convergence with PMSCs). While some states have expressed preference for a complete post-sale handover of maintenance from the company to the state, they also simultaneously identify a lack of technical expertise to carry out such maintenance in-house (Afina, p. 48). Second, from early on, militaries recognised that “[y]ou don’t buy AI like you buy ammunition”. This is now leading to a move away from waterfall procurement models, whereby states set out static product requirements, towards more agile approaches, procuring minimum viable capabilities that will continue to be developed iteratively.  

Beyond warfare: social media companies and freedom of speech

Against the background of this “newness”, the traditional international law toolbox may not suffice to capture the dynamic, transformative role being played by private technology companies, and the ensuing trade-offs states are making to capitalise on their industrial ecosystem. Such trade-offs raise questions of “who is really in control”, as recently exemplified by the high-profile (and ongoing) dispute between the US Department of War and private company Anthropic. While legal-regulatory discourse proceeds from the basis that governments are “in a symmetrical position relative to industry”, this is becoming an increasingly difficult assumption to maintain (see also Chesterman and Ferrari).

Further legal examination of this phenomenon arguably requires moving away from the positivist, state-centric conception of international law, given particularly the absence of any top-down, binding regulation in the military AI space. This includes letting go of the limited frame of non-state actors needing to fit within the neat boundaries of being “subjects” for legal analysis purposes (see Bianchi and Klabbers), and the related assumption that companies are only capable of being law-takers, probing instead whether they may actually also be law-makers (see also Bode and Huelss).

In this regard, food for thought can be gained from other fields of research in international law, which are more advanced in analysing the interplay of technology, private companies and international law. In particular, research on the role being played by private social media companies as the “new governors” of free speech entails examination of both algorithmic content moderation, as well governance architectures established within and beyond these companies. Analysing the role of Meta’s Oversight Board, di Stefano concludes that:

“difficulties linked to the ability to qualify this phenomenon through traditional understandings of international law are in fact not symptomatic of the inadequacy of international human rights law to address these issues, but rather instances of change in international law”

(p. 521)

In this regard, Krisch and Yildiz’s conceptualisation of change in international law invites one to think about how international legal rules are:

“in fact in constant movement, but the statist approaches leave us with few tools to capture and understand this dynamism in the life of international law”

(p. 2)

requiring instead an interdisciplinary approach towards this enquiry.

For some, the potential change for IHL is positive: AI stands to “objectivise” indeterminate language within IHL targeting rules, and improve compliance such that it would be irresponsible to not use AI in decision-support. On the flipside is not only the risk of worse IHL compliance, but that AI-enabled technologies will “absorb traditional legal systems” (p. 104) through quantification logics, and by moving technology:

“inside the cognitive practice of law, displacing or modifying human cognition in an unprecedented way”

(p. 106).

Of course, private companies will not have the final word (if there is such a thing) in determining the legal standards of appropriate AI design. States and their armed forces remain key actors in pre- and post-deployment of AI systems and ex-post facto processes of accountability and responsibility also stand to play an important role in adjudicating algorithmic design decisions (see here and Blanchard on the recent Meta and Youtube “social media addiction” trials in the US). But with such adjudication in relation to IH/CL violations being the (slow) exception rather than the norm, and the infamous lack of transparency in this field, the risk is that in the meantime, there will be irreversible harm in life/livelihood-and-death decisions and that, over time, de facto corporate practices can become legal conceptions of compliance.

Concluding thoughts

So, what’s new? Ultimately, this question comes down to being able to identify and nuance what is different, an analytically more important exercise than ascribing the label of being “new”. Across the comparison with PMSCs and conventional weapons manufacturers, I have shown that there are two analytically key differences which pose challenges to qualifying this phenomenon from an I(H)L perspective: (i) the changing nature of, and amplified significance of, ex-ante design decisions pertaining to machine-learning based AI; (ii) the iterative nature of these systems, leading to new forms of public-private relations throughout the AI lifecycle. A research framing to further analyse this phenomenon can be found by looking at the ongoing interdisciplinary inquiry into the role of social media companies in relation to freedom of speech.

Going down such a route of considering legal change and governance (and the role of private companies therein) further opens the door to important questions of legal (constitutional) and political theory. The venture capital logics of start-ups, the rise of “Silicon Sovereigns”, platformisation dependencies and monopolisation, all raise questions as regards democracy, justice, authority, legitimacy, and state sovereignty, to name a few. Further conceptualisation, theorisation and analysis of private companies’ role within this phenomenon is a precursor to examining these perspectives, as well as potential (legal) solutions within these trade-offs.