Expedient or Reckless? Reconciling Opposing Accounts of the IDF’s Use of AI in Gaza

Expedient or Reckless? Reconciling Opposing Accounts of the IDF’s Use of AI in Gaza

[Christopher Elliott is a war crimes researcher based in Canada. He has a Master of Anthropology from the Australian National University and a Doctor of Philosophy from King’s College London.]

As the Israel-Hamas war drags through its sixth month, one topic of immense and continuing controversy is the use of AI tools by the IDF as part of its military targeting process. Among these tools, two of them – the Lavender and Gospel systems – have become particularly controversial because of their prominent role in a devastating series of media reports by the Israel-based +972 Mag and The Guardian.

The Allegations

Succinctly put, the newspapers allege that in the initial weeks of the war, Lavender and Gospel were employed as cold enablers of mass death – the centrepieces of an overly permissive air campaign that contained “an element of revenge”. Citing numerous members of the Israeli intelligence community, the newspapers allege that the tools were used by Israeli targeting cells to automatically “generate” actionable targets at previously unprecedented rates, thereby facilitating the creation of a digitized “mass assassination factory”.

Gospel achieved this, the sources allege, by processing enormous amounts of data (too much for a single human to sift through) in order to track down “relatively junior” Hamas or Islamic Jihad operatives in their family homes.

Lavender, meanwhile, is used to make functional characterization assessments (to use a military term of art) about individuals – that is, the machine aggregates a bunch of data about a person and then spits out algorithm-informed suppositions about who is and who is not a member of Hamas.

Based on these reports, many lawyers, ethicists and activists have vocalized fierce opposition to such indiscriminate practices, with Renic and Schwarz observing that, as described, “target expansion, not refinement, is the point and outcome of such systems.” More philosophically, but no less important from an ultimate-fate-of-humanity perspective, Agenjo argues that such depersonalization in the crucible of war amounts to the “oblivion of human dignity”.

Lavender, in particular, has been labelled a “Crimes Against Humanity Machine” – a system whose information-processing efficiencies are enabling a state policy of spree killings from the air. 

The Israeli Response

Perhaps inevitably, the IDF is defensive of its employment of AI tools and rejects “outright” the claim that they form part of a “policy to kill tens of thousands of people” with maximum efficiency. In a response to The Guardian, the IDF also sought to correct the newspapers’ characterization of Lavender, saying:

‘The [Lavender] ‘system’… is not a system, but simply a database whose purpose is to cross-reference intelligence sources, in order to produce up-to-date layers of information on the military operatives of terrorist organizations. This is not a list of confirmed military operatives eligible to attack.’

The IDF does not appear to deny that it is leveraging digital technology to increase “the productions of targets large-scale” (as an IDF public affairs statement put it). But consistent with the all-pervasive message of its spokespeople, the IDF argues that its targeting staff “work[] day and night to quickly close circles”. 

In step with the IDF, the Israeli academics Tal Mimran and Gal Dahan also claim that the use of tools like Lavender and Gospel is only “a very preliminary” part of the IDF’s targeting process. Afterwards, they say, more careful analysis occurs when legal advisors and “more superior intelligence officers” receive the AI-generated recommendations in a “target room”. Mimran also claims that the machine-proposed targets are at times rejected and returned by the supervising lawyers and target engagement authority (i.e, the commander).

Reconciling the Accounts

Despite serious disagreements over framing, the two opposing accounts are not necessarily irreconcilable. Rather, on key aspects – such as the issue of what these AI tools actually do – the parties actually agree.

Based on the IDF’s technical description, Lavender appears to be an intelligence repository which allows users to visualize persons of interest (potential targets) in a viewable “layer”. Perhaps the layer is a map layer (like when you hit the “Coffee” button in your Google Maps app to search for nearby cafés) or a network diagram depicting people and their connections (as you might see on an office wall in a detective show). It is not made explicit. Either way, automation comes into play when the machine populates the layer with data. In Lavender’s case, the data is people. The compilation of this data appears to happen without active human prompting, but it is guided by pre-set parameters which describe what the targeteers are interested in. 

Lavender, then, is an example of “supervised classification” where the human creates a model declaring that “a Hamas member has X, Y and Z attributes” and the machine finds Palestinians who fit into those classes before plonking their profiles onto a viewable layer. As with all such models, the explanatory power is limited by the chosen classifiers. For example, if an indiscriminate net is cast, the model might define the X attribute as “fighting aged”, the Y attribute as “male” and the Z attribute as “connected to somebody in Hamas’ political, military or administrative apparatus”. Such a low-resolution model will not tell the analyst a great deal about the person’s targetability. Certainly, to kill someone based solely on that output would be reckless and undeniably criminal.

According to the newspapers’ intelligence sources, Lavender’s classifiers are set by the IDF’s cyber-intelligence organization, Unit 8200. Those sources also allege that “during the first weeks of the war” 8200 “tweaked” the search parameters to open up the aperture. We do not know what those parameters are, but four newspapers’ sources say that the new model “generated” 37,000 Palestinian men onto the visualized layer as “recommendations” – that is, potential targets. 

Gospel appears to work similarly, but for buildings. Imagine, for argument’s sake, that the IDF has in its digital intelligence holdings a sheaf of tagged, machine-readable reports about a particular militant. In some of these reports the same location might be mentioned again and again. These reports might be satellite images of a building, cell phone data (perhaps fixated on a mobile handset of interest) or reports arising from handler communications with covert human sources. As an information aggregator, Gospel appears to be able to say “hey, if you’re interested in this guy, you might be interested in this place”. It is wholly probable that in Gaza, that place would correspond with the militant’s family home. 

Used correctly, these tools do not make for necessarily immoral technologies. Indeed, if the IDF employed enough intelligence analysts to do further analysis on each of the locations recommended by Gospel and each of the 37,000 individuals recommended by Lavender, the layers might make for a useful start state in targeting work – like a user-tailored, specialty search engine. But data aggregation is not “analysis”. And herein lies the ethical and legal issue: what happens after the machine has had a first look? Is there second-line analysis and vetting before identified targets are sent to the effects team that handles the strike?

The accounts differ but they are at least partially consonant.

The newspapers’ intelligence sources allege that humans are in the loop only in the sense that they are a “rubber stamp” who avers that Lavender’s “recommendations” are valid targets and ought to be actioned. They allege that in practice the analyst spends about “20 seconds” deliberating before inputting their stamp of approval. As others have observed, it is difficult to see how an intelligence analyst’s good judgment can be appropriately exercised in a work environment driven by such a need for speed. 

The IDF does not explicitly deny the 20 second workflow for analyst review but emphasizes that its targeting processes rely on multiple sources of information; that a Lavender recommendation is not a “confirmed” militant; and that (as we frequently hear from the Israeli government) the IDF’s procedures are fully compliant with its international humanitarian law obligations.

A Breakdown in the Operational Planning Process? 

Both parties agree then that a human is technically in the loop. But are the newspaper intelligence sources correct when they claim that the timeline for review was recklessly compressed during the first weeks of the Gaza War?

The IDF’s own statements certainly suggest that its first wave of targeting decisions were done in an environment of great haste. 20 hours after the commencement of its air campaign on October 8th, the Israeli Air Force announced that 2,000 munitions had been dropped on 800 “targets”. 100 hours in, that number had ballooned to 6,000 bombs dropped on 3,600 “targets”. That is, targets were being nominated for strikes at a rate of approximately 36 targets per hour.

Some targets were likely already “validated” as known Hamas locations (for example, recurring rocket launch sites). But proper precaution would demand that Israeli targeteers conduct robust and up-to-date “collateral damage estimates” (CDE) before prosecuting a previously examined target.

In their defence of IDF targeting processes, Mimran and Dahan suggest that another AI tool used for pre-authorized targets is called “Fire Factory” – which they describe as “an amalgamate of phase 2 (target development) and phase 3 (capabilities analysis) of the targeting cycle”. For military people with professional familiarity of the targeting cycle, this should read as an immediate red flag. Indeed, what such an “amalgamate” would actually represent is a collapse of a very important firewall that is supposed to exist between intelligence staff (whose “sense” function vests them with target development responsibilities) and operations staff (whose “act” responsibilities involve selecting and employing the appropriate munition for a strike). In a properly functioning military organization, the two sides should be clearly distinct, if not separate, mostly to ensure that operations staff are not “situating the estimate” – that is, to ensure that the killers are not also deciding who should be killed with an air-launched munition. If in the first weeks of the war it is true that pre-validated targets were fed straight into Fire Factory for an “amalgamate[d]” Stage 2/3 check, that would represent a serious breakdown in any reasonable operational planning process – one which carries inherent significant risks for massive numbers of civilians.

The facts of tempo in the IDF’s public statements also raise questions about manpower availability and the mathematics of legal due diligence. The number of lawyers assigned to the IDF’s Southern Command (the headquarters responsible for prosecuting the war in Gaza) appears to be close-held information but Mimran claims that a maximum of 50 lawyers on a rotating watch is a “close” estimate of current force composition. 

If we assume that in the war’s first week 1.) some of these lawyers were not already mobilized; 2.) some were otherwise occupied with non-targeting duties; and 3.) the remainder were split between, say, 12-hour shifts, this leaves a (generous) estimate of perhaps 10 lawyers on-station doing targeting determinations at any one time for the referent period. (Perhaps the IDF will one day publish the number of Gaza-focussed legal advisors during the month of October 2023).

Given the speed at which these targets were being validated (36 per hour), it seems implausible that such a small workforce could sustain diligent review. Even putting aside questions about the reliability of the underlying intelligence reporting on October 7th (recalling that the Israeli national security apparatus collectively failed to detect and respond to Hamas’ attack plans), it is difficult to believe that these advisers could make well-informed, dispassionate legal determinations at that pace in a social milieu of nation-wide trauma and fury. Either way, important steps were obviously skipped – all of it backdropped by a zeitgeist of rapid retaliation.

Reconciling the two accounts is somewhat straightforward then. The Israelis correctly describe what the machines are designed to do (automated collation of intelligence information). They also describe how the targeting process is supposed to work, according to US co-opted targeting doctrine (which is cited in the Mimran and Dahan article). 

The newspapers’ intelligence sources, meanwhile, are describing how these tools are actually used in the wild. They became recommendation tools whose suppositions are promptly green-lit by target engagement authorities without much deliberation at all. 

Thus, a plausible reconciled hypothesis is as follows: before October 7th , AI tools like Lavender, Gospel and Fire Factory were created to make a targeting cell’s life easier. The tools were designed to achieve this by creating fancy layers that visualize data that meets fungible search parameters. These tools were designed to aid but not replace the human-in-the-loop, by enabling better intelligence analysis and statistically useful collateral damage estimates. In the white-flashing heat of an unexpected and spectacularly violent war though, the tools instead became more than that. They became a validation tool. In practice, they transformed into a digital check-in-the-box used to channel the outwardly aimed rage of the Israeli masses. In so doing, they became an accelerant for Israel’s collective revenge against Gaza’s civilian population. 

Topics
Artificial Intelligence, Autonomous Weapons, Featured, General, International Humanitarian Law, Middle East, Technology
5 Comments
Oldest
Newest Most Voted
Brian Cox
2 years ago

As an academic with diverse and extensive prior military experience, I make it a point not to be overly critical of analysis in public discourse of targeting procedures or operations. Not once in my career as a judge advocate (the last 7 of 22 total years in the military) did I refer to or even account for characterizations in public discourse of targeting processes when developing legal advice for an organizational (military) client. If I spent all my time now as an academic refuting every mischaracterization of military targeting procedures I encounter in the public domain, I’m not sure this would leave time for anything else. Since it doesn’t really matter that much for actual military operators, I often ask myself: why bother? Forget it, I tell myself. Just ignore it and move on.

Sometimes, though, ignoring egregious mischaracterizations of military targeting processes in public discourse is just not possible. Often, though not always, such an occasion emerges in response to characterizations developed by a publicist who also has at least some military experience. This is one such occasion.

Though a bio (external to OJ) indicates the author has “previously served in the Australian Army in infantry roles,” there are a number of glaring inaccuracies related to military targeting procedures that simply cannot pass without comment. Military experience is helpful for contextualizing LOAC in the applied setting, for example – but it does not guarantee deeply knowledgeable and unbiased engagement in public discourse with military targeting processes. Any number of specific concerns could be raised regarding the analysis above, but for present purposes I endeavor to focus just on the most important points.

– CDE and targeting selection are two separate components of the targeting process. Among the two, target identification and selection happens first. After a potential target is identified, assessing the degree of incidental harm expected, if any, from attacking the target is a separate process. The computer models described above are purportedly involved in identifying targets, not with evaluating the degree of incidental damage anticipated from an attack. As such, CDE has no place in this analysis.

– Speaking (rather, typing) of CDE – it is a specifc doctrinal methodology, not a shorthand reference for identifying anticipated incidental harm in general. Not every attack requires a formal CDE analysis. For dynamic strikes (though I can’t be too specific in the public domain), a person far away from the relatively calm environment of an operations center performs a truncated “field CDE” process. For attacks in defense of troops, no collateral damage estimate is required. That does not mean LOAC rules aren’t applied in each scenario (deliberate, dynamic, and self-defense), but a formal CDE analysis is required only for deliberate targets (which is a term of art). In any event, CDE methodology involves use of force policy and procedure. It is a tool for assisting with compliance of relevant LOAC obligations, but a “failure” to implement CDE methodology is not in itself indicative of LOAC compliance.

– On legal review of target selection packets – not all files require review by a military lawyer. In a counter-insurgency environment, when the number of potential targets to review is not expected to overwhelm staffing availability, having a lawyer review each file is certainly a best practice. I have been involved in thousands of such reviews, and I have very often identified concerns that simply would not be recognized by someone without highly specialized experience with LOAC in the applied context. On several occasions, raising these concerns has averted a targeting mishap that would have undoubtedly resulted in civilian casualties. My best case scenario as a military lawyer, though, was for my commanders or staff members to come to *me* with legal concerns. This is an indication that I’ve trained them well enough to be familiar enough with legal requirements to make their own decisions – and to raise concerns when they weren’t sure. And this experience was fairly late in the US involvement in Afghanistan, in an operational environment that does not even come close to the large scale combat operations currently occurring in Gaza. Suggesting that an inability to have a military lawyer review every targeting packet is an indication of an overly permissive organizational targeting culture is simply unsopportable. Legal review of every packet is a best practice when feasible, but doing so is not always possible.

– Insisting that the intel collection/analysis and effects enterprises remain separate is simply misguided. They work intimately together in the targeting cycle, and for good reason. Separating them would degrade the effectiveness of the targeting process, which would actually increase the risk to the civilian population. Creating a firewall between intel staff and ops staff, as the author suggests is vital, is a recipe for missed targeting opportunities and increased risk of civilian harm. In targeting operations, both communities (among others) work together in fusion cells – and the improved effects against the adversary that result actually improve conditions for the civilian population.

– The use of unnamed sources in media reporting raises major concerns. Maybe perspectives developed from these sources are unbiased and drawn from extensive personal knowledge. Maybe not. For a stark example of the deleterious effects on the reliability of media narratives developed extensively from unidentified sources, see the NYT coverage of airstrikes by the US military near Baghuz, Syria. An informed and unbiased scholarly analysis would treat such media coverage with extreme caution. The opposite manner of analysis would endorse such media coverage with no critical engagement and just treat it as though it is completely factually accurate. The above analysis is a pristine example of the opposite manner.

This critical engagement could go on and on, but I’ll stop here. The intended takeaway is that the manner of analysis above is incredibly dangerous if we care about the continued viability of international law involving armed conflict. Another completely reasonable interpretation of the factual background at the center of the above analysis is that the military involved, in this case the IDF, cares deeply about complying with existing LOAC obligations and has leveraged current technology to do so while improving targeting outcomes – which can actually reduce the risk of incidental harm from targeting operations. I’m not affiliated with the IDF or any advocacy organization, so I don’t have a personal stake in which narratives circulate in public discourse about whether the IDF complied with LOAC obligations while utilizing technology in the targeting process. As an academic, I do care about the continued viability of international law. When it is this easy to mischaracterize and manipulate doctrinal targeting processes to support an unsubstantiated claim that a military – regardless of national affiliation – has leveraged technology as an “accelerant [for] collective revenge against [a] civilian population,” I am absolutely concerned about the state of international law in public discourse. On occasion, then, this concern will invariably prompt an intervention such as that reflected in the current comment.

Brian L. Cox
2 years ago

Thank you for presenting the points of clarification based on my comment, Chris. I have a few points to raise in response to your clarifications, which I will do here. Since we’re engaged in a mutual exchange based on your post, I’ll refer my observations to you directly rather than making general observations.

– On CDE. Your engagement with the topic of CDE in the main post suggests you have some general familiarity with CDE methodology (CDEM), but little (if any) direct and in-depth knowledge and experience with the subject. Your clarifying comment about a soldier preparing to throw a grenade performing a “‘field CDE’ of sorts” further supports the conclusion regarding your level of knowledge with CDEM. To demonstrate why that is, please allow me to draw directly from US military doctrine for the explanation of the proper use of field CDE. In the process, two caveats are in order. One, this excerpt from US military doctrine is from an instruction that has since been superseded – and the update is not available to the general public. However, the definition and treatment of formal and field CDE is substantially similar in the update. Second caveat – obviously, the IDF has its own doctrine – for targeting processes and all other military matters. Even so, as you correctly suggest in the main post (though I don’t endorse the precise manner in which you do so), IDF targeting processes are heavily influenced by those of the US. The reverse is true as well, of course – between the DoD and IDF as well as between the DoD and many partners and allies. Specific differences and similarities notwithstanding, some manner of differentiation between what US doctrine refers to as Formal CDE and Field CDE is fairly standard among many partners/allies with whom I have worked over the years. The explanation of Field CDE from US military doctrine is suitable for present purposes. I will copy below a relevant excerpt from CJCSI 3160.01A, which was current in 2012, and which can be found in the public domain at this web address: https://info.publicintelligence.net/CJCS-CollateralDamage.pdf . The excerpt is drawn from Appendix A, Enclosure D, para. 2.g. (p. D-A-36, asterisks added for emphasis):

Field CDE is typically used in dynamic targeting when there is a time-constrained opportunity to engage a fleeting target. Field CDE differs from Formal CDE, which is conducted by certified/qualified CDE analysts, typically in a JOC/AOC/TOC. Field CDE is *typically conducted by JTACs [joint terminal attack controllers], FOs [forward observers], or by aircrew.* Field CDE is a degraded mode of CDE, and creates the opportunity for increased risk. A Field collateral damage estimate will typically be a verbal call, and should be *documented for future review/analysis.* CCDRs [Combatant Commanders] will establish authorities for Field CDE, as necessary. When authorizing Field CDE commanders should *ensure Field CDE utilizes existing concepts and reference tables derived from the CDM,* and that somebody in the Field CDE fires approval chain is a *certified and qualified CDE analyst.* Typically, CDE Level 4 analysis is not authorized for Field CDE.

As this doctrinal excerpt indicates, there is no such thing as “‘field CDE’ of sorts”. There is Formal CDE, Field CDE, and attacks that use neither. A soldier throwing a grenade implements neither. I have thrown hundreds of inert (though fused) grenades in training, dozens of live grenades also for training, and on multiple occasions I have employed live grenades in combat. Not once did I perform a Field CDE analysis when doing so. At the time, I was not a qualified and certified CDE analyst – but even if I were, CDEM is not utilized for that application.

Your treatment of CDE suggests you conceptualize it as a general process of assessing the risk and severity of collateral damage – such that an apparent failure to perform CDE is akin to reckless disregard for the effected civilian population. This is an inaccurate understanding of CDE. It is an established doctrinal methodology which, at the very core, is simply utilized as a standardized, repeatable method of identifying the correct Target Engagement Authority (itself a defined term). It is not a substitute for assessing the anticipated incidental harm, which still must be completed whether or not a specific attack requires Formal CDE, Field CDE, or no CDE. In any event, using the term “CDE” in a vague manner or suggesting that a soldier throwing a grenade conducts “‘field CDE’ of sorts” indicates a general lack of familiarity with actual CDE methodology. Most readers probably are not familiar enough with CDEM doctrine to know the difference. That is not the case for this reader – so I share these observations related to your treatment of CDE so a reader with little or no familiarity of the topic can make an informed assessment regarding your characterizations of the CDE process and the conclusions you develop based on this analysis.

– On legal review and claims by IDF of the lawfulness of operations. First, I should emphasize that the suggestion that legal advisors don’t adequately review each proposed target is based purely on conjecture. Assuming, arguendo, the veracity of the claim, this does not necessarily detract from an assertion of the lawfulness of operations. Operators and commanders are trained on how to comply with LOAC and use of force policy obligations – and usually that training is provided by legal advisors (for the IDF, that’s the MAG Corps). Operators and commanders are capable of independently assessing LOAC compliance, and they routinely do so without directly consulting a military legal advisor. That isn’t to suggest that we have no role in the operational process at all, of course – but legal review is not necessarily required to ensure legal compliance.

– On the firewall between intel and ops. It’s not that I suggest the firewall between intelligence analysts and operational staff is unimportant – what I’m saying is that no firewall exists between the two at all. Now, neither intel analysts nor operational staff section members have the responsibility for *approving* targets or achieving effects by employing munitions in the battlespace. So, there is a “firewall” of sorts between analysts/advisors and an actual attack. If my targeteers were developing intel *and* approving targets *and* launching attacks, no doubt that would be a problem. But they don’t – nor is it important for intel and ops personnel to have a firewall between them. In fact, it’s important that they do *not* have one.

– You may be comfortable as a journalist relying on unnamed and unidentified sources to build a narrative. Doing so is obviously an important aspect of working as a journalist – for a number of reasons. However, in this forum you aren’t telling a compelling story as a journalist. You are presenting what you describe as a plausible hypothesis, and the accusation at the pointy end of the hypothesis is that a military organization, on behalf of a sovereign state, is engaged in a widespread campaign of serious violations of international law. I don’t get the sense that there is much ground for accord between us on this particular occasion, but I do assess that we can agree that a military engaged in “collective revenge” against a civilian population would, in the process, be committing widespread and serious violations of international law. By sourcing your factual basis for these assertions from media reporting that is itself based primarily on unnamed and unidentified sources, you are in essence making allegations of widespread and egregious criminal conduct – and you’re basing the allegations on hearsay drawn from sources for which you have no ability to independently evaluate important factors such as credibility or reliability. Imagine being allowed to enter a courtroom and support a criminal conviction based on this information. By offering the plausible hypothesis you develop in the conclusion, your post has the effect of convicting a military of a campaign of war crimes in the court of public opinion. In reality, your analysis and conclusions are, in essence, a sprinkling of doctrinal military terminology glued together by conjecture, innuendo, and hearsay. This is why I believe this manner of analysis is so dangerous – and not to single you out specifically, because unfortunately this methodology is all too common in contemporary public discourse. If a conclusion (rather, plausible hypothesis) that a person or organization is engaged in extensive and egregious criminal conduct can be supported purely by conjecture, innuendo, and hearsay, the underlying law has lost all utility. I don’t typically share my own work when commenting on the work of other folks. On this occasion, I will share a link for a post (actually, part one of two-part post) exploring my concerns with the perceived legitimacy of international law in the context of narratives that emerged following the explosion at the al-Ahli hospital. I don’t share this for the purpose of self-promotion, but rather to provide more detail in case you’re interested regarding my concerns for the continued effectiveness of LOAC based on how it is routinely used in public discourse. Your post is an example of such discourse that contributes to my concern – which, incidentally, is why I was moved to comment (now, twice) in the first place. Link is here: https://sites.duke.edu/lawfire/2023/12/03/brian-cox-on-the-al-ahli-hospital-explosion-in-gaza-indicates-international-law-is-the-first-casualty-in-war-part-i/

– Apologies if it seemed that I was engaging in an ad hominem personal attack by looping in biographical information external to OJ and referring to you as a publicist. The reference to your prior military experience is, I believe, useful information since you’re engaging with a topic that inherently involves military operations (that is, thankfully there isn’t a lot of targeting activity going on in the purely civilian sector). My intent when referring to you as a publicist was to connote someone who is engaged in public discourse. I’ve spent the last several weeks deeply immersed in the writings of Hugo Grotius and his contemporaries, so it didn’t even occur to me that “publicist” in the present sense can also connote, essentially, a public relations representative. Just to clarify – I intended to use the term in the “person engaged in public discourse” usage, and it didn’t even occur to me to suggest that you are (essentially) a PR agent or that you have an inherent (and undisclosed) conflict of interest. For not realizing the potential alternate understanding and clarifying or adjusting my choice of terminology accordingly, I do apologize.