Evaluating Digital Open Source Imagery: A Guide for Judges and Fact-Finders

[Raquel Vazquez Llorente is the Associate Director, Technology Threats and Opportunities, at WITNESS.

Sarah Zarmsky is an Assistant Lecturer and PhD Candidate at the University of Essex Human Rights Centre. In 2023, she was a Visiting Scholar at the Human Rights Center at the University of California, Berkeley School of Law. 

Daragh Murray is a Senior Lecturer in International Human Rights Law, and Fellow of the Institute for Humanities and Social Sciences, Queen Mary University of London, and a UKRI Future Leaders Fellow.]

Translations of this post are available in Arabic, French, Spanish, and Ukrainian.

This blog post introduces ‘Evaluating Digital Open Source Imagery: A Guide for Judges and Fact-Finders’, available in English, French, Spanish, Arabic and Ukrainian. The guide is the result of a collaboration between Queen Mary University of London, Open Society Justice Initiative, the TRUE Project, the University of Essex Human Rights Centre, the Hertie School Centre for Fundamental Rights, Mnemonic, the Human Rights Center at the University of California, Berkeley, the University of Oxford Bonavero Institute of Human Rights, and WITNESS. 

It is widely accepted that digital open source information can play a key role in the investigation and prosecution of human rights, humanitarian law, and international criminal law violations. Content recorded in the context of a protest can, for example, provide insight into the makeup and nature of the crowd, capture police responses, and help to establish a timeline of events. This content can come from sources as diverse as X, Instagram, Youtube, Snapchat, TikTok, or  a Facebook live stream. This open source information can shine a light into the events of a day and provide a variety of perspectives (and viewpoints) that were simply unimaginable fifteen years ago. But how can Courts or other accountability mechanisms leverage this information for accountability and justice? When these institutions receive open source content from a third party–such as a NGO, academic researcher or investigative journalist–how can they evaluate the merits of their analysis? How should they assess the quality and reliability of an open source submission? It is precisely this problem that ‘Evaluating Digital Open Source Imagery: A Guide for Judges and Fact-Finders’ seeks to address.

Over the past years, we have seen investigative bodies and legal professionals grappling with the nuances of digital evidence–its potential and pitfalls. In an era where digital information is abundant and readily accessible, open source information, or information that any member of the public can observe, purchase, or request, has become increasingly used as evidence  in legal proceedings and fact-finding missions. For example, at the International Court of Justice earlier this year, South Africa presented multiple open source videos to demonstrate Israel’s alleged violations of the Genocide Convention. At the International Criminal Court, the arrest warrant for Mahmoud Mustafa Busayf Al-Werfalli was based largely off videos posted to Facebook (although the case has since been closed due to the presumed death of the accused), and a digital platform compiling satellite imagery and open source photos and videos was used as evidence by the Office of the Prosecutor in the Mali cases. Open source materials were also central to the findings of the UN-mandated fact-finding missions on Venezuela and Myanmar, along with the Commission of Inquiry on protests in the Occupied Palestinian Territory and reports from the Group of Eminent Experts on Yemen. From social media posts and satellite imagery to witness videos, this digital content can provide crucial insights into alleged violations of human rights, humanitarian law, and criminal law.

Yet, the relative novelty of this evidence brings forth questions of authenticity, reliability, and probative value. Particularly in judicial proceedings, the increasing reliance on digital open source imagery requires a higher standard of technological literacy amongst those in the courtroom to ensure that principles of integrity and fairness of the proceedings are upheld. We hope this guide, published in May 2024, will emerge as  a valuable companion not only to judges but also to investigations carried out by international accountability bodies such as Commissions of Inquiry, Fact-Finding Missions, United Nations mandates and quasi-judicial institutions. 

Unpacking the Guide: A resource to demystify digital open source evidence

Written by a collective of lawyers, academic experts, human rights advocates, and open source investigators, the guide aims at demystifying the process of evaluating digital open source imagery, providing a roadmap for assessing the credibility, reliability, and probative value of this increasingly crucial form of evidence. Our aim was to create a comprehensive resource to help legal and investigative professionals at international institutions navigate the technicalities of digital open source information when it has been submitted by a party to the proceeding or a third party, or obtained via an external report. The document is not a guide on how to conduct open source investigations, as there are many resources already available and other outlets are better placed to provide that kind of guidance. Instead, this Guide is intended to assist judges, officially appointed fact-finders and other decision-makers in their assessment of open source information, by explaining some of the most common open source investigative techniques and providing insight into how open sources investigations are conducted.

The guide tackles the unique challenges posed by digital open source information, such as the lack of traditional indicia of authenticity, anonymous or unknown sources, the widespread dissemination of content across the internet, and the myriad of ways in which a piece of content may be ‘inauthentic’. We also acknowledge that while online open source imagery may not always be what it is purported to be–for multiple reasons, such as misattribution, editing, modification of metadata, staging of events, and the use of artificial intelligence to create or manipulate content–many of these can be identified using appropriate verification techniques. Moreover, inauthentic digital imagery may still have evidential value. 

This document builds on the Berkeley Protocol on Digital Open Source Investigations with a specific focus on supporting judges and other fact-finders. The guide is organized around a number of key issues that a court or fact-finding body may need to address in their evaluation of open source information, including determining the authenticity of a digital image, as well as analyzing relevant metadata, source, location, and time information. For each issue, the guide defines relevant terms and techniques, explaining their shortcomings, and provides examples that can help inform judges and fact-finders’ own evaluative process. In each section, there is a ‘Key Takeaways’ box, which gives a summary of the information for quick reference. There is also a glossary of common terms relevant to the evaluation of digital open source information. 

In the guide we cover pivotal considerations that a judge or investigative body may want to address with the submitting party, including content verification, metadata analysis, and the assessment of source credibility. Most importantly,  we emphasize the importance of corroborative analysis, reinforcing the idea that digital open source information should complement, rather than replace, traditional forms of evidence. We also offer a forward-looking perspective on the role of artificial intelligence and synthetic media in future legal proceedings, indicating how these developments may make their way into investigations for international crimes and human rights violations. As technology continues to evolve, so too will the nature of digital evidence. Our hope is that the guide stands as a critical resource for anyone involved in the evaluation of evidence. 

This document underscores the significance of interdisciplinary approaches in adapting to the digital evolution within legal frameworks, and it can serve as a bridge between the worlds of technology and international investigations. By examining and spelling out  the intricacies of open source imagery, we hope to contribute to ongoing efforts to enhance digital literacy among legal professionals, empowering them to make informed decisions about the admissibility and weight of digital open source evidence. Different jurisdictions will differ in their rules of admissibility, and in whether expert evidence is required, and, if so, what kind of expertise. However, we hope the guide’s impact will extend beyond the courtroom, to enhance the credibility and transparency of human rights investigations, fact-finding missions, and accountability mechanisms worldwide. 

Coming next: Launch events 

Launch events and informational sessions are currently being planned across different countries, please get in touch with Dr. Daragh Murray if you would like to host a launch event or informational session, virtually or in-person at d.murray@qmul.ac.uk.