A Response to David Zaring

First, I would like to thank David Zaring again for his comments on my essay: Empiricism and International Law: Insights for Investment Treaty Dispute Resolution. I was deeply humbled to read David’s thoughts about both the utility of the essay and the direction of my current research. David’s comments, however, raise a variety of issues worthy of a bit deeper exploration.

I wholeheartedly agree that empirical work in international law is decidedly not déclassé. As a proud alumna of the University of Minnesota Law School, I had the privilege of taking classes with Bob Hudec. I have profound respect for Hudec’s empirical exploration of international trade dispute settlement system and have been grateful to see others continue the empirical exploration of international economic law in the trade context. Perhaps more selfishly, as my research has developed, I have found myself wishing that Hudec was still with us so that I could benefit from his methodological insights related to the analysis of international economic dispute resolution. I would nevertheless hope that Hudec would have been pleased the development of empirical methodologies to areas of international law outside of the trade context. The work of junior scholars such as Oona Hathaway, Elena Baylis, Bill Burke-White, and Melissa Waters spring immediately to mind although there are certainly others engaging in research. In the meantime, I will look forward to hearing the remarks of David Trubek and my co-panelist Juscelino Colares at the forthcoming Society of International Economic Law as they both have critical observations about Hudec’s legacy to empirical assessment of international economic law phenomena.

David also makes an interesting point about how research in international investment law might evolve in the future. I am sympathetic to the reference to Wright & Miller’s recent exploration of content analyses and the value of using rigorous social science methodology to study the content of judicial decisions. No doubt this literature, and its underlying methodological rigor, will aid the evolution of methodological approaches involving the analysis of investment treaty arbitration awards. As the essay also suggests, there are other critical methodological approaches that could be likewise as this area evolves. For example, provided that scholars are sufficiently transparent in the description and analysis of their quantitative research, meta-analysis may be possible. As discussed eloquently in Jeremy Blumenthal’s article Meta-Analysis: A Primer for Legal Scholars, meta-analysis can synthesize empirical analysis across studies in order to summarize the research and identify variables influencing the findings of particular research. These additional methodological approaches only scratch the surface of potential ways to develop international law empiricism. One might even imagine – much like social science counterparts – the development of research methodologists, who are dedicated to the exploration and improvement of empirical methodologies, as a type of international law sub-specialty. But my suspicion is these last two evolutionary advances may be years in the offing.

Finally, in providing the cautionary observation about the need for training, David observes that inter-disciplinary collaboration can help bridge the methodological divide by providing much needed skills in this regard. I decidedly agree with David about the value that collaboration brings on the methodological side and offer two counter-points.

First, there are also practical benefits to be garnered from collaboration beyond methodological insights. Chief among these benefits is: sleep. Empirical research, while rewarding, can take time. The development of datasets can be labor intensive – even with the help of able research assistants (and I have been profoundly blessed in that department with the assistance of Melanie Neely and Jenna Perkins). Working in collaboration with others means that some of the most laborious aspects of empirical research – namely data collection – can be shared. In other words, it means that you are less likely to be sleep deprived and able to work more effectively. (And yes, there is empirical literature in related contexts to back up this claim.) Sharing of such tasks may also mean, provided proper research protocols related to inter-coder reliability are followed, that the reliability of data collection may be enhanced. It also means that research methodology choices can be considered with a view to considering multiple perspectives; and as none of us is perfect, the use of group-think to develop research and analysis can be invaluable. Collaboration also creates research efficiencies. For example, those with an expertise in or aptitude for the creation of graphs, tables and charts can develop them readily. I may, however, say this given my own graduate coursework at the University of Nebraska Law and Psychology JD/PhD program with Cal Garbin on multivariate research design and data analysis (see here and here) and the learning curve I have experienced in the creation of graphs during my work for Cal this summer.

The second counterpoint suggests that there may be a gap to fill within legal education. More particularly, while there are benefits to collaborating in an inter-disciplinary manner, one wonders whether such collaboration may be even more fruitful if legal scholars had access – for example in law school – to methodological classes to provide basic training. There are certainly useful programs for professors such as the Northwestern/WashU “bootcamp” or programs for quantitative methods at the University of Michigan, University of Essex or the European Consortium for Political Research’s program at the University of Ljubljana. Nevertheless, the essay explores the unique benefit of systematically providing the next generation of research assistants, lawyers and law professors with training in the law school context. Some law schools, such as Berkeley, Cornell, Harvard, Illinois, Leiden University, Northwestern, Penn, Stanford, University of Chicago, Vanderbilt, Washington University, and Yale have classes related to the empirical methods and the law (and apologies for the lack of a complete list in this regard for other law schools with separate courses focused on empirical methods). A casebook with an accessible teacher’s manual, such as the one being developed by faculty at the University of Illinois, goes a long way to filling this particular gap in U.S. legal training. I understand Empirical Methods in Law (Aspen, forthcoming), written by Bob Lawless, Jennifer Robbennolt and Tom Ulen, should be available for Fall 2009; and based upon the draft chapters I have seen, I am looking forward to getting my copy.

Ultimately, if we are willing to take on the challenge, we are at the first step of a journey of empirical assessment of investment treaty dispute resolution. There are inevitably places where we can grow and develop in collaboration or consultation with others. And that, at least in my view, is certainly a worthy undertaking.