Microsoft’s Moral Abdication

Nearly 100 employees recently released an open letter to Microsoft demanding that the company cancel its nearly $500 million contract with the Army to develop an Integrated Visual Augmentation System (IVAS). I will let the employees explain why:

The contract’s stated objective is to “rapidly develop, test, and manufacture a single platform that Soldiers can use to Fight, Rehearse, and Train that provides increased lethality, mobility, and situational awareness necessary to achieve overmatch against our current and future adversaries.” Microsoft intends to apply its HoloLens augmented reality technology to this purpose. While the company has previously licensed tech to the U.S. Military, it has never crossed the line into weapons development. With this contract, it does. The application of HoloLens within the IVAS system is designed to help people kill. It will be deployed on the battlefield, and works by turning warfare into a simulated “video game,” further distancing soldiers from the grim stakes of war and the reality of bloodshed.

Intent to harm is not an acceptable use of our technology.

Not surprisingly, Microsoft has defended the contract. Here is what Microsoft’s CEO, Satya Nadella, said yesterday:

“We made a principled decision that we’re not going to withhold technology from institutions that we have elected in democracies to protect the freedoms we enjoy.”

This is a remarkable abdication of Microsoft’s responsibility for considering how its technology could be used — or misused. To begin with, as the contractual language makes clear, this is not a situation in which Microsoft is being asked to “withhold” from the Army technology it has already developed. Instead, Microsoft is actively developing the technology the Army needs for the “increased lethality” of its soldiers. That is a distinction with a difference — as Nadella seems to fully understand, given his misdescription of what Microsoft is actually doing.

Far more troubling, though, is Nadella’s invocation of democracy as a justification for developing IVAS. Scientists are not freed from the obligation to consider how a government may use their technology simply because that government is a democracy — especially when the technology in question relates to weaponry, and especially when they are being asked to develop new technology, not simply sell old technology to a new buyer. How different might things have been had Dr. Louis Fieser thought about how a military could use the napalm he developed specifically to increase the lethality of flamethrowers? The military almost immediately began to use napalm for area bombing, and it killed more Japanese during WW II than the atomic bomb. And, of course, we all know what napalm did during the Vietnam War.

The cautionary tale here, of course, is Robert Oppenheimer, the great physicist who ran the Manhattan Project. Oppenheimer helped develop the atomic bomb for the most noble of purposes — ending WW II. And he, too, invoked the “democratic excuse” when other scientists working on the project began to doubt the wisdom of developing a weapon with such destructive capability:

Such peculiar detachment stemmed from Oppenheimer’s conviction, at least at first, that simply to produce the weapon was his exclusive concern. How the bomb was to be used remained the prerogative of statesmen and military brass…. When Project physicists began to voice misgivings about the morality of their handiwork, to the point of wondering whether they could decently continue working at Los Alamos, Oppenheimer did his best to curb their doubts. Speaking to groups of the querulous and disenchanted, with gentle eloquence he admitted that atomic bombs would make fear a permanent feature of ordinary life, but then proclaimed that they might nevertheless mean an end to war… Oppenheimer’s assumption of moral leadership over his team convinced the scientists it was not up to them to settle political and military questions, and further assured them that the men qualified to settle such matters would come up with the right answers.

We all know how that turned out — the same democracy that helped saved the world from the Nazis used the atomic bomb to commit one of the most horrific war crimes in history. And over time Oppenheimer became increasingly remorseful about his role in creating the bomb:

But the war fever soon cooled. As Life magazine proclaimed the Los Alamos physicists superheroes of scientific intelligence, Oppenheimer was lamenting the subservience of science to innate human cruelty in an address to the American Philosophical Society: “We have made a thing, a most terrible weapon, that has altered abruptly and profoundly the nature of the world … a thing that by all the standards of the world we grew up in is an evil thing. And by so doing … we have raised again the question of whether science is good for man.” This public admission of personal despair at the moral collapse of the modern world’s leading intellectual enterprise could not be more nakedly penitent. The heartbreak of everlasting loss is unmistakable here: with the creation of the atomic bomb, the world will never again be what it once was. Modern science had permanently altered the nature of moral and political life.

[snip]

Oppenheimer’s struggle with despair never quite ended. His savage self-loathing reached its nadir when he met with President Truman and announced, “Mr. President, I feel I have blood on my hands.” Truman despised Oppenheimer’s theatrics on this occasion more than Oppenheimer despised himself. According to legend, Truman took out his handkerchief and presented it to the blood-soaked scientist, saying, “Well, here, would you like to wipe your hands?” After the encounter, Truman observed, “Blood on his hands, damn it, he hasn’t half as much blood on his hands as I have. You just don’t go around bellyaching about it.” Denouncing Oppenheimer as “a cry-baby scientist,” Truman, in a display of true manliness, insisted that nuclear war be conducted without tears.

To be absolutely clear, I am not comparing IVAS to the atomic bomb. Microsoft is not the Manhattan Project. The underlying principle, however, is the same: it is unacceptable for scientists to blind themselves to the consequences of their actions. Perhaps, after reflection and consultation, Microsoft would decide that the potential benefits of IVAS outweigh the costs. Perhaps IVAS will make it easier for American soldiers to distinguish between combatants and civilians and will thus, video game or not, reduce the amount of unnecessary suffering in future conflict. But it is simply unacceptable for Microsoft to justify not engaging in that reflection and consultation because it is selling its technology to a democratic government. Even great democracies do terrible things.