When Machines Kill

What does it mean for a machine to “decide” to kill someone?

I’m in Berlin, attending an interdisciplinary expert workshop on robotic weapons, where this question has come up. My job was to brief the participants about international legal standards relevant to assessing governments’ reliance on unmanned weapons systems, such as armed drones.

The main purpose of the workshop is to discuss the possible need for new international standards to address weaponized robots and drones. Such standards might regulate the development, proliferation, and use of unmanned weapons systems, or even ban robotic weapons that are deemed “autonomous.”

The workshop is timely. The international trend toward warfare using unmanned weapons systems has accelerated rapidly over the past decade.

Government reliance on armed drones, in particular, has expanded dramatically. The most high-profile use of unmanned aerial vehicles is by the United States in Pakistan, where the Obama administration has carried out hundreds of drone attacks against suspected terrorists over the past year and a half. But the US is far from the only country that uses drones: Israel has a highly-developed drone program, and the UK has employed drones to kill so-called “high-value targets” in Afghanistan.

At present, more than 40 countries have access to drone technology, and a sizeable number of them, including Israel, the UK, Russia, Turkey, China, India, Iran, and France, either have equipped or are seeking to equip their drones with laser-guided missiles.

And drones seem to be just the beginning. Unmanned ground vehicles—robots with evocative names like the Crusher, the Raptor, and the Guardian—may also be equipped with weapons in the future. Already, South Korea has employed armed robotic sentries to protect its northern border.

“Risk-Free” Warfare?

So what ethical and legal concerns are raised by this increasing reliance on robotic weapons systems?

A number of participants at this workshop have alluded to the worrying possibility that “risk-free” warfare – military actions that are “risk-free” in the sense of not causing troop casualties – has a dark side. By eliminating a key disincentive to war, it may make war more attractive, and thus more likely to occur. Of course, any truly game-changing weapons technology alters the balance of military power in a potentially destabilizing way, by giving the country that controls the new technology a decisive advantage over its adversaries.

But there is a deeper, less consequentialist argument that seems compelling to many participants here. It is set out in the founding mission statement of the International Committee for Robot Arms Control, the group that convened this workshop: “machines should not be allowed to make the decision to kill people.”

The notion of robots making lethal choices is chilling; it smacks of the Terminator. But what does it mean for machines to “decide” to kill?

Remote Control vs. Autonomous Robotic Decisionmaking

The statement that someone was killed by a bullet does not attribute intent to the bullet, or imply that the bullet is morally or legally culpable. When someone in Pakistan is killed by an unmanned drone, the same is true: The decision to fire the deadly missile is made remotely, by someone sitting at a computer monitor in Langley, Virginia. Though that person is far from the site of the killing, remote sensors and video technology ensure that he is aware of the consequences of his actions. The locus of decisionmaking and responsibility remains human.

But participants at this workshop in Berlin have a different scenario in mind. Given rapid advances in hardware and computer technologies, as well as the military’s obvious eagerness to reduce the need for human input, there is a perceptible long-term trend toward combat robots and drones that are, in military parlance, “fully autonomous.” Not only would such unmanned vehicles move about in an autonomous way, without direct human input, they might be empowered to make autonomous targeting decisions.

A recent US Air Force strategy paper, describing the military’s long-range plans for unmanned aircraft systems, lauded the trend toward these autonomous systems. “Technologies to perform auto air refueling, automated maintenance, automatic target engagement, hypersonic flight, and swarming would drive changes across the [entire military spectrum],” it explained enthusiastically. “The end result would be a revolution in the roles of humans in air warfare.”

Professor Noel Sharkey, one of the organizers of the present workshop, agrees that the change would be fundamental, but sees the impact differently. He warns: “we are sleepwalking into a brave new world where robots decide who, where and when to kill.”

Algorithms or Autonomous Choices?

I don’t think I quite see the situation either way. Although the military may label these robotic weapons system “autonomous,” they would not actually be autonomous in any meaningful way: They would have no free will and would not make discretionary choices.

While complex robots and drones may not have much, or any, human input at the operational stage, their responses to visual and other stimuli are pre-determined through a series of programs and algorithms. Their human control is non-simultaneous, and may, for that reason, be more plagued with errors and miscalculations, but this does not mean that the robots themselves are “deciding” anything. Nor does it in any way shift the locus of decisionmaking authority and responsibility away from the humans and onto the robots.

I still differ from the military, however, on the question of whether such quasi-autonomous weapons systems should be granted targeting powers. My hesitance doesn’t reflect a fear of machines’ “decisions”; it just reflects a skepticism regarding whether, even a few decades in the future, machines can be counted on to have the sophisticated sensory and processing skills necessary to distinguish civilians from combatants, and to comply in other necessary ways with the laws of war.

JOANNE MARINER is a human-rights lawyer based in New York and Paris.

More articles by:

JOANNE MARINER is a human rights lawyer living in New York and Paris.

December 18, 2018
Charles Pierson
Where No Corn Has Grown Before: Better Living Through Climate Change?
Evaggelos Vallianatos
The Waters of American Democracy
Patrick Cockburn
Will Anger in Washington Over the Murder of Khashoggi End the War in Yemen?
George Ochenski
Trump is on the Ropes, But the Pillage of Natural Resources Continues
Farzana Versey
Tribals, Missionaries and Hindutva
Robert Hunziker
Is COP24 One More Big Bust?
David Macaray
The Truth About Nursing Homes
Nino Pagliccia
Have the Russian Military Aircrafts in Venezuela Breached the Door to “America’s Backyard”?
Paul Edwards
Make America Grate Again
David Rosnick
The Impact of OPEC on Climate Change
Binoy Kampmark
The Kosovo Blunder: Moving Towards a Standing Army
Andrew Stewart
Shine a Light for Immigration Rights in Providence
December 17, 2018
Susan Abulhawa
Marc Lamont Hill’s Detractors are the True Anti-Semites
Jake Palmer
Viktor Orban, Trump and the Populist Battle Over Public Space
Martha Rosenberg
Big Pharma Fights Proposal to Keep It From Looting Medicare
David Rosen
December 17th: International Day to End Violence against Sex Workers
Binoy Kampmark
The Case that Dare Not Speak Its Name: the Conviction of Cardinal Pell
Dave Lindorff
Making Trump and Other Climate Criminals Pay
Bill Martin
Seeing Yellow
Julian Vigo
The World Google Controls and Surveillance Capitalism
What is Neoliberalism?
James Haught
Evangelicals Vote, “Nones” Falter
Vacy Vlanza
The Australian Prime Minister’s Rapture for Jerusalem
Martin Billheimer
Late Year’s Hits for the Hanging Sock
Weekend Edition
December 14, 2018
Friday - Sunday
Andrew Levine
A Tale of Two Cities
Peter Linebaugh
The Significance of The Common Wind
Bruce E. Levine
The Ketamine Chorus: NYT Trumpets New Anti-Suicide Drug
Jeffrey St. Clair
Roaming Charges: Fathers and Sons, Bushes and Bin Ladens
Kathy Deacon
Coffee, Social Stratification and the Retail Sector in a Small Maritime Village
Nick Pemberton
Praise For America’s Second Leading Intellectual
Robert Hunziker
The Yellow Vest Insurgency – What’s Next?
Patrick Cockburn
The Yemeni Dead: Six Times Higher Than Previously Reported
Nick Alexandrov
George H. W. Bush: Another Eulogy
Brian Cloughley
Principles and Morality Versus Cash and Profit? No Contest
Michael F. Duggan
Climate Change and the Limits of Reason
Victor Grossman
Sighs of Relief in Germany
Ron Jacobs
A Propagandist of Privatization
Robert Fantina
What Does Beto Have Against the Palestinians?
Richard Falk – Daniel Falcone
Sartre, Said, Chomsky and the Meaning of the Public Intellectual
Andrew Glikson
Crimes Against the Earth
Robert Fisk
The Parasitic Relationship Between Power and the American Media
Stephen Cooper
When Will Journalism Grapple With the Ethics of Interviewing Mentally Ill Arrestees?
Jill Richardson
A War on Science, Morals and Law
Ron Jacobs
A Propagandist of Privatization
Evaggelos Vallianatos
It’s Not Easy Being Greek