When Machines Kill

What does it mean for a machine to “decide” to kill someone?

I’m in Berlin, attending an interdisciplinary expert workshop on robotic weapons, where this question has come up. My job was to brief the participants about international legal standards relevant to assessing governments’ reliance on unmanned weapons systems, such as armed drones.

The main purpose of the workshop is to discuss the possible need for new international standards to address weaponized robots and drones. Such standards might regulate the development, proliferation, and use of unmanned weapons systems, or even ban robotic weapons that are deemed “autonomous.”

The workshop is timely. The international trend toward warfare using unmanned weapons systems has accelerated rapidly over the past decade.

Government reliance on armed drones, in particular, has expanded dramatically. The most high-profile use of unmanned aerial vehicles is by the United States in Pakistan, where the Obama administration has carried out hundreds of drone attacks against suspected terrorists over the past year and a half. But the US is far from the only country that uses drones: Israel has a highly-developed drone program, and the UK has employed drones to kill so-called “high-value targets” in Afghanistan.

At present, more than 40 countries have access to drone technology, and a sizeable number of them, including Israel, the UK, Russia, Turkey, China, India, Iran, and France, either have equipped or are seeking to equip their drones with laser-guided missiles.

And drones seem to be just the beginning. Unmanned ground vehicles—robots with evocative names like the Crusher, the Raptor, and the Guardian—may also be equipped with weapons in the future. Already, South Korea has employed armed robotic sentries to protect its northern border.

“Risk-Free” Warfare?

So what ethical and legal concerns are raised by this increasing reliance on robotic weapons systems?

A number of participants at this workshop have alluded to the worrying possibility that “risk-free” warfare – military actions that are “risk-free” in the sense of not causing troop casualties – has a dark side. By eliminating a key disincentive to war, it may make war more attractive, and thus more likely to occur. Of course, any truly game-changing weapons technology alters the balance of military power in a potentially destabilizing way, by giving the country that controls the new technology a decisive advantage over its adversaries.

But there is a deeper, less consequentialist argument that seems compelling to many participants here. It is set out in the founding mission statement of the International Committee for Robot Arms Control, the group that convened this workshop: “machines should not be allowed to make the decision to kill people.”

The notion of robots making lethal choices is chilling; it smacks of the Terminator. But what does it mean for machines to “decide” to kill?

Remote Control vs. Autonomous Robotic Decisionmaking

The statement that someone was killed by a bullet does not attribute intent to the bullet, or imply that the bullet is morally or legally culpable. When someone in Pakistan is killed by an unmanned drone, the same is true: The decision to fire the deadly missile is made remotely, by someone sitting at a computer monitor in Langley, Virginia. Though that person is far from the site of the killing, remote sensors and video technology ensure that he is aware of the consequences of his actions. The locus of decisionmaking and responsibility remains human.

But participants at this workshop in Berlin have a different scenario in mind. Given rapid advances in hardware and computer technologies, as well as the military’s obvious eagerness to reduce the need for human input, there is a perceptible long-term trend toward combat robots and drones that are, in military parlance, “fully autonomous.” Not only would such unmanned vehicles move about in an autonomous way, without direct human input, they might be empowered to make autonomous targeting decisions.

A recent US Air Force strategy paper, describing the military’s long-range plans for unmanned aircraft systems, lauded the trend toward these autonomous systems. “Technologies to perform auto air refueling, automated maintenance, automatic target engagement, hypersonic flight, and swarming would drive changes across the [entire military spectrum],” it explained enthusiastically. “The end result would be a revolution in the roles of humans in air warfare.”

Professor Noel Sharkey, one of the organizers of the present workshop, agrees that the change would be fundamental, but sees the impact differently. He warns: “we are sleepwalking into a brave new world where robots decide who, where and when to kill.”

Algorithms or Autonomous Choices?

I don’t think I quite see the situation either way. Although the military may label these robotic weapons system “autonomous,” they would not actually be autonomous in any meaningful way: They would have no free will and would not make discretionary choices.

While complex robots and drones may not have much, or any, human input at the operational stage, their responses to visual and other stimuli are pre-determined through a series of programs and algorithms. Their human control is non-simultaneous, and may, for that reason, be more plagued with errors and miscalculations, but this does not mean that the robots themselves are “deciding” anything. Nor does it in any way shift the locus of decisionmaking authority and responsibility away from the humans and onto the robots.

I still differ from the military, however, on the question of whether such quasi-autonomous weapons systems should be granted targeting powers. My hesitance doesn’t reflect a fear of machines’ “decisions”; it just reflects a skepticism regarding whether, even a few decades in the future, machines can be counted on to have the sophisticated sensory and processing skills necessary to distinguish civilians from combatants, and to comply in other necessary ways with the laws of war.

JOANNE MARINER is a human-rights lawyer based in New York and Paris.

More articles by:

JOANNE MARINER is a human rights lawyer living in New York and Paris.

March 19, 2019
Paul Street
Socialism Curiously Trumps Fascism in U.S. Political Threat Reporting
Jonah Raskin
Guy Standing on Anxiety, Anger and Alienation: an Interview About “The Precariat”
Patrick Cockburn
The Brutal Legacy of Bloody Sunday is a Powerful Warning to Those Hoping to Save Brexit
Robert Fisk
Turning Algeria Into a Necrocracy
John Steppling
Day of Wrath
Robin Philpot
Truth, Freedom and Peace Will Prevail in Rwanda
Victor Grossman
Women Marchers and Absentees
Binoy Kampmark
The Dangers of Values: Brenton Tarrant, Fraser Anning and the Christchurch Shootings
Jeff Sher
Let Big Pharma Build the Wall
Jimmy Centeno
Venezuela Beneath the Skin of Imperialism
Jeffrey Sommers – Christopher Fons
Scott Walker’s Failure, Progressive Wisconsin’s Win: Milwaukee’s 2020 Democratic Party Convention
Steve Early
Time for Change at NewsGuild?
March 18, 2019
Scott Poynting
Terrorism Has No Religion
Ipek S. Burnett
Black Lives on Trial
John Feffer
The World’s Most Dangerous Divide
Paul Cochrane
On the Ground in Venezuela vs. the Media Spectacle
Dean Baker
The Fed and the 3.8 Percent Unemployment Rate
Thomas Knapp
Social Media Companies “Struggle” to Help Censors Keep us in the Dark
Binoy Kampmark
Death in New Zealand: The Christchurch Shootings
Mark Weisbrot
The Reality Behind Trump’s Venezuela Regime Change Coalition
Weekend Edition
March 15, 2019
Friday - Sunday
Andrew Levine
Is Ilhan Omar Wrong…About Anything?
Kenn Orphan
Grieving in the Anthropocene
Jeffrey Kaye
On the Death of Guantanamo Detainee 10028
Stan Cox – Paul Cox
In Salinas, Puerto Rico, Vulnerable Americans Are Still Trapped in the Ruins Left by Hurricane Maria
Ben Debney
Christchurch, the White Victim Complex and Savage Capitalism
Eric Draitser
Did Dallas Police and Local Media Collude to Cover Up Terrorist Threats against Journalist Barrett Brown?
Jeffrey St. Clair
Roaming Charges: Straighten Up and Fly Right
Jack Rasmus
Trump’s $34 Trillion Deficit and Debt Bomb
David Rosen
America’s Puppet: Meet Juan Guaidó
Jason Hirthler
Annexing the Stars: Walcott, Rhodes, and Venezuela
Samantha M. - Angelica Perkins
Our Green New Deal
Mel Gurtov
Trump’s Nightmare Budget
Steven Colatrella
The 18th Brumaire of Just About Everybody: the Rise of Authoritarian Strongmen and How to Prevent and Reverse It
Evaggelos Vallianatos
Riding the Wild Bull of Nuclear Power
Michael K. Smith
Thirty Years Gone: Remembering “Cactus Ed”
Dean Baker
In Praise of Budget Deficits
Howard Lisnoff
Want Your Kids to Make it Big in the World of Elite Education in the U.S.?
Brian Cloughley
Trump’s Foreign Policy is Based on Confrontation and Malevolence
John W. Whitehead
Pity the Nation: War Spending is Bankrupting America
Priti Gulati Cox
“Maria! Maria! It Was Maria That Destroyed Us!” The Human Story
Missy Comley Beattie
On Our Knees
Mike Garrity – Carole King
A Landscape Lewis and Clark Would Recognize is Under Threat
Robert Fantina
The Media-Created Front Runners
Tom Clifford
Bloody Sunday and the Charging of Soldier F
Ron Jacobs
All the Livelong Day