Image by Taylor Vick.

MIRI (the nonprofit Machine Intelligence Research Institute) warns that on the current AI trajectory, “the expected outcome is human extinction,” and exhorts that our top priority should be creating an off-switch. The fact that the very captains of AI industry are sounding the alarm lends unprecedented credibility to the dire prediction.

The purpose of this post is to crowdsource development of a literal circuit breaker system designed to trip automatically and cut power to AI compute in an emergency.1

This is akin to a “deadman’s switch,” but designed to trip before we die. It’s different from other proposed ‘kill switches,’ which leave selective action in the hands of government or industry itself.2

I refer to AI (what we have now) and AGI (AI that broadly equals or exceeds human cognition—what we are racing toward) jointly as AI, since the seeds of AGI-generated human extinction (“X-risk”) are already sewn into the AI field.

The idea is this:

• Install dumb, mechanical, decentralized, disconnected-from-the-internet circuit breakers at the power connection point to every data center and other facility operating AI computing resources above a defined threshold, which are

• defaulted to automatically trip and cut power, at the same time once per week the world over, unless the power is affirmatively left flowing by the human systems operator at each facility within not more than three hours before each weekly worldwide shutoff event.

This could:

• Start out as crypographically- protected software switches while a hardware system is built out;

• Be overseen by an international body that sets minimum standards and specs;

• Be implemented country-by-country through law, regulation, and inspection;

• Skirt the problem that AI developers can’t/won’t pump the brakes themselves despite asking to be regulated, because they’re in an international arms race. That is, an AI circuit breaker system wouldn’t in of itself require developers to pause, slow, or do anything differently at all now (but would function like laboratory sprinklers that self-activate unless overridden);

• Become known to AIs themselves, such that they might hesitate before eliminating the hands they need to keep the power on;

• Enable us to focus jointly on a single, simple (by comparison to other approaches), tangible, initial solution;

• Facilitate straightforward inspection and whistleblowing; and

• Serve as an emergency backstop while we continue to discuss and implement other guardrails.

Needless to say, there is no guarantee we would foresee an emergency, or heed it by following the circuit breaker protocol. AIs could conceivably build themselves into corporeal existence in order to prevent the power from being shut off, or to turn it back on. But it is farfetched at this stage to imagine they could conceal such a plot from us.

We simply can’t afford to stake our survival on nebulous goals that spawn endless expert debate and depend on levels of idealism that defy human nature and history, such as self-limiting government or industry action, pending more work on alignment (between human and AI goals). Before then, we need ways to bottleneck the genie’s escape and vacuum it up the moment it does.

This stop gap proposal need not be perfect to be worthwhile. We wear seatbelts even though they can’t prevent fatalities in sufficiently energetic collisions. Meanwhile, nothing in this proposal prevents us from pursuing more holistic and long term protections—against both AI and tech-capitalism run amok.

Implementing such a circuit breaker system would also require us to:

• Engineer the hardware interrupt switches;

• Develop necessary treaties, conventions, laws, regulations, best practices, and individual facility manuals;

• Keep improving the system;

• Keep pursuing other guardrails; and

• Make sure our older broadcast communications systems work in the event of major internet disruption.

In the best case scenario, we’re facing a massive, chaotic, viral, electronic swarm that will wreak havoc on world economies and make McAffee and Norton look like two klutzes waving butterfly nets at it.

MIRI points out that AI need not be malevolent to harm us, but simply pursue the tasks we assign it—or its own goals, like resources, efficiency, and survival—according to basic evolutionary principles which render the human race expendable, or worse, an impediment.

MIRI warns against the fallacy of thinking we even have a 50 percent chance of survival if we do nothing, likening this to saying, “I’m either going to win the lottery or lose it, therefore my odds of winning [are] 50 percent.”

I suspect that among the reasons we are not united in a frenzy is that this threat arives amidst global conflict, despair, and pre-existing existential challenges which already feel overwhelming. We are living inside of a surreal choose your own apocalyptic adventure game, in which life can feel cheap and expendable. This leaves us somewhat numb to our potential demise, or even feeling like we deserve to be replaced.
Conversely, the upsides of coming together to avert annihilation by AI, on top of continued existence, include a revitalized sense of our human worth, and the potential to bridge divides which pale in comparison to a common alien threat.

1

It’s absolutely bonkers to be having this conversation. I long for the quaintness of the Y2K bug.

This piece first appeared on Ben’s Substack.