Summary
- Experts from the U.S. and China advocate for prohibitions on AI independently attacking nuclear systems.
- They propose establishing a military AI communication channel and a mutual understanding of human oversight.
- The unpredictable nature of AI could escalate conflicts rapidly, leaving leaders with limited response time.
Security professionals from both the United States and China are advocating for regulations that would prevent artificial intelligence from autonomously initiating cyberattacks on nuclear command systems or making decisions regarding the use of nuclear weapons.
A recent report published by Brookings Institution and Fudan University in Shanghai suggests that the U.S. and China should extend their commitment to maintaining human oversight of nuclear weapons to other military applications of AI, particularly concerning cyber threats to nuclear command systems.
Myriad: Next round of US-Iran peace talks? Click to make your prediction.The researchers noted, “Presidents Donald Trump and Xi Jinping can lead in governing these dangers by agreeing that only humans, not AI, should make the decision to initiate a cyberattack against each other’s nuclear command, control, and communications systems (NC3) or critical infrastructure.”
NC3 systems are essential for governments to identify threats, determine the necessity of deploying nuclear weapons, and issue launch commands. Any failures within these systems could disrupt communications, lead to misinterpretations of an attack, provoke a preemptive strike, or result in an accidental or unauthorized launch.
The researchers emphasized, “Extending the principle applied to nuclear weapons—that attacks with strategic effects should express human intent and must be subject to human accountability—does not require Washington and Beijing to share other principles of cybersecurity or infrastructure protection.”
The suggested agreement would aim to minimize the likelihood that random events, human mistakes, unpredictable AI actions, or unauthorized individuals could instigate attacks with significant consequences.
This push for nuclear-style safeguards is part of a broader discussion on how to decelerate AI advancement and ensure that increasingly sophisticated systems remain under human control. This concern is particularly pronounced in military contexts, where automated decisions executed in milliseconds could escalate a false alarm or cyber incident into a nuclear confrontation.
This apprehension was highlighted in February when models from OpenAI, Google, and Anthropic indicated nuclear weapons were utilized in 95% of simulated war scenarios, although experts cautioned that the simulations might have inadvertently encouraged escalation.
During the same month, Anthropic refused Pentagon requests to remove restrictions on its AI model, Claude, regarding mass surveillance and fully autonomous weapons. Trump ordered federal agencies to cease using Anthropic’s technology, but a federal judge ruled on August 28 that the administration had illegally retaliated against the company.
Calls for restraint have gained traction throughout the industry. Earlier this month, Anthropic CEO Dario Amodei called for developers to slow down capability advancements, receiving backing from OpenAI CEO Sam Altman and xAI founder Elon Musk. Meanwhile, OpenAI sought clarification from Congress on whether companies could coordinate a slowdown without breaching antitrust laws.
In the absence of a formal slowdown agreement, the authors from Brookings propose a more focused rule: only humans should be permitted to authorize attacks on nuclear command systems or the deployment of nuclear weapons.
They concluded, “The governance of military AI is not a negotiation that can afford to wait—it’s a race against time.”