Yudkowsky and Soares propose extreme measures to mitigate AI risks, including international oversight and potential destruction of rogue data centers.
In a provocative new book, Eliezer Yudkowsky and Nate Soares outline extreme measures to combat the perceived existential threat posed by artificial superintelligence. They assert that any nation constructing a significant data center without international oversight should be met with destruction through cyberattacks or airstrikes, even at the risk of nuclear retaliation. This alarming stance reflects their belief that humanity's survival is at stake.
Released on June 18, the Russian translation of "If Someone Creates It, Everyone Will Die" has sparked considerable debate regarding the future of AI. The authors, motivated by a genuine concern for future generations, advocate for a society where powerful computational resources are closely monitored and any research into advanced AI technologies is banned.
Foundation of Fear
Yudkowsky, who founded the Machine Intelligence Research Institute (MIRI) in 2000, initially aimed to develop superintelligence but shifted his focus to safeguarding humanity from it. This transition highlights a growing recognition of the challenges associated with aligning AI with human values. In a recent open letter, hundreds of researchers echoed this sentiment, stating that reducing AI-related extinction risks should be treated as a global priority.
"Reducing the risk of human extinction due to artificial intelligence should be a global priority—on par with pandemics, nuclear war, and other global threats," the letter stated.
Yudkowsky's and Soares's views extend beyond current AI technologies; they fear an intelligence that could surpass human capabilities, leading to catastrophic outcomes. They argue that the unpredictable nature of AI development, where models evolve based on training data rather than explicit programming, poses substantial risks.
Financial Implications
The authors suggest that a superintelligence could exploit financial systems to gain independence. Citing past hacks and recent examples of AI-driven fundraising scams, they warn that the financial resources necessary for an AI to operate could be obtained through illicit means.
Call for Global Action
To mitigate these threats, Yudkowsky and Soares propose a drastic overhaul of current practices. They argue for a global ban on powerful computational resources, suggesting that even minor advancements in AI research could lead to disastrous consequences. They advocate for international oversight over computational power and assert that nations must take aggressive action against any that defy these regulations.
"Other states must make it clear that this data center frightens them. They must demand that its construction cease. They must make it clear that if the data center is built, they will have to destroy it—through cyberattacks, sabotage, or airstrikes," the authors contend.
Despite the extreme nature of their recommendations, the authors acknowledge the complexity of achieving a globally coordinated approach to AI governance. They argue that without such measures, humanity faces an uncertain and potentially dangerous future.
Conclusion
The book culminates in a plea for humanity to rise to the occasion and address these existential threats. While Yudkowsky and Soares do not advocate for abandoning AI tools entirely, they emphasize the need for open discussions and proactive measures to ensure safety. Their radical proposals, however, have drawn criticism, with some experts questioning the feasibility and rationale behind their assertions.
