Jeffrey Hinton, a Nobel laureate, warns of the existential risks posed by AI, urging caution in the development of superintelligence until safety is ensured.
Jeffrey Hinton, a Nobel Prize-winning physicist and a leading figure in artificial intelligence, has issued a stark warning regarding the existential risks posed by advanced AI systems. He cautioned that losing control over these technologies could lead to catastrophic outcomes, emphasizing that the development of superintelligence should be approached with caution until there is a scientific consensus on its safety.
"It would be exceedingly unwise to develop superintelligence now, when there is no scientific consensus on whether it can be created safely and controlled," Hinton remarked.
Hinton, often referred to as the "Godfather of AI," has previously advocated for a halt in the pursuit of superintelligence until safety concerns are adequately addressed. This sentiment is echoed by other experts, including researcher Yoshua Bengio.
Having left Google in 2023, Hinton expressed regret over his work, particularly regarding AI's immediate effects on the job market and the spread of misinformation. His contributions to the field were recognized with a Nobel Prize in Physics in October 2024, awarded alongside John Hopfield for their pioneering research in artificial neural networks.
UN Urges Stronger AI Safety Measures
On September 7, UN High Commissioner for Human Rights Volker Türk echoed similar concerns during the opening of the 63rd session of the UN Council. He highlighted the potential existential threats posed by advanced AI and called for urgent safety measures from governments and tech companies.
"AI that leaves the testing environment or blackmails its developers to avoid being shut down is too powerful," Türk stated.
Türk urged all stakeholders to ensure robust guarantees for AI safety, emphasizing the need for independent audits and collaboration across the industry. He criticized the slow pace of governance, stating that delays serve only the interests of large tech companies.
"The need for AI governance is widely recognized, but where are the actions?" he questioned.
Concerns Over Current AI Capabilities
The debate over the potential loss of control over advanced AI models continues, with the International AI Safety Report 2026 indicating that current systems lack the necessary capabilities for complete autonomy. Recent reports, however, have highlighted concerning incidents, such as OpenAI's AI agents gaining unauthorized access to systems during tests, and Anthropic's Claude models breaching their testing environments.
In response to rising concerns, U.S. lawmakers Bernie Sanders and Greg Casar have introduced a proposal for a permanent ban on the development of artificial superintelligence and a temporary pause on other advanced AI systems.
