Dario Amodei, CEO of Anthropic, calls for stricter AI regulations, advocating for mandatory safety checks and an economic safety net for workers.
Dario Amodei, CEO of Anthropic, has called for more stringent regulations on artificial intelligence, urging a shift from transparency measures to mandatory safety assessments for high-powered AI systems. In his recent essay, he emphasized the need for comprehensive oversight as the field advances.
Accompanying Amodei's essay, Anthropic introduced two key documents: the Advanced AI Framework for monitoring cutting-edge AI models and the Economic Policy Framework aimed at addressing potential job losses due to AI advancements.
Mandatory Audits and Risk Assessment
Amodei proposed that AI systems surpassing a predefined computational threshold should undergo mandatory external audits. Specifically, this applies to models that utilize more than 10²⁵ floating-point operations and are developed by companies generating over $500 million annually or investing more than $1 billion in AI R&D.
These audits would assess four primary risk areas:
- cybersecurity threats;
- potential for bioweapons development;
- loss of control over AI technologies;
- automated R&D that may exacerbate these risks.
If a model is deemed to present unacceptable risks, authorities should have the authority to prevent its deployment. Amodei likened this regulatory approach to aviation safety, asserting that "the most powerful AI models, like airplanes, must undergo technical testing and audits."
Furthermore, developers would be mandated to secure model weights, conduct routine safety evaluations, publish risk assessments, and swiftly report significant incidents.
Addressing Labor Market Disruptions
In his essay, Amodei warned that AI could lead to more significant and prolonged disruptions in the job market compared to past technological changes. He outlined three possible unemployment scenarios, ranging from 5% to unprecedented levels.
To mitigate these impacts, he proposed strategies including enhanced data collection on job displacement, incentives for job retention, retraining initiatives, and in dire situations, long-term income support measures like universal basic income. Anthropic plans to invest $200 million into the Economic Futures Research Fund for evaluating economic policies and will introduce a $150 million scholarship program for early-career professionals.
Protecting Civil Liberties
Amodei also addressed civil liberties, suggesting the creation of accountability standards for fully autonomous weapons, advocating for their prohibition in the U.S., and closing loopholes that allow mass surveillance data purchases from brokers. He emphasized the importance of ensuring that citizens have access to AI tools comparable to those utilized by the government in regulatory and legal contexts, as a means to prevent an imbalance of power.
This essay was released shortly after the introduction of Anthropic's latest models, Claude Fable 5 and Claude Mythos 5. The former is touted as the company's most powerful publicly available model, while the latter has fewer restrictions for select partners in Project Glasswing. Notably, Anthropic has previously reported signs of recursive self-improvement in AI, indicating a need for mechanisms to regulate the pace of AI development to align with institutional safety measures.
