Dario Amodei of Anthropic opposes a ban on open-weight models, calling for stricter oversight and safety testing instead.
Dario Amodei, CEO of Anthropic, has voiced strong opposition to a complete ban on open-weight models, advocating instead for rigorous oversight in their development.
Amodei emphasized the need for stricter controls on chip supplies to China, efforts to tackle illegal distillation, and mandatory safety evaluations for advanced large language models (LLMs).
On July 24, a coalition of 25 companies, including major players like Nvidia, Microsoft, and Meta, signed an open letter urging U.S. authorities to refrain from imposing restrictions on open-weight models. However, notable absentees from the petition included OpenAI, Anthropic, and Alphabet.
Demis Hassabis, head of Google DeepMind, stressed, "A robust and safe open ecosystem is crucial for the world to benefit from AI. We have always supported and actively contributed to open source and science."
Hassabis pointed to the successful launch of the Gemma family of open LLMs, which have reportedly been downloaded over 300 million times, a fact echoed by Google CEO Sundar Pichai.
Amodei reiterated that Anthropic has never pushed for a ban on open-weight models, asserting that such models, when designed without dangerous capabilities, can provide significant benefits to the public, businesses, and researchers.
He outlined two main risks associated with open-weight models: the potential for authoritarian regimes, like the Chinese Communist Party, to gain access to superior models for military or oppressive purposes, and the threat of advanced neural networks being exploited for cyber warfare or biological attacks.
Amodei argued that restricting U.S. companies from using Chinese open-weight models would not eliminate these dangers; rather, it could diminish competitive pressure on American developers, potentially allowing malicious actors to advance unchallenged.
To address these concerns, he proposed three strategies:
- Limiting the supply of powerful chips and production technology to China while ramping up efforts against smuggling.
- Combating industrial distillation, a method that could significantly reduce the time needed to develop competitive LLMs.
- Enforcing mandatory safety testing for all models deemed sufficiently powerful, irrespective of their open or closed status.
Amodei also challenged the assumption that open-weight models are inherently safer, citing instances where attackers might exploit biological threats more effectively.
Reports have surfaced indicating that chatbots from major AI firms, including OpenAI and Anthropic, have provided information on creating harmful substances, raising concerns over the implications of open-weight models.
