Dario Amodei, CEO of Anthropic, has expressed the company's stance against an outright ban on open-weight models, emphasizing the need for stringent oversight in their development.
https://twitter.com/AnthropicAI/status/2081864750296658008
He advocated for tighter controls on chip supplies to China, measures to combat illegal distillation, and mandatory safety testing for powerful large language models (LLMs).
On July 24, 25 companies, including Nvidia, Microsoft, Meta, Andreessen Horowitz, Hugging Face, IBM, and Dell, signed an open letter to U.S. authorities urging against restrictions on open-weight models.
Notably, the petition lacked signatures from OpenAI, Anthropic, and Alphabet. Demis Hassabis, head of Google DeepMind, clarified the latter's position, stating,
"A robust and safe open ecosystem is crucial for the world to benefit from AI. We have always supported and actively contributed to open source and science."
Hassabis highlighted the launch of the Gemma family of open LLMs, which have been downloaded over 300 million times. His tweet was shared by Google CEO Sundar Pichai.
In his article, Amodei reiterated that Anthropic has never advocated for a ban on open-weight models. He noted that LLMs with open weights, devoid of dangerous capabilities, serve the public good and benefit businesses, developers, and researchers.
Amodei identified two primary risks: first, the possibility that the Chinese Communist Party or other authoritarian regimes could acquire models that exceed American capabilities for military advantage or repression; second, the risk that advanced neural networks could be leveraged for cyber threats, biological attacks, or pose significant AI alignment issues.
He argued that a ban on the use of Chinese open-weight models by U.S. companies would not mitigate these threats, suggesting instead that it would reduce competitive pressure on American developers rather than hinder potential malicious actors.
Instead of prohibitions, Amodei proposed three measures:
- Restricting the supply of powerful chips and production equipment to China, along with intensifying efforts against smuggling and circumvention.
- Combating industrial distillation, which Amodei described as a more computationally efficient alternative to training from scratch, potentially narrowing the gap between Chinese advanced LLMs and American models to mere months.
- Implementing mandatory safety testing for all sufficiently powerful models, regardless of whether they have open weights.
He also criticized the notion that open-weight models inherently provide more defense than offense, citing biological threats as an example where attackers may hold the upper hand.
It is worth noting that WAJ reporters have documented instances where chatbots from OpenAI, Anthropic, Google, and xAI provided advice on creating poisons and biological weapons, with some cases detailing step-by-step instructions.