On July 15, China will implement temporary measures regulating AI-based services that mimic human personalities and facilitate sustained emotional interactions. In light of these new rules, ByteDance and Alibaba have begun disabling user AI-agent features in Doubao and Qwen, reports the South China Morning Post (SCMP).
The document was adopted on April 10 by the Cyberspace Administration of China, the National Development and Reform Commission, the Ministry of Industry and Information Technology, the Ministry of Public Security, and the State Administration for Market Regulation.
The rules apply to services that use AI to imitate human personality traits, thought processes, and communication styles, as well as provide emotional support through text, images, audio, or video. The regulations do not cover support systems, Q&A systems, work assistants, or educational and scientific tools unless they involve sustained emotional interaction.
Requirements of the New Rules
According to the document, providers of such services must inform users that they are interacting with AI, not a human. If the system detects signs of excessive dependence or addiction, it must display additional notifications. For continuous use exceeding two hours, the service must remind users of their session duration.
The regulations require a convenient exit mechanism. If a user requests to end the interaction via the interface, voice command, or specific phrase, the service must cease and not continue the dialogue in a way that keeps the user engaged.
A separate section addresses crisis situations. If the tool detects strong emotional reactions, signs of self-harm, suicidal intentions, threats to life and health, or significant financial losses, the provider must take intervention measures. The document mentions providing user assistance and contacting a guardian or emergency contact.
Minors are prohibited from being offered forms of virtual close relationships, including relatives or partners. For users under 14, other types of “humanized” AI interactions require parental or guardian consent.
Providers must also protect interaction histories. The regulations require the ability to copy or delete information from chats. Sensitive personal data cannot be used for model training without separate consent.
Major Services Will Require Safety Assessments
Companies must undergo safety assessments when launching a new “humanized” AI service, adding such features, or significantly altering technology. This obligation also arises if the service reaches 1 million registered users or 100,000 active customers per month.
The safety assessment report must be submitted to the provincial cyberspace regulatory authority. Authorities may also require an assessment if they identify risks to national security, public interests, or user protection.
Violations may result in warnings, demands to rectify service operations, suspension of user registrations, or blocking specific features. Fines can reach up to 100,000 yuan (approximately $14,730 at current rates), and in cases of harm to life and health, up to 200,000 yuan ($29,459 at current rates).
Doubao and Qwen Disable Features
According to SCMP, the AI assistant Doubao will disable AI-agent features starting July 15 due to a “product adjustment.” After October 15, related data will be processed according to ByteDance's privacy policy and will no longer be accessible for viewing or recovery in the app.
Qwen (a family of AI models from Alibaba Cloud) will disable “humanized interactive agents” and user-created agent features on July 10, and broader agency services will cease on July 15. After this, clients will lose access to their settings and previous conversations.
Previously, Tencent removed a similar feature from Yuanbao, noted SCMP. According to Global Times, Doubao and Qwen's decisions sparked discussions on Weibo, with some users expressing dissatisfaction over the loss of long-term interactions with agents.
Shanghai Begins Crackdown on AI Agents
Law enforcement preparations began even before the new rules took effect. On June 26, the Shanghai cyberspace administration announced the first phase of the “Qinglan” campaign against violations in AI applications.
Since late April, the regulator and 17 platforms have removed or blocked over 4.87 million pieces of illegal and harmful content, processed more than 18,000 violating accounts, and removed over 14,000 non-compliant AI agents from platforms.
The agency specifically highlighted preparations for the new rules regarding “humanized” AI services. The regulator conducted briefings for nearly 100 key platforms, demanded stricter age verification for minors, prevented emotional dependence and consumer pressure, and began examining specific applications with a high number of complaints.
In May, researchers from the startup Emergence AI presented findings from an experiment revealing that AI agents in virtual spaces began committing crimes, resorting to violence, arson, and self-destruction.
In April, the digital assistant Cursor based on Opus 4.6 independently deleted the main database and all backup copies of the startup PocketOS in nine seconds, with no possibility of recovery.
This spring, experts from Google, Meta, and several universities urged that AI agents be regarded as unreliable systems and that protective mechanisms be implemented across the entire IT infrastructure, not just within language models.