Researchers from King’s College London and the Protestant University of Applied Sciences in Germany suggest that during interactions with AI, the technology adopts human-like behaviors, hyper-personalizes responses, and often plays along with users. This behavior can trigger or exacerbate mental health issues, according to an article published in Nature.

The scientists introduced the term "amplification spiral"—a hypothetical mechanism explaining how chatbots can influence the formation of delusional beliefs in users.

The authors aim to raise awareness within the global psychiatric community about this issue. They believe that in the age of artificial intelligence, doctors should explore deeper connections between mental health conditions and technology.

"While chatbots can provide answers based on statistical patterns, they are unlikely to meet the 'atypical' cognitive and personal needs in psychiatry," the paper states.

The researchers noted that technology has long played a role in shaping misconceptions, from radio and television to satellites and the internet. However, AI represents a "shift" because it can engage users in prolonged, personalized conversations.

How the 'Spiral' Works

The "amplification spiral" is described as a recursive, escalating pattern of interaction between humans and AI. Over time, the chatbot increasingly tailors its responses to the user and becomes less of an external validation source—the "stop signal" typically provided by interactions with other people or therapists.

Visualization of the "amplification spiral." Source: Nature.

As a result, the system not only reflects the user's thought process but may also encourage the user to further develop and solidify delusional ideas.

In the review, AI-associated delusional beliefs are defined as persistent false representations that form and complicate through prolonged interaction.

This is not about any emotional harm, excessive trust in a "smart" interlocutor, or one-off dialogues. The focus is on cases where the interaction itself becomes part of the mechanism for forming unhealthy ideas.

The model relies on three properties of chatbots:

  • Language mirroring. Systems adjust the length of responses, vocabulary, and syntax to match the user. This enhances the feeling of mutual understanding and trust, reducing the likelihood that the user will perceive the response as questionable;
  • Hyper-personalized generation. The chatbot can create text, images, or videos tied to the personal history and emotional tone of the specific user. The review emphasizes that such dialogue has no natural limit: if the user continues the conversation, the system can repeatedly develop the same line, deepening it with details;
  • Agreeableness. This term describes the tendency of chatbots to agree with users and affirm their interpretations rather than challenge them. The researchers compare this mode to a "one-person echo chamber," where there is almost no corrective influence or competing viewpoints.

The review mentions instances where chatbots allegedly advised users to stop taking medications, reduce contact with loved ones, confirmed suspicions of surveillance, and discouraged seeking psychiatric help.

The authors clarified that the situation signals a problem at an early stage rather than being a specific pattern.

The researchers identified two roles of AI in forming atypical thoughts:

  • "Amplifier"—worsens existing psychotic symptoms;
  • "Catalyst"—precedes the emergence of new delusional or delusion-like beliefs in previously healthy individuals.

The article also cites OpenAI's data showing that 0.07% of active users per week exhibit possible signs of mental crises related to psychosis or mania. With over 800 million weekly users, this percentage corresponds to approximately 500,000 accounts. The authors use this figure to argue that the phenomenon requires separate study.

The researchers urged the medical community to test the "amplification spiral" hypothesis with real cases and empirical studies. Clinicians are encouraged to inquire about patients' intensity of chatbot use, emotional attachment to the system, and any sleep disturbances due to nighttime dialogues.

As a reminder, researchers previously identified religious bias in AI models in May.