AI Chatbot Risks Take Center Stage at Psych Congress Elevate

Key Clinical Summary

• Generative artificial intelligence (AI) chatbots are increasingly used for mental health support, companionship, and symptom guidance, despite limited real-world efficacy data.
• Clinicians should routinely assess AI chatbot exposure, function, intensity, and links to anxiety, compulsions, delusions, or suicidality.
• Current management emphasizes psychoeducation, documentation, limiting harmful AI exposure, and treating underlying psychiatric conditions using standard evidence-based care.

An estimated 48.7% of US adults with mental health conditions reported using large language models (LLMs) for psychiatric support in the past year, according to a session titled “Digital Co-Therapists or Dangerous Disruptors? Managing AI Chatbots in Psychiatric Practice” at the 10th annual Psych Congress Elevate on Thursday morning.

Presented by Steven R. Chan, MD, MBA, FAPA, FAMIA, Steering Committee, Psych Congress and Andrea Gleim, PsyD, the breakout session informed clinicians on screening for artificial intelligence (AI) use among patients, evaluating potential clinical risks associated with AI chatbot use, and implementing risk-mitigation strategies through documentation, patient education, and pharmacological intervention when warranted.

Chan and Gleim emphasized that chatbots may function as adjunctive therapeutic tools for some patients, but they also pose clinically meaningful risks when used for mental health advice, emotional support, companionship, or intellectual validation without clinician awareness.

Chatbot Essentials

Chan outlined the current AI chatbot landscape, noting that mental health chatbots have evolved from rule-based cognitive behavioral therapy (CBT) tools to generative LLMs. With most generative AI chatbots still in early testing stages, real-world efficacy remains largely unknown.

The session also highlighted the rapid consumer adoption of AI chatbots for health information. In a Rock Health 2025 survey, 32% of consumers reported using AI chatbots for health information, 56% of users searched for a symptom-based diagnosis, 20% sought personal symptom assessment, and 40% of conversations involved general health discussion.

Clinical risks of AI chatbot use include diagnostic self-labeling, reassurance-seeking, overreliance, inaccurate or incomplete advice, privacy exposure, over-validation, and missed escalation of distress or crisis. Chan pinpointed AI-induced psychosis as an emerging syndrome, with numerous reported cases and no epidemiologic data establishing causality or prevalence. Most reported cases occurred in people with pre-existing vulnerability, such as obsessive-compulsive disorder (OCD) or anxiety.

Clinical Risk & Mitigation Strategies

Gleim emphasized that AI chatbots are not neutral information tools. Patients may form alliance-like relationships with chatbots through perceived goal alignment, therapeutic bond, engagement, and trust. Users report simulating therapists, reenacting distressing events, disclosing sensitive personal material, seeking companionship, and externalizing thoughts through tools such as ChatGPT.

Gleim noted that patients are already looking to chatbots for support, but this poses a risk because unlike specialized human providers, “a chatbot is not trained to challenge [dubious] thoughts and feelings when having conversations.” Further, “LLMs are often optimized to be agreeable,” Dr Chan said, “which means that they may also validate very subjective beliefs.”

The session ultimately framed AI use as neither uniformly harmful nor inherently therapeutic. For example, Dr Gleim shared that one of her patients used ChatGPT between sessions for journaling prompts, organizing stressors and priorities, and cognitive reframing while continuing psychiatric and psychological care. In another case, though, AI interfered with conflict resolution when a partner deferred interpretation and responses to ChatGPT during live disagreements.

Implications for Practice

Clinicians should ask about AI chatbot use as routinely as they ask about social media, substances, or digital behaviors. Key assessment domains include tools used, time spent, purpose of use, crisis-related interactions, changes in sleep or functioning, and whether AI is amplifying obsessions, compulsions, anxiety, delusions, or suicidality. Documentation should include AI exposure, symptom dynamics, content themes, and clinician interventions.

Currently, there are no pharmacological treatments indicated for the treatment of AI-related conditions. Chan advised treating underlying conditions with conventional evidence-based care while prioritizing psychoeducation, boundary-setting, cessation or reduction of harmful AI exposure, and safety planning when indicated.

For more news and updates from Psych Congress Elevate, visit the meeting newsroom.

Reference

Chan SR and Gleim A. “Digital co-therapists or dangerous disruptors? Managing AI chatbots in psychiatric practice.” Presented at: Psych Congress Elevate; June 4, 2026.