Mental health chatbots are increasingly used to support people with depressive symptoms, and large language models (LLMs) make these systems more flexible than rule-based chatbots. However, it remains unclear how well LLM–based chatbots deliver structured psychological interventions. A new study by Florian Onur Kuhlmeier, Leon Hanschmann, Melina Rabe, Stefan Lüttke, Eva-Lotta Brakemeier, and Alexander Maedche, published in the Journal of Medical Internet Research (JMIR) Mental Health, evaluates how well a GPT-4o–based chatbot delivered a behavioral activation intervention for young people with depression using sessions with artificial users and clinical expert assessment. It also identified limitations and potential refinements. 

The chatbot showed satisfactory overall quality. It performed best in mood assessment, activity planning, safety, clarity, and nonjudgmental communication. Key weaknesses were positive reinforcement, activity–mood monitoring, therapeutic rapport, and natural conversation flow. Experts considered the chatbot structured, clear, and safe but identified limited clinical reasoning, particularly when evaluating the appropriateness and feasibility of activities, barriers, solutions, and rewards. Overall, the chatbot delivered behavioral activation reliably at a procedural level but requires improved clinical reasoning, follow-up, and monitoring before evaluation with human users. The paper is online available: https://mental.jmir.org/2026/1/e94781