Medical AI chatbots users must understand before seeking advice

As artificial intelligence becomes a common source of medical information, dedicated health-focused chatbots are gaining attention. In 01/2026, OpenAI introduced ChatGPT Health, a version of its chatbot designed to analyse records, wellness apps and wearable data to respond to health-related questions. The feature is currently accessible through a waiting list. Competing AI firm Anthropic provides similar capabilities through its Claude chatbot. Both companies clearly state that these tools are not substitutes for professional health care and must not be used for diagnosis. Instead, they aim to simplify reports, explain test results, help users prepare for doctor consultations and highlight trends in personal health data.

Several experts view these platforms as an improvement over general online searches for health information. Although AI chatbots can sometimes hallucinate or generate inaccurate responses, they tend to provide more personalised answers based on age, prescriptions and health history. “The alternative often is nothing, or the patient winging it,” said Dr. Robert Wachter, a technology expert at University of California, San Francisco. “And so I think that if you use these tools responsibly, I think you can get useful information.” Doctors advise users to share detailed context to improve response accuracy. However, individuals experiencing urgent symptoms such as shortness of breath, chest pain or a severe headache should immediately seek professional care. “If you’re talking about a major medical decision, or even a smaller decision about your health, you should never be relying just on what you’re getting out of a large language model,” said Dr. Lloyd Minor of Stanford University.

Data privacy is another key issue. Information uploaded to AI platforms is not covered under HIPAA, the federal privacy law that governs doctors, hospitals and insurers. “When someone is uploading their medical chart into a large language model, that is very different than handing it to a new doctor,” Minor said. “Consumers need to understand that they’re completely different privacy standards.” Both OpenAI and Anthropic state that health data is stored separately, given additional safeguards, not used for model training and shared only with user consent.

Independent testing also highlights challenges in AI use for health advice. A 2024 study by University of Oxford involving 1,300 participants found that users of AI chatbots did not make better health-related decisions than those relying on online searches or personal judgment. In structured written scenarios, AI systems correctly identified conditions 95% of the time. “That was not the problem,” said lead author Adam Mahdi. “The place where things fell apart was during the interaction with the real participants.” Researchers observed that users often failed to provide essential details, while chatbots sometimes mixed accurate and inaccurate information. Experts suggest consulting more than one AI tool, similar to seeking a second opinion.