With hundreds of millions of people already using chatbots for advice, technology companies have begun marketing programs intended specifically for health questions. In January, OpenAI rolled out “ChatGPT Health,” which the company says can analyze medical records, wellness apps and wearable device data to help answer health and medical questions. Anthropic has offered comparable capabilities for some users of its Claude chatbot, according to the report.
Both companies, however, frame the tools as assistants rather than clinicians. They say the large language models are not a substitute for professional care and should not be used to diagnose medical conditions, while emphasizing uses such as summarizing and explaining complicated test results, helping people prepare for a doctor’s visit, and analyzing health trends that are buried in medical records and app metrics.
Some doctors and researchers who have worked with ChatGPT Health and similar systems see potential benefits compared with relying on general web searches. Robert Wachter, a medical technology expert at the University of California, San Francisco, said: “The alternative often is nothing, or the patient winging it,” adding that he believes responsible use can yield “useful information.” Wachter and others said one advantage of newer chatbots is their ability to respond with context from a user’s medical history, including prescriptions, age and doctor’s notes.
Even so, doctors stress that people should not treat chatbot output as definitive when symptoms suggest urgent problems. Wachter and others said users should skip AI and seek medical attention for warning signs that could indicate emergencies, including shortness of breath, chest pain or a severe headache. Lloyd Minor, dean of Stanford University’s medical school, said patients should approach AI tools with “a degree of healthy skepticism,” and he added that people should not rely on a large language model for major health decisions.
Privacy is another practical concern. Many of the benefits companies tout depend on users sharing personal health information, but the report says that information is not protected by HIPAA in the way it would be when handled by doctors, hospitals or insurers. Minor said: “Consumers need to understand that they’re completely different privacy standards,” and he noted that when someone uploads their medical chart into a large language model, the privacy standards differ from handing it to a new doctor. OpenAI and Anthropic said users’ health information is kept separate from other types of data, that it is subject to additional privacy protections, that the companies do not use health data to train their models, and that users can opt in and disconnect at any time.
Independent testing is still developing, the report said, and early studies suggest chatbots can “stumble” when interacting with people. It cites a 1,300-participant study by Oxford University that found people using AI chatbots to research hypothetical health conditions did not make better decisions than people using online searches or personal judgment. In the study, the systems correctly identified the underlying condition 95% of the time when medical scenarios were presented in a comprehensive, written form, but lead author Adam Mahdi said the problem “wasn’t” accuracy in that setting; “The place where things fell apart was during the interaction with the real participants.”
Mahdi and his team said the study identified communication problems, including that people sometimes did not provide enough information for the chatbot to correctly identify the issue. The report also says the AI systems sometimes responded with a mix of good and bad information, and users had trouble distinguishing between the two. The study, conducted in 2024, did not use the latest chatbot versions, including newer offerings such as ChatGPT Health.
Wachter pointed to one area where conversational tools may improve: the ability to ask follow-up questions and elicit key details. He said: “I think that’s when this will get really good, when the tools become a little bit more doctor-ish in the way they go back and forth” with patients. For now, Wachter suggested that some users can improve confidence by consulting multiple chatbots, similar to seeking a second medical opinion. He said he sometimes puts information into ChatGPT and into Google’s Gemini, and “when they both agree,” he feels “a little bit more secure that that’s the right answer.”
This report is part of AP’s “Be Well” coverage focusing on wellness, fitness, diet and mental health. The Associated Press Health and Science Department receives support from the Howard Hughes Medical Institute’s Department of Science Education and the Robert Wood Johnson Foundation, and the AP is solely responsible for all content.