A Florida man has sued OpenAI in San Francisco Superior Court, alleging ChatGPT gave him “extremely dangerous medical recommendations” that discouraged him from seeking care as his pulmonary embolism worsened – just one day before the company launched a feature that encourages users to share their health records with the chatbot.
The new capability, called Health in ChatGPT, allows eligible users to connect Apple’s Health app to the chatbot, giving it access to medical records, sleep, activity, and other data stored on the iPhone. OpenAI said the integration provides “additional context for health-related conversations” and allows “more personalized responses based on a user’s health information.”
“OpenAI launches Health in ChatGPT a day after a lawsuit over dangerous medical advice; shadow banning also raised as risk.”
The company acknowledged that users often bypassed a dedicated Health experience tested earlier this year, with more than 70 percent of health-related conversations occurring outside that space. Now, health data can flow into general ChatGPT chats with user permission.
But the launch comes amid serious concerns. The Florida plaintiff claims he now faces years of intensive recovery after the chatbot repeatedly told him his symptoms were not serious and urged him to stay home. An OpenAI spokesperson told The New York Times that ChatGPT’s terms of service make clear it is not intended for medical diagnosis or treatment, and said the company’s models have improved since the incident.
OpenAI insisted it worked with hundreds of physicians to test the feature for performance and safety, and noted in its announcement that ChatGPT “is not a doctor and should never be used as a substitute for qualified medical professionals.” The company also said it will not use connected health data to train its foundation models or serve ads.
Separately, a BBC podcast episode on Tech Life this week raised concerns about “shadow banning” – a practice that may limit access to health education messages for women and young girls. The episode explores what shadow banning is, how it works, and whether it could be affecting health advice online.
Together, the two developments underscore growing unease about the role of algorithms and AI in health information – particularly for vulnerable groups seeking reliable guidance.