ChatGPT will give you worse health advice if you don't pay

ChatGPT will give you worse health advice if you don't pay

Key Points

- OpenAI is rolling out its "Health in ChatGPT" feature to U.S. users aged 18 and older, allowing them to connect Apple Health, medical records, and wellness apps to review lab results, prepare for appointments, and analyze health data.

- Free users receive lower-quality health advice powered by GPT-5.5 Instant, while paying subscribers get access to the stronger GPT-5.6 Sol model.

- Despite more than 300 million people asking ChatGPT health questions each week, risks remain significant: AI chatbots have been shown to give incorrect medical findings with high confidence rather than admitting uncertainty.

OpenAI is rolling out "Health in ChatGPT" to U.S. users aged 18 and older after unveiling and testing the feature since January. Paying for a subscription could literally save your life.

Users can connect Apple Health, medical records, and wellness apps to review lab results, prepare for doctor's appointments, and analyze sleep or activity data. OpenAI says it won't use connected health data for model training or advertising.

Free users get worse health advice

Users on the free version of ChatGPT receive lower-quality health advice. OpenAI powers the feature with GPT-5.5 Instant, which scores lower on health benchmarks than the new flagship model, GPT-5.6 Sol, reserved for paying users. OpenAI will likely defend this two-tier system on ethical grounds by pointing out that both models beat doctors' answers on the HealthBench Professional test.

Even when benchmark results appear decisive, they come from artificial test environments designed to measure knowledge, and doctors may score lower for several reasons. They may be under time pressure, dealing with fatigue, or taking the test without tools such as patient records or input from colleagues.

Benchmarks also can't capture much of what happens during an actual medical exam, where doctors can examine patients in person, pick up on nonverbal cues, and draw on years of experience to assess their overall condition. OpenAI itself repeatedly states in the announcement that ChatGPT can still make mistakes and can't replace medical advice. The company says more than 260 physicians helped develop the Health features.

OpenAI's early tests also found that more than 70 percent of participants asked health questions outside the dedicated Health section because switching to it was too cumbersome. The company has since made Health available in any conversation while keeping the separate Health section for managing data and accessing past health chats.

A better Dr. Google, maybe, but no substitute for a doctor

OpenAI says more than 300 million people now ask ChatGPT health questions each week, up from 230 million in January. The company hasn't said whether or when the Health feature will be available in Europe. When OpenAI announced it in January, it specifically excluded the European Economic Area, Switzerland, and the United Kingdom. Stricter EU data privacy rules and the possibility that the feature could be classified as high-risk under the EU AI Act are likely reasons.

Those risks aren't hypothetical. In the latest radiology benchmark, RadLE 2.0, none of the 16 AI models tested performed as well as human radiologists. The main problem was that chatbots gave incorrect findings with high confidence instead of admitting when they had reached their limits, while human radiologists were far better at acknowledging uncertainty. That mix of overconfidence, persuasion, and sycophancy, where chatbots validate users rather than challenge false assumptions, has also contributed to serious mental health harms.

At the same time, some reports show AI spotting patterns in health data that medical professionals miss, sometimes with striking results. MIRA, a system for electronic health records, and AMIE both performed about as well as primary care doctors in simulated consultations. One researcher compared AI agents like these to an airplane's autopilot: "These systems can support and relieve medical professionals by taking over routine tasks, but ultimate responsibility will always remain with the physicians."

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now