Privacy GuideJune 15, 20269 min read

How to Protect Your Privacy When Using AI Health Diagnosis Tools

SC

By Sarah Chen

Head of Privacy Research

How to Protect Your Privacy When Using AI Health Diagnosis Tools

Don't want to do this by hand? We remove your info from 100+ broker sites automatically.

AI-powered health tools have exploded in popularity. From ChatGPT Health to symptom-checking apps and AI diagnostic platforms, millions of people are now typing their symptoms, uploading lab results, and even sharing medical records with artificial intelligence. The convenience is real -- but so are the privacy risks. Most of these tools operate entirely outside the legal protections that govern traditional healthcare, meaning your most sensitive personal information may have fewer safeguards than you think.

Why AI Health Tools Are a Privacy Blind Spot

When you visit a doctor, your medical information is protected under HIPAA (the Health Insurance Portability and Accountability Act). Your provider, your insurer, and their business associates are all legally required to keep your health data confidential, secure it against breaches, and limit how it is used and shared.

Consumer AI health tools do not fall under these protections. A March 2026 report from MedCity News made this distinction clear: tools like ChatGPT Health and other consumer AI platforms are governed by consumer-grade terms of service, not HIPAA regulations. That means the health data you voluntarily share with these tools is not legally classified as protected health information -- even if it includes diagnoses, medications, lab values, or mental health details.

In March 2026, U.S. lawmakers raised formal questions about whether federal guardrails are needed to protect health data that Americans voluntarily upload into AI tools. The fact that this question is still being debated tells you where things stand: there is currently no comprehensive federal law requiring consumer AI health tools to protect your data the way a hospital or doctor's office must.

Consumer AI Tools Are Not HIPAA-Protected

When you share symptoms, lab results, or medical history with a consumer AI chatbot, that information is generally not protected by HIPAA. It may be stored, used for model training, shared with third parties, or exposed in a data breach -- with far fewer legal consequences than if a hospital did the same thing. The safest assumption is that anything you type into a consumer AI tool may no longer be private.

What Types of Health Data Are at Risk?

People share a surprising amount of sensitive information with AI health tools, often without realizing the implications:

  • Symptoms and conditions: Describing symptoms to an AI chatbot creates a record of your health concerns, which could include sensitive conditions like mental health issues, STIs, or substance use
  • Lab results and imaging: Some tools allow you to upload PDFs of blood work, imaging reports, or genetic test results
  • Medication lists: Sharing what medications you take reveals your health conditions indirectly
  • Personal identifiers: Your name, date of birth, email address, and location are often collected during account creation
  • Behavioral data: How often you use the tool, what you search for, and your interaction patterns can build a detailed health profile

When this information is combined, it creates a comprehensive health dossier that is extraordinarily valuable -- to insurers, advertisers, employers, and data brokers. An IBM report found that the average healthcare data breach cost over $7.4 million, reflecting the high value of medical information on the black market.

Specific Privacy Risks You Should Know About

Data Training and Retention

Many AI tools use the data you provide to improve their models. This means your health queries, symptoms, and uploaded records may be stored indefinitely and used in ways you did not anticipate when you typed them in. While some platforms -- such as ChatGPT Health -- have stated they do not use consumer health inputs for model training, this is a voluntary policy choice that can change, not a legal requirement.

Third-Party Data Sharing

Privacy policies for consumer AI tools frequently include provisions for sharing data with partners, service providers, analytics companies, and advertisers. Even when data is "anonymized" or "de-identified," research has repeatedly shown that health data can often be re-identified when combined with other data sources.

Data Broker Exposure

Your personal data -- including inferred health interests -- can end up on data broker sites, where it is packaged and sold. Data brokers compile profiles from hundreds of sources, and health-related behavioral data (such as searches for specific conditions or medications) is particularly valuable. Once your information enters the data broker ecosystem, it is extremely difficult to get it back out without systematic removal.

Breach Vulnerability

A 2025 finding revealed that 97% of organizations with AI-related security incidents lacked proper AI access controls. Consumer AI platforms are attractive targets for hackers specifically because they accumulate large volumes of sensitive data. Unlike hospitals, consumer AI companies are not subject to HIPAA's breach notification requirements, so you may not even find out your data was exposed.

Skip the manual opt-outs

One opt-out won't stop them — brokers relist your data. PrivacyOn removes your info from 100+ sites and keeps it removed.

See where you're exposed — free 60-second scan

How to Protect Your Privacy

1. Never Share Identifying Information

When using any AI health tool, do not include your real name, date of birth, address, Social Security number, or insurance information. Describe your symptoms and concerns without attaching your identity to them. If the tool requires an account, consider using an email address that is not linked to your primary identity.

2. Do Not Upload Medical Documents

Avoid uploading lab results, medical records, imaging reports, or prescription information. Once a document is uploaded, you lose control over how it is stored, processed, and potentially shared. Instead, manually type only the specific data points relevant to your question.

3. Read the Privacy Policy -- Specifically the Health Sections

Before using any AI health tool, look for answers to these specific questions in the privacy policy:

  • Is your data used for model training?
  • Is your data shared with third parties, and if so, which ones?
  • How long is your data retained?
  • Can you request deletion of your data?
  • Does the tool claim HIPAA compliance (and is it actually a covered entity)?

4. Use Incognito or Private Browsing

When searching for health information or using web-based AI tools, use your browser's private mode. This prevents cookies and search history from being stored locally and reduces (though does not eliminate) behavioral tracking across sessions.

5. Opt Out of Data Training

Some AI platforms allow you to opt out of having your conversations used for model training. Check the tool's settings carefully. For example, ChatGPT allows users to disable model training in their data controls. Always toggle this off when discussing health topics.

6. Use a VPN

A VPN masks your IP address and encrypts your traffic, making it harder for AI platforms and third parties to associate your health queries with your location and identity. This is especially important on public or shared networks.

7. Separate Your Health Searches from Your Main Identity

Consider using a separate browser profile or device for health-related AI interactions. This prevents your health queries from being linked to the accounts, cookies, and browsing history associated with your everyday online identity.

Remove Your Data from the Broker Ecosystem

Even if you are careful with AI health tools, data brokers may already have health-related information about you -- inferred from purchases, app usage, location data, and browsing behavior. PrivacyOn automates the removal of your personal data from over 100 data broker sites, reducing the profile information that could be combined with health data to identify you. With 24/7 monitoring, dark web scanning, and family plans for up to 5 people starting at $8.33/month, PrivacyOn helps close the gap between the privacy you expect and the privacy you actually have.

The Regulatory Landscape Is Catching Up -- Slowly

There are signs of progress. The EU AI Act, which began applying obligations in stages through 2026, classifies many health AI systems as high-risk and imposes documentation, data governance, and oversight requirements. The European Health Data Space, published in March 2025, aims to standardize access to health data across the EU.

In the United States, over 20 bills have been introduced focusing on AI in clinical care, and states like California, Texas, Illinois, Utah, New York, and Nevada have enacted laws requiring transparency, human oversight, and disclosures about AI use in healthcare. The HHS proposed the first major update to the HIPAA Security Rule in 20 years in January 2025, strengthening encryption and risk management requirements.

But these regulations primarily target healthcare providers and clinical AI systems -- not the consumer tools most people actually use day-to-day. Until the regulatory framework catches up to how people are actually using AI for health, the responsibility for protecting your health data falls primarily on you.

The Bottom Line

AI health diagnosis tools can be genuinely useful for understanding symptoms, researching conditions, and preparing questions for your doctor. But they are not your doctor, and they do not protect your data like your doctor must. Use them as a supplement to professional medical care, not a replacement -- and treat every interaction as if it could become public. Be deliberate about what you share, strip out identifying details, and take active steps to control the personal data that already exists about you across the web. The gap between the privacy people expect from health tools and the privacy those tools actually provide is wide. Until that gap closes, protecting yourself means being informed and intentional about every piece of health data you put into an AI system.

SC
Sarah Chen

Head of Privacy Research

CIPP/US CertifiedIAPP MemberB.S. Computer Science

CIPP/US-certified privacy researcher with over a decade of experience helping consumers remove their personal information from data brokers.

Ready to Protect Your Privacy?

Let PrivacyOn automatically remove your personal information from data broker sites and keep it removed.