Can I send PHI to an LLM?
Yes, but only if the LLM provider signs a BAA with you that covers the exact service and endpoint you call, and you treat it like any other vendor that handles PHI. Without a BAA, send only properly de-identified data, or keep the model inside an environment you already cover.
Last updated
Why the BAA is the deciding point
Under HIPAA, a vendor that creates, receives, maintains or sends PHI for a covered entity or a business associate is itself a business associate. An LLM API that receives prompts containing patient data fits that description, so the contract rules in 45 CFR 164.504(e) apply.
Many AI providers offer a BAA only on certain plans, accounts or endpoints. A consumer chat app, a free tier or a beta feature is often outside it. Check the BAA's scope against the exact API, model and features you use, including logging, file storage and any data retention settings.
What to set up before sending PHI
A signed BAA is necessary, not sufficient. The Security Rule still applies to the whole data path:
- 1Send the minimum PHI the task needs. Strip identifiers the model does not need to see.
- 2Turn off or limit provider-side retention and training on your data where the provider allows it, and confirm this in writing.
- 3Keep audit logs of which user or service sent which data, without copying PHI into general logs or analytics tools.
- 4Check every other vendor in the path: vector databases, observability tools, queues and email. Each one that touches PHI needs its own BAA.
- 5Add the AI feature to your risk analysis and test it with evals and guardrails, since a hallucination in a clinical setting is a safety issue, not only a quality one.
When de-identification is the better route
HIPAA does not apply to data de-identified under 45 CFR 164.514, either by removing the listed identifiers (Safe Harbor) or by an expert determination. For analytics, summarisation of trends or model testing, de-identified data can remove the need for a BAA. Free-text clinical notes are hard to de-identify reliably, so treat them as PHI unless an expert says otherwise.
Check it for your case.
Read the official text.
- 45 CFR 160.103, Definitions (eCFR)checked 2026-10-25
- 45 CFR 164.504(e), Business associate contracts (eCFR)checked 2026-10-25
- 45 CFR 164.514, De-identification of PHI (eCFR)checked 2026-10-25
- 45 CFR Part 164 Subpart C, the HIPAA Security Rule (eCFR)checked 2026-10-25
Common follow-up questions.
Is a provider's "HIPAA eligible" label enough?
No. The label usually means the provider will sign a BAA for that service. You still need the signed BAA in place before sending PHI, and your own safeguards around it.
Can staff paste patient notes into a public chatbot?
Not unless that product is covered by a BAA with your organisation. Most consumer chat apps are not, so this is a common source of accidental disclosures.
Does running an open model on our own servers avoid the issue?
It removes the AI provider from the data path, but your hosting provider and the servers are still part of the system that handles PHI, so the Security Rule and any hosting BAA still apply.
Related questions and guides.
- Does my app need to be HIPAA compliant?Question
- What is a BAA?Question
- How much does a telehealth app cost?Question
- AI agent vs workflow automationComparison
- RAG vs fine-tuningComparison
- GlossaryEvery term, in plain words.
- HIPAA guideWho HIPAA applies to and what it asks of software.
- Business associateGlossary
- AI automationHow OceanLab automates company workflows with AI.
