HIPAA-Compliant AI: How to Build AI on Patient Data Without Losing Control of It

You want an LLM to read charts, draft referral letters, and summarize records. Your security reviewer keeps asking the same one question before anything ships: where does the data go? You’ll need a better answer than the marketing page gives you.

What makes HIPAA compliant AI real: a signed BAA (the contract that lets a vendor touch patient data) makes the vendor eligible to see your PHI. Compliance comes from the deployment pattern, the configuration, and every hop the PHI takes. The BAA is the door; the architecture is the lock.

• In a 2023 EMNLP paper, Morris, Kuleshov, Shmatikov, and Rush inverted embeddings of clinical notes to recover full names: 92% of 32-token inputs came back exactly.
• In 2018, Yoo, Thaler, Sweeney, and Zang found hospital data stripped to HIPAA’s Safe Harbor standard still re-identified 3.2% of Maine patients and 10.6% of Vermont patients.

What you get: three deployment patterns, the BAA chain hop by hop, the leak points, and a pre-launch checklist to hand your security reviewer before the very first prompt goes out.

How do you build AI on patient data without losing control of it?

A signed BAA gets a vendor in the door; the system keeps the data safe. Run the model inside your own cloud account with your keys, and sign a BAA for every hop the PHI takes, from the vector store to logging and observability. Put a named reviewer with an audit log in front of every output headed for a chart. Log redaction and that review queue are what turn a vendor’s BAA into audit evidence.

Key Takeaways:

1. A signed BAA makes a vendor eligible; the deployment pattern makes the system safe. For production PHI workloads, run the model inside your own cloud account, with your keys and your logging stack.
2. Every hop your PHI touches needs its own BAA. Logging, observability, and evaluation tooling are the hops teams forget, and they’re where audits find the gaps.
3. The review queue is your audit evidence. Security Rule audit controls (45 CFR 164.312(b)) and integrity controls (164.312(c)) both land in one log: who saw the AI output, which source text they checked it against, what they did with it, and when.

Table of contents

1. What makes AI HIPAA-compliant?
2. Three ways to deploy AI on PHI, and what each puts at risk
3. The BAA chain for an AI feature, hop by hop
4. Where PHI leaks in AI pipelines
5. De-identification: when it helps and when it does not
6. The approval queue is your audit trail
7. The HIPAA Security Rule update and what it means for AI
8. HIPAA-eligible AI services on AWS, Azure, and Google Cloud
9. HIPAA-compliant AI pre-launch checklist
10. Why choose Topflight Apps for HIPAA-compliant AI development

What makes AI HIPAA-compliant?

Here’s the direct answer: AI is HIPAA-compliant when every system that creates, receives, stores, or transmits protected health information (PHI) is covered by a business associate agreement (BAA), configured with the required safeguards, logged, and limited to the minimum necessary data. It’s a configuration, and it’s yours to build and defend.

The label means less than the architecture

The label is a vendor’s claim about its own architecture. The architecture is what a security review will actually read. Every AI vendor claims the label, and it no longer tells a buyer anything: there is no federal HIPAA certification, seal, or registry for AI products.

OCR “does not pre-clear, certify, or endorse any product as compliant,” as Fisher Phillips LLP put it in June 2026. A vendor calling itself “HIPAA certified” is making a self-assessment.

HHS backs that up. The agency’s own FAQ says no certification process exists and no company can certify compliance (The HIPAA Journal, April 2025). When a vendor leads with “certified,” ask for the BAA and the covered-services list instead. Those are documents.

A HIPAA-compliant LLM is a configured system, not a model

So what is a HIPAA compliant LLM, exactly? A large language model running on a HIPAA-eligible service, under a BAA, in a configured environment, with logging and minimum necessary data. Everything that fills its prompt and context window is PHI. No model is compliant standing alone.

No model is compliant standing alone; the five layers around it are the compliance.

In March 2024, then-OCR Director Melanie Fontes Rainer told the AHIMA Journal that HIPAA applies no matter what emerging technology or platform you’re using, and that the Security Rule is technology neutral.

In April 2026, OCR Director Paula Stannard told the National HIPAA Summit that the Security Rule applies to AI “in the same manner as any other technology,” and named the three risks the agency watches:

• data leakage,
• data poisoning,
• and PHI exposure without a BAA.

Both directors are describing the same thing from two angles: HIPAA regulates data, and the software processing it inherits the rules. A model is a component. The compliant thing is the system it runs in: the covered service, the signed BAA, the safeguards, the logs, the minimum-necessary limit. Buy the system, and the model comes with it.

HIPAA does not prohibit AI

HIPAA does not prohibit AI. The duty sits with covered entities (the hospitals and clinics that hold the data) and their business associates (everyone they hire to touch it), and that includes any AI vendor processing PHI on your behalf.

If a vendor processes PHI on your behalf, it’s a business associate whether its marketing page says so or not. For the wider AI in healthcare compliance picture, our guide walks through how the safeguards become design decisions in a real build.

Three ways to deploy AI on PHI, and what each puts at risk

There are three ways to run AI with PHI. Same model, three deployment patterns: what changes is where the PHI sits and who holds the keys.

The same model, three ways to deploy it. Only Pattern 3 keeps the PHI inside the account you already govern.

Pattern 1: the AI SaaS tool with a BAA

Pattern 1 is the HIPAA-compliant AI tools category: SaaS products you sign up for this week and roll out to staff. Hathr AI signs a BAA for every account and runs Claude models on GovCloud.

The ChatGPT example in the table, ChatGPT for Healthcare, sits on OpenAI’s current BAA-eligible list, alongside the sales-managed ChatGPT Enterprise with Regulated Workspace; the consumer app and ChatGPT Business are not covered. A HIPAA-compliant ChatGPT tier exists, and its coverage has edges.

The right use is individual staff productivity, live in days. The failure mode: your logs and visibility sit with the vendor, and its BAA covers listed features, not the whole product.

Pattern 2: the model vendor’s API under a BAA

Pattern 2 is the model vendor’s API under a BAA: your app calls OpenAI or Anthropic directly, and the PHI crosses the wire during inference. Is OpenAI HIPAA-compliant?

For the API, case by case: OpenAI signs API BAAs per customer, Modified Retention is required, and roughly 20 endpoints are HIPAA-eligible, while live Web Search is not. Anthropic’s wrinkle: its Covered Models require 30-day retention and don’t combine with Zero Data Retention, a 2026 rule.

The right use is a prototype or a low-volume feature; if you’re launching a healthcare AI prototype, this is the fast lane. The failure mode: retention and abuse-monitoring defaults you have to configure away, per org and per project.

Pattern 3: the model inside your own cloud account

Pattern 3 is the recommendation for production PHI: the private LLM healthcare pattern. The model runs inside your own cloud account, so the PHI never leaves an account your security team already governs. Your keys, customer-managed; your private networking; your logging stack. The full settings table for the three clouds lives in the HIPAA-eligible services section.

Anthropic also signs BAAs for its first-party API and Claude Enterprise with an in-product HIPAA mode; consumer plans and beta features are out.

The builder alert: a BAA covers services, not a company

The alert: a BAA covers services, not a company. OpenAI’s ChatGPT BAA covers an explicit feature list:

• chat,
• Spaces,
• projects,
• GPTs,
• Voice,
• Canvas,
• and skills.

It excludes agent-owned connections, cloud Work browsing, scheduled tasks, Sites, and improved memory. AI agents in healthcare live on that edge: agent-owned connections sit on the excluded side. If ChatGPT HIPAA compliance is your question, that list is the answer.

Rule: check the vendor’s current covered-services list before every new integration. Every time.

The BAA chain for an AI feature, hop by hop

The BAA question is per hop, not per vendor. Every service your PHI passes through needs its own agreement, including the ones teams forget to ask about. You have a BAA for the app and the cloud; the back half of the list is where audits find the gaps. Here’s the full chain for a typical AI feature.

The walk, hop by hop

Nine hops, two questions each: does it see PHI, and is there a BAA? Walk them in order.

1. App and API layer: sees PHI; your BAA, your logging.
2. Cloud hosting: sees PHI at rest and in transit; your cloud’s BAA covers it.
3. Model inference: sees PHI; the model vendor’s or cloud’s BAA, scoped to the exact service and model.
4. Vector database: the retrieval-augmented generation (RAG) store; sees PHI-derived embeddings; treat it as a PHI store and get the vendor’s BAA.
5. Document storage: sees PHI; your cloud’s BAA.
6. Logging and monitoring: sees prompts and completions unless you redact; your log vendor’s BAA.
7. LLM observability and evaluation tools: see every request by default; BAA availability varies by tool, from signed and dedicated to none at all.
8. Analytics: sees PHI if you send it; your analytics vendor’s BAA.
9. Support tooling: sees PHI in payloads unless scrubbed; your support stack’s BAA, or no PHI.

Nine hops, nine BAA questions. The highlighted three are the ones teams forget to ask.

The observability hop is the one to check hardest

Ask five LLM observability vendors whether they’ll sign a BAA and you get five different answers, sometimes from five tools in the same pricing tier.

Langfuse signs one through Ironclad for a dedicated HIPAA cloud region (Oregon us-west-2, hipaa.cloud.langfuse.com) on Pro and higher plans, and tells customers not to send PHI before the BAA is signed, which is the right instinct for a vendor to publish; see Langfuse’s HIPAA compliance page.

Datadog’s LLM Observability is an Extended Eligible Service under Datadog’s BAA, so a signed BAA covers PHI in traces.

Helicone publishes no BAA document at all; its HIPAA mention sits in a paid-plan feature row, and self-hosting with omit-logs is its path to not storing content. LangSmith lists HIPAA on its trust center and can provide a DPA through support, but publishes no self-serve BAA page; its position is a platform claim.

Weights & Biases publishes a real BAA, but it covers only Dedicated Cloud BYOB deployments; Weave on the multi-tenant cloud is explicitly out of scope.

One category, five different positions, and each one signed differently. That’s why this hop gets checked hardest.

The subcontractor rule

A vendor’s BAA must flow down to its subprocessors as subcontractor BAAs. The BAA you signed covers what its chain covers, so check each hop’s list before launch. Ask for the subprocessor list and the covered-services list in the same email. If a vendor can’t show you the chain in five minutes, you just learned something about the chain.

Where PHI leaks in AI pipelines

Most teams watch the model call. The exposure that fails an audit sits around it.

1. Application logs and traces: they capture full prompts and completions. Fix: redact PHI at the logging layer before anything ships.
2. Embeddings and vector databases: an embedding of a clinical note is derived from PHI; treat the store as a PHI store. Fix: the vector store goes on the BAA list and inside the governed account.
3. LLM observability and prompt-management tools: they store every request by default. Fix: pick a tool with a signed BAA and bounded retention, or self-host.
4. Evaluation datasets: real encounters copied into spreadsheets and notebooks. Fix: synthetic or redacted eval sets, stored where PHI is allowed.
5. Fine-tuning data: sent to a vendor without checking whether training is covered under the BAA. Fix: verify the fine-tuning path’s BAA coverage and retention before uploading.
6. Abuse monitoring: some hosted services retain samples for review unless you apply for the exemption. Fix: apply for the opt-out and verify the setting.
7. Error-tracking and support tools: crash reporters and help desks receiving PHI in payloads. Fix: scrub payloads at the boundary.

Embeddings are PHI with extra steps

An embedding is the model’s compressed representation of a text, and researchers keep showing that the compression leaks. In Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023), Morris, Kuleshov, Shmatikov, and Rush inverted embeddings to recover 92% of 32-token inputs exactly, and recovered full names from embeddings of MIMIC-III clinical notes.

The consequence: a vector database of clinical-note embeddings is a PHI store, and it gets the same BAA and access controls as the notes themselves.

Observability tools keep what you did not ask them to keep

Langfuse’s cloud stores traces indefinitely by default, with data access windows up to three years on Pro and Enterprise plans. Datadog’s LLM Observability keeps spans for 15 days by default, but keeps prompt-registry entries and dataset records for 3 years and metrics for 15 months. You asked for debugging; the tool keeps an archive.

Abuse monitoring keeps samples unless you opt out

Azure OpenAI may sample flagged prompts and completions for review, automated by default with human review in limited cases, and stores them in a per-resource abuse-monitoring store; Limited Access customers can apply to modify abuse monitoring and verify with ContentLogging=false.

The default posture differs by cloud: Bedrock invocation logging is off and opt-in, Azure stores prompts only in that abuse-monitoring store, and Google keeps abuse-monitoring prompt logging on unless an exception is granted. The abuse monitoring opt-out is a setting you verify after you apply it.

Fine-tuning data is PHI in a new place

Fine-tuning sends your data somewhere new, so check the path before you upload. Azure OpenAI keeps fine-tuning data in the customer’s tenant and doesn’t use it for training without permission, but the abuse-monitoring copy can hold for up to 30 days and the Global tier copies outside the region; PHI fine-tuning belongs in Standard tier.

The riskiest PHI exposure is rarely the model call. Design log redaction before the first prompt goes to production; every leak on this list has a named fix you can ship.

De-identification: when it helps and when it does not

De-identification is the mitigation everyone reaches for first, and the one that fails most quietly. Strip the identifiers, the thinking goes, and the data stops being PHI. It narrows exposure but does not end it.

The two methods, and why free text defeats them

HHS defines two methods. Safe Harbor de-identification removes 18 identifier types, plus actual knowledge; Expert Determination requires a qualified expert to conclude the re-identification risk is very small and to document the method and result.

• Safe Harbor is a checklist;
• Expert Determination is a judgment call that has to be written down.

Free-text clinical notes are the hard case for both methods. In Maine and Vermont statewide hospital data, researchers matched 28.3% and 34% of named individuals, and even after HIPAA Safe Harbor-compliant redaction, 3.2% and 10.6% were still re-identified (Yoo, Thaler, Sweeney, and Zang, Technology Science, 2018). Names, dates, and rare conditions hide in prose, where a checklist can’t find them.

LLMs make it worse. An adversarial LLM attack re-identified 9% of clinical notes even after ClinicalBERT masked all identified PII (Morris, Campion, et al., AMIA 2025). The canonical demo is older: Sweeney re-identified 43% of Washington State patients (Harvard Data Privacy Lab, 2013).

Redact before inference, re-insert after, where the workflow allows

The workable pattern: PHI redaction or tokenization before the prompt goes to the model, and re-insert the values in the output after, only when the workflow can carry them through.

The trade-off is real: redaction can strip the exact context the model needs to answer well. Use it where the workflow lets you carry the mapping; skip it where the context matters more than the risk. If the answer needs that context to be right, redaction is the wrong tool for that workflow.

Redact before the model, re-insert after, and only where the workflow can carry the mapping.

The approval queue is your audit trail

The review step is where the pipeline’s promises become evidence. Everything before it is configuration; the queue is where the auditor looks.

The review log is a Security Rule control

Any AI output that reaches a chart, a patient, or a payer passes a named reviewer first: clinician review for anything clinical, named and logged. It’s the human-in-the-loop step the pipeline cannot skip, and it’s where the ambient AI scribe case sits: every draft note queues for sign-off.

The Security Rule backs this with two implementation specifications. Audit controls (45 CFR 164.312(b)) obligate covered entities and business associates to implement mechanisms that record and examine activity in systems containing ePHI.

Integrity controls (45 CFR 164.312(c)(1)) require protection against improper alteration or destruction, and (c)(2) requires mechanisms to corroborate that ePHI was not altered or destroyed without authorization. Read them together and you get one requirement: the system has to show what happened, and show that nothing was changed without authorization.

The review queue is where both land in one log, and that log is your audit trail: who saw the AI output, against which source text, and what they did with it.

Four design requirements

Four requirements, in the order a request passes them on the way to the chart:

• an approval queue every AI output passes through
• the source text shown beside each AI claim, the check that catches a hallucination
• edit tracking with an override path
• an audit log of who approved what and when

Each of these is a requirement. Skip one and the review log stops being evidence. The log they produce is the control the Security Rule asks for.

The review screen doubles as the audit record. The source text is beside the claim, and every action is logged.

Planning an AI feature that touches PHI? Talk to a Topflight Apps engineer about where your data goes before you pick a model.

The HIPAA Security Rule update and what it means for AI

The biggest HIPAA Security Rule change in twenty years has been pending since January 2025. It’s still not law, and its controls already describe the direction enforcement is heading. If you’re building AI on PHI now, the proposal tells you what your 2027 audit looks like.

Proposed, delayed, and still the direction of travel

The HIPAA Security Rule overhaul was proposed on January 6, 2025 (90 FR 898), and it is still a proposed rule as of October 2026. The Fall 2026 Unified Agenda lists final action for July 2027, a target rather than a commitment. The full text sits on the Federal Register.

A delayed rule still describes the direction of enforcement. OCR’s Risk Analysis Initiative reached its 11th and 12th settlements in early 2026 and is expanding to cover failure to complete a detailed risk management plan. The risk analysis is the floor the AI controls sit on, and OCR is enforcing it now.

The controls that matter for AI systems

The proposal makes encryption of ePHI at rest and in transit required, adds multi-factor authentication (MFA) for ePHI access, adds a written asset inventory and network map, and requires vulnerability scanning and penetration testing on fixed schedules.

For an AI feature, that means the prompts, the vector store, and the logs all sit under encryption, and the console behind the model sits behind MFA. The preamble discusses AI directly: regulated entities must be prepared to identify, mitigate, and remediate AI-related risks, while EHRA has asked OCR for non-binding AI guidance instead.

Build to those controls now, whichever vendor you run on. Encryption at rest and in transit, MFA, an asset inventory, scheduled scanning and penetration testing: they’re good practice either way, and they’re the list audits will ask about. Adopting them now costs configuration work, the same work you’d do after the final rule anyway.

HIPAA-eligible AI services on AWS, Azure, and Google Cloud

Compliance is decided in the settings column.

AWS Bedrock

Bedrock sits on the AWS HIPAA Eligible Services Reference; the standard AWS BAA covers it. It supports PrivateLink VPC endpoints and customer-managed KMS keys. Its 2026 retention modes include none, meaning zero retention. AWS HIPAA compliance for model workloads means Bedrock on the eligible list plus those settings. AWS Bedrock HIPAA coverage is that list, nothing more.

Azure OpenAI Service

Azure OpenAI is in scope under the Microsoft DPA, but Microsoft confirms no named services for Azure AI Foundry, and serverless models like Mistral lack confirmed eligibility. For PHI, use Standard regional deployments, since Global and DataZone may process outside the region; prompts and completions are never used to train models.

Google Cloud

Google’s BAA covers only its HIPAA implementation guide list, which names Gemini Enterprise Agent Platform and the Gemini Enterprise products, not plain Google Vertex AI. Its zero data retention is documented, but the Interactions API store defaults to true and abuse-monitoring logging is on unless you request an exception; Grounding with Search and Maps carries fixed retention.

Verify each provider’s current HIPAA-eligible services list before every integration; these lists move quarterly.

HIPAA-compliant AI pre-launch checklist

This is the checklist we run before a client’s first production prompt: six groups, each recapping one section of this post. It’s the front page of your audit trail. Every box is phrased as a verifiable statement, so “done” means documented, not remembered.

Data flow. Where every hop lands. If you can’t draw the PHI map, the checklist ends here.

• [ ] A PHI map of every hop the data takes
• [ ] Minimum necessary data in every prompt

Contracts. Every hop signed, against the vendor’s current covered-services list.

• [ ] A BAA for every hop, verified against the vendor’s current covered-services list

Infrastructure. The account is the boundary.

• [ ] HIPAA-eligible services only
• [ ] Customer-managed keys
• [ ] Private networking
• [ ] Role-based access control (RBAC)
• [ ] Secrets management
• [ ] Region pinned to your geography

Pipeline hygiene. What ships clean.

• [ ] Log redaction in place before launch
• [ ] Vector store on the BAA list
• [ ] Eval data synthetic or redacted

Human review. Who signs off, and the log that proves it.

• [ ] Approval queue
• [ ] Source text beside each claim
• [ ] Edit tracking
• [ ] Audit log of approvals

Operations. What keeps it that way.

• [ ] Documented risk analysis
• [ ] Incident response
• [ ] Access reviews
• [ ] The Security Rule update tracked

One assumption sits underneath the whole list: the application around the AI is already compliant. If you’re still building that part, start with our HIPAA compliant app development guide. For the operations group’s compliance program, work through the first-year HIPAA compliance walkthrough. The checklist is a living document; re-run it with every new integration.

Why choose Topflight Apps for HIPAA-compliant AI development

That’s how we run HIPAA compliant software development at Topflight Apps. We map PHI flows before design, because the flow map is where your BAA list comes from. Models get deployed inside your own cloud account under your BAA. The review queue and audit trail are designed first, before the first prompt exists. Then we deliver at a fixed price, and hand over all code when we’re done.

You keep the account, the keys, and the logs, because it’s all in your cloud. We’ve been shipping HIPAA-bounded healthcare software for over a decade, and every pattern in this post comes out of those builds. The fixed price is the point: compliance work quoted hourly has a way of becoming nine-month sandbox projects.

Want a second opinion on your AI architecture? Get the HIPAA readiness checklist, or talk to our team about your workflow, and we’ll show you the HIPAA compliant cloud pattern that fits before you commit.

Bring your current architecture and your BAA folder; we’ll point at the hops your security reviewer will ask about. If you’re mid-vendor-selection, bring the covered-services lists you’ve collected and we’ll show you how to read them. Start with the checklist; it’s the fastest way in.