HIPAA-Compliant Medical AI: 10 Checks

A medical AI product is not safe for protected health information simply because its website mentions HIPAA, encryption, or healthcare security. Compliance depends on the organization using it, the exact data entered, what the vendor does with that data, the contract in place, and the safeguards around the complete workflow.

For clinicians evaluating AI clinical decision support, the practical question is not only, “Is this tool HIPAA compliant?” The better question is, “Can our approved use of this tool protect electronic protected health information from input through deletion?”

Quick Answer

Before using medical AI with patient information, verify whether the vendor will create, receive, maintain, or transmit protected health information; whether a business associate agreement is required and available; which data the system stores; whether prompts or outputs are used for model training; which subcontractors receive data; how access is controlled; how information is deleted; and how incidents are reported. A business associate agreement is important, but it is not a substitute for the healthcare organization’s own risk analysis, policies, training, and workflow controls.

What “HIPAA-Compliant Medical AI” Actually Means

HIPAA applies to covered entities and business associates, not automatically to every company that handles health-related information. The U.S. Department of Health and Human Services explains that a business associate is generally an outside person or organization performing certain services for a covered entity that involve protected health information, or a subcontractor handling that information on behalf of another business associate.

This means compliance is a relationship and a workflow, not a permanent property of an app. The same system may be used in one workflow without patient-identifiable data and in another workflow that creates, receives, maintains, or transmits electronic protected health information. Those uses do not carry the same legal or security requirements.

HHS does not endorse, certify, or recommend particular technology products. A logo, security page, or “HIPAA-ready” label should therefore be treated as the beginning of review—not the conclusion. This is especially important when evaluating a broad AI medical assistant that may accept typed prompts, uploaded documents, or clinical context.

The Most Important Insight: Trace the Data Path

Many evaluations stop at the prompt box. That is too narrow. A single clinical question can produce multiple copies or derivatives of information across the service. The review should trace the full data path:

Prompt text, dictated questions, uploaded files, images, and pasted chart excerpts
Retrieved context, generated answers, citations, summaries, and exported documents
Conversation history, user activity logs, device information, and diagnostic telemetry
Support tickets, quality-review queues, abuse monitoring, and human-access workflows
Backups, disaster-recovery copies, analytics systems, and vendor subprocessors
Deletion requests, account closure, contract termination, and residual backup retention

In practical terms, entering a patient-specific prompt may be a disclosure of information to an external service. The privacy review must cover every location that information travels, not only the visible answer screen.

10 Questions Clinicians and Clinics Should Ask

1. Does This Workflow Involve PHI?

Start with the actual use case. Will a clinician enter a name, date of birth, medical record number, exact dates, contact information, images, documents, or enough clinical detail to identify a person? Will the tool receive information directly from an electronic health record or another clinical system? Do not assume that removing the patient’s name makes the remaining information de-identified.

2. Is the Vendor Acting as a Business Associate?

If an outside service creates, receives, maintains, or transmits PHI on behalf of a covered entity, it may be a business associate. HHS guidance notes that a cloud provider maintaining encrypted electronic PHI can still be a business associate even when it does not hold the decryption key. Ask legal or privacy leadership to evaluate the role based on the service and data flow—not the vendor’s marketing category.

3. Is a Business Associate Agreement Required and Available?

When a business associate relationship exists, the parties generally need a written agreement that defines permitted and required uses and disclosures, safeguards, incident reporting, subcontractor obligations, and return or destruction of PHI at termination. Review the actual agreement for the purchased plan. A generic privacy policy is not a substitute for a business associate agreement.

4. Can the Data Be Used for Model Training or Product Improvement?

Ask whether prompts, uploaded content, generated outputs, user feedback, logs, or support interactions can be used to train models, evaluate outputs, improve services, or build datasets. Confirm the answer in the contract and product settings. “We do not train on your data” is incomplete if humans, subprocessors, analytics services, or quality systems can still access or retain it for other purposes.

5. Are Users Limited to the Minimum Necessary Information?

Where the HIPAA minimum necessary standard applies, covered entities generally must make reasonable efforts to limit uses, disclosures, and requests for PHI to what is needed for the intended purpose. In an AI workflow, that principle can be translated into prompt templates, prohibited-data rules, role-based access, and default use of de-identified or generalized clinical context. Review the HHS minimum necessary guidance with privacy counsel because exceptions and context matter.

6. Is the Information Truly De-Identified?

Removing a name is not automatically sufficient. HHS recognizes two HIPAA de-identification methods: Expert Determination and Safe Harbor. Even properly de-identified information retains a small residual risk of identification. Before a clinic labels an AI workflow “de-identified,” it should use an approved method and account for combinations of rare diagnoses, exact dates, geography, occupation, or narrative details that may identify a person. See the HHS de-identification guidance.

7. How Long Are Inputs, Outputs, Logs, and Backups Retained?

Request specific retention periods for every data category. Determine whether administrators can shorten retention, disable history, delete individual conversations, export records, and confirm deletion. Ask what remains in backups after deletion and what happens to data when the account or contract ends. “Delete” may mean removal from the user interface rather than immediate removal from every system.

8. Which Security Controls Protect the Workflow?

Review authentication, unique user accounts, multifactor authentication, role-based permissions, encryption, audit logging, session controls, device access, workforce access, vulnerability management, backup, recovery, and availability. Encryption matters, but HHS explicitly notes that encryption alone does not address every confidentiality, integrity, and availability requirement.

9. Which Subprocessors and Locations Are Involved?

Ask for a current subprocessor list and identify which organizations can create, receive, maintain, or transmit PHI. Review where data is processed and stored, how subprocessors are added, what notice customers receive, and whether equivalent contractual restrictions flow downstream. The primary vendor’s controls do not answer every question about the rest of the service chain.

10. What Happens After a Security Incident?

The contract and incident-response plan should define how suspected incidents are reported, investigated, contained, documented, and communicated. The HIPAA Breach Notification Rule includes notification obligations following breaches of unsecured PHI. Clinics should know whom to contact, what evidence the vendor will provide, how quickly notice will occur, and how affected workflows will continue safely.

A BAA Is Necessary in Some Workflows, but It Is Not Enough

A signed business associate agreement does not automatically make every use of a product compliant. The covered entity still needs to understand the service, conduct an appropriate risk analysis, configure access, train users, define permitted use cases, monitor the workflow, and respond to changes. The vendor and customer may each control different safeguards.

HHS describes risk analysis as a foundational Security Rule requirement for regulated entities. The analysis should address all electronic PHI an organization creates, receives, maintains, or transmits and should be updated as systems and risks change. An AI pilot should therefore be part of the organization’s broader clinical decision support implementation process, not an informal individual experiment.

Red Flags During Medical AI Review

The vendor will not clearly state whether it creates, receives, maintains, or transmits PHI.
The BAA is unavailable for the plan being purchased or does not match the intended workflow.
The privacy policy allows broad secondary use of prompts, outputs, or uploaded content.
Retention periods are vague, deletion cannot be confirmed, or backup retention is unexplained.
Users share one account, access roles are unavailable, or meaningful audit logs cannot be produced.
Subprocessors, human-review processes, or data locations are undisclosed.
The product makes a broad compliance claim but will not provide documentation for legal, privacy, and security review.
The organization has no written policy for permitted use, prohibited information, verification, escalation, or incident reporting.

A Safer Rollout Workflow for Clinics

Define one narrow use case. Start with a specific evidence-retrieval or education workflow and define what the tool must not do.
Map the data. Document inputs, outputs, storage, logs, integrations, subprocessors, support access, and deletion.
Complete legal, privacy, and security review. Determine HIPAA roles, required agreements, other applicable laws, risk controls, and approval conditions.
Configure and train. Use individual accounts, least-privilege access, approved prompt patterns, and clear incident-reporting instructions.
Pilot with de-identified questions where possible. Test source quality, clinical usefulness, privacy behavior, and failure handling before expanding scope.
Review continuously. Reassess after feature changes, integrations, new subprocessors, policy updates, incidents, or material workflow changes.

How This Applies to Evidence-Based Medical AI

Many clinical evidence questions can be asked without patient identifiers. A clinician can often request current guideline sources, compare treatment evidence, review diagnostic criteria, or investigate a drug class using generalized clinical context. ZoeMD’s guide to evidence-based medical AI at the point of care explains why evidence visibility and clinician verification matter in that workflow.

Privacy and clinical reliability are separate review tracks. A system can protect data yet return weak evidence, or provide strong citations while being used in an unapproved privacy workflow. Clinicians should evaluate both. The source-checking process in AI Medical Assistant With Citations addresses the clinical evidence side; this checklist addresses the data-governance side.

The same separation applies to high-risk uses. An AI-supported diagnostic review or drug-interaction question needs both appropriate data handling and independent clinical verification before the output influences care.

Frequently Asked Questions

Does HIPAA prohibit clinicians from using medical AI?

No. HIPAA does not prohibit a specific technology category. It establishes requirements for covered entities and business associates when they use or disclose PHI and protect electronic PHI. Whether a particular workflow is permissible depends on the parties, data, purpose, agreements, safeguards, and applicable rules.

Is a BAA enough to make an AI workflow HIPAA compliant?

No. A BAA may be required, but the organization must still perform risk analysis and risk management, configure safeguards, limit access, train users, define permitted uses, and monitor the workflow. The agreement and the real operating practice must align.

Can clinicians enter PHI into a medical AI tool?

Only when the tool, plan, contract, configuration, and organizational workflow have been approved for that use and all applicable requirements are satisfied. If that approval is absent, do not enter patient-identifiable information. Use appropriately de-identified or generalized questions when the clinical task permits.

Does removing a patient’s name make a prompt de-identified?

Not necessarily. Other identifiers and combinations of details may identify a person. HIPAA de-identification requires use of the Safe Harbor method or Expert Determination standard. Organizations should not invent an informal definition for AI prompts.

Does HIPAA cover every health app and every piece of health data?

No. HIPAA applies to covered entities, business associates, and PHI within that regulatory framework. Other federal or state privacy, consumer-protection, breach-notification, professional, contractual, or sector-specific requirements may still apply when HIPAA does not.

Bottom Line

HIPAA-compliant medical AI is not a feature switch or a vendor badge. It is an approved relationship and a controlled data workflow. Before patient information enters any system, identify the HIPAA roles, trace the full data path, execute required agreements, limit information to what is necessary, verify retention and secondary use, review security and subprocessors, and prepare for incidents. If those answers are incomplete, keep patient-identifiable information out of the tool.

ZoeMD is designed to help clinicians move from medical questions to cited evidence. Explore ZoeMD for source-linked clinical research, and follow your organization’s privacy and security approval process before using any medical AI workflow with patient data.

Legal and Medical Disclaimer

This article provides general educational information and is not legal, privacy, cybersecurity, compliance, or medical advice. HIPAA applicability depends on specific facts, contracts, parties, data, and workflows. Consult qualified legal, privacy, security, and compliance professionals and follow organizational policy. Medical AI does not replace patient evaluation, professional clinical judgment, approved references, local protocols, pharmacist review, specialist consultation, or emergency care.

HIPAA-Compliant Medical AI: What Clinicians Should Verify Before Using Any Tool