The short answer
You can use AI in a HIPAA-regulated organization, but HIPAA applies to it like any other system or vendor. If an AI service creates, receives, maintains or transmits protected health information (PHI) on behalf of a covered entity, the vendor is a business associate and you need a business associate agreement (BAA) before PHI goes in. The Security Rule's safeguards apply, the minimum necessary standard applies, and people stay responsible for what the AI produces.
The most common risk is not a sophisticated attack. It is a staff member pasting a patient note or case file into a free chatbot that has no BAA and may keep the data. A short policy, approved tools and a little training prevent most of it.
This article is general information, not legal advice. Your privacy officer and counsel should confirm how HIPAA applies to your organization.
When HIPAA applies to an AI tool
- Covered entities (health plans, health care clearinghouses and providers that conduct standard electronic transactions) and their business associates must comply.
- An AI vendor becomes a business associate when it handles PHI for you, for example an assistant that summarizes clinical notes, a transcription service, or an analytics tool that receives patient-level data.
- Self-hosting does not remove the need for a BAA. If you run an open-weight model on a cloud provider's servers, that provider stores or processes the PHI, so it is a business associate and you still need a BAA with it, even if the data is encrypted (see private AI and self-hosted models).
- De-identified data that meets HIPAA's de-identification standard is not PHI, which is why de-identified or summary data is usually the safest starting point for AI analysis.
- Grant-funded programs and community organizations may not be covered entities, but contracts, funder terms and state laws often impose similar duties. Check yours.
Questions to ask before PHI goes into an AI tool
- Will the vendor sign a business associate agreement that covers this specific service and feature?
- Does the vendor use your data to train or improve its models, and can that be switched off in writing?
- Where is data processed and stored, and for how long are prompts, files and outputs kept?
- Who at the vendor can access your data, and under what conditions?
- Is access controlled through your identity system, with multifactor authentication and role-based permissions?
- Are prompts, outputs and actions logged so you can investigate an incident?
- Can you use de-identified or limited data instead of full records for this task?
- Who reviews the AI's output before it is used in care, eligibility or reporting?
Where AI helps health programs, with the right controls
| Use | Typical value | Key control |
|---|---|---|
| Summarizing long case or program notes | Saves staff reading time | BAA in place; staff check every summary |
| Drafting letters, reports and grant updates | Faster first drafts | No PHI unless the tool is covered; human edits |
| Answering staff questions from policies and procedures | Fewer interruptions and searches | Answers only from approved documents, with sources |
| Coding open-ended survey responses | Faster qualitative analysis | De-identified responses; a person checks the coding |
| Reading referral and intake forms into a system | Less keying | Confidence checks and a review queue |
AI that supports clinical decisions needs extra care. Under the Section 1557 rule, covered health programs must identify patient care decision support tools that use race, color, national origin, sex, age or disability as inputs, and reasonably mitigate the risk of discrimination. Texas also requires providers to disclose AI used in a patient's care, and other states have similar rules. Our overview of AI laws in the US summarizes them.
Rather talk it through? If you are not sure which of your AI tools touch PHI, we can map them and check the agreements and controls. Talk to a Promatics specialist
Security safeguards still apply
The HIPAA Security Rule requires administrative, physical and technical safeguards for electronic PHI, and they extend to AI systems: risk analysis that includes AI tools, access controls, audit controls, integrity protections and transmission security. AI adds a few specific risks to cover in that analysis:
- Prompt injection, where instructions hidden in a document or message make an assistant reveal information or take an action it should not.
- Over-broad access, where an assistant can search more records than the person using it is allowed to see.
- Data leakage through outputs, where summaries or drafts expose details that should have stayed in the record.
- Unapproved tools, where staff use personal AI accounts outside your agreements (shadow AI).
The NIST AI Risk Management Framework is a practical reference for organizing these risks alongside your existing HIPAA program.
When to bring in help
A small program using one approved, BAA-covered assistant for drafting can often manage with a clear policy and training. Bring in help when AI will touch patient-level data at scale, when it supports clinical or eligibility decisions, when you are building AI into your own systems, or when a funder, agency or auditor asks how AI risk is managed.
Our AI governance and security service reviews your AI use, vendor terms and controls with HIPAA in view, and our health and public sector team understands how programs, funders and agencies work. We work alongside your privacy officer and counsel, who remain responsible for legal interpretation.
Sources and further reading
Product capabilities and guidance change. These are the primary sources this article relies on, checked on the review date above.
- 45 CFR 160.103, definitions (business associate), Electronic Code of Federal Regulations
- 45 CFR 92.210, nondiscrimination in the use of patient care decision support tools, Electronic Code of Federal Regulations
- AI Risk Management Framework, National Institute of Standards and Technology
This article is general information, not legal, accounting or security advice for your specific situation. Examples are hypothetical unless stated otherwise.