Compliance 14 min read

AI Governance in Life Sciences: A Practical Guide

J

November 14, 2026

Most AI governance content on the internet is written for banks and marketing departments, then lightly reskinned for whoever asks. That approach breaks down fast in life sciences, because our AI systems don't just risk a bad customer experience or a biased loan decision. They can end up inside a device's safety architecture, inside a batch record, or inside the data package that supports a regulatory submission. The predicate rules don't go away because the tool got smarter.

I'm Jared Clark, Principal Consultant at Certify Consulting, and I spend a fair amount of my time these days helping quality and regulatory teams figure out where their existing frameworks stop covering AI and where something new has to be built. This guide is my attempt to lay out what that "something new" actually looks like, grounded in the specific standards and guidance documents you'll need to point to when an auditor or inspector asks how you're managing it.

What AI Governance Actually Means in a Life Sciences Context

AI governance, generically, is the set of policies, roles, and controls an organization uses to manage how AI systems are developed, deployed, and monitored. In life sciences, that generic definition has to absorb three things that don't exist in most other industries: predicate device regulation, Good Manufacturing Practice expectations for electronic records, and the possibility that a model's output becomes part of a submission FDA or EMA will rely on.

That means AI governance here isn't a standalone program. It's an extension of your existing quality system, and it needs to interoperate with design controls, CAPA, change control, and validation the same way any other GxP process does. A governance framework that lives in a slide deck next to your quality manual, instead of inside it, is not going to hold up.

Why This Has Become Urgent in 2026

Three regulatory developments have converged to make this a now-problem instead of a someday-problem.

First, the EU AI Act (Regulation (EU) 2024/1689) entered into force on August 1, 2024. Its high-risk obligations reach any AI system that serves as a safety component of a medical device already regulated under the EU Medical Device Regulation 2017/745 or the In Vitro Diagnostic Regulation 2017/746, per the classification rule in Article 6(1). If your device needs a notified body under MDR or IVDR and it has an AI component, that AI component inherits the high-risk classification automatically.

Second, FDA finalized its guidance "Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions" in December 2024. This is a meaningful shift: it allows manufacturers to pre-specify how a model is permitted to change after clearance, so long as the plan describing those changes was itself reviewed as part of the original submission. Before this, every meaningful model update risked triggering a new 510(k).

Third, FDA issued a January 2025 draft guidance, "Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products," under docket FDA-2024-D-4488. It proposes a risk-based credibility framework for AI models used anywhere in the drug development lifecycle, not just in devices. That's the agency signaling that AI governance expectations are no longer confined to SaMD.

None of these documents replace your existing quality system. All three assume you already have one, and they're asking you to extend it.

What Regulators Mean by "AI" — and Why the Definition Matters

One of the most common governance mistakes I see is treating "AI" as a single category that either does or doesn't apply to a company. In practice, the frameworks draw different lines, and where your tool falls determines which obligations attach to it.

Framework How "AI" Is Scoped What Triggers Obligations
FDA Good Machine Learning Practice (GMLP), Oct. 2021 AI/ML functions within Software as a Medical Device (SaMD) Use of adaptive or locked algorithms in a regulated device function
EU AI Act (Reg. (EU) 2024/1689), Art. 3(1) & Art. 6(1)/Annex I, Sec. A Machine-based systems that infer outputs influencing environments High-risk tier applies when the AI is a safety component of an MDR/IVDR-regulated device
ISO/IEC 42001:2023, clause 4.1 Any AI management system an organization chooses to certify Voluntary; scope is self-defined but must cover the organization's actual AI use cases
FDA draft guidance FDA-2024-D-4488 (Jan. 2025) AI models supporting regulatory decision-making in drug/biologic development Model is used to generate or analyze data submitted to support a regulatory decision

A large language model summarizing literature for a medical writer sits in a different governance lane than a computer vision algorithm flagging visual defects on a fill line, even though both are "AI" in casual conversation. Your governance framework has to sort tools into these lanes before it can assign controls, because the controls are not the same.

The Frameworks You Need on Your Shelf

There are more AI governance frameworks published than any single company needs to adopt in full. In my experience, life sciences organizations get the most mileage out of four, used together rather than in isolation.

Good Machine Learning Practice (GMLP). Jointly issued by FDA, Health Canada, and the UK's MHRA in October 2021, GMLP lays out ten guiding principles covering multidisciplinary expertise, representative training data, and performance monitoring across the AI/ML lifecycle. It's guidance, not a binding standard, but FDA reviewers reference it consistently in SaMD submissions.

Predetermined Change Control Plans (PCCP). The December 2024 final guidance gives you a mechanism to pre-authorize specific model changes rather than resubmitting for every update. Writing a strong PCCP requires you to define your model's performance boundaries up front, in the sponsor's own words, which is exactly the discipline most AI governance programs are missing.

EU AI Act. For any organization selling AI-enabled devices into the EU, this is not optional. Article 9 requires a risk management system running throughout the AI system's lifecycle. Article 14 requires human oversight measures that let a human "fully understand the capacities and limitations" of the system and intervene or shut it down. Article 12 requires automatic logging capable of ensuring traceability. These map cleanly onto GxP concepts you already have — risk management per ISO 14971, design controls under 21 CFR 820.30 — but the AI Act adds AI-specific documentation obligations on top.

ISO/IEC 42001:2023. This is the AI management system standard, structured like ISO 9001 or ISO 13485 so it slots into an existing integrated management system. Clause 6.1.2 requires a documented AI risk assessment process. Clause 8.1 covers operational planning and control of AI systems throughout their lifecycle. Annex A provides a control set covering data governance, third-party and supplier management for AI, and impact assessment. It's not mandatory anywhere yet, but it's becoming the reference architecture auditors expect to see echoed in your internal policy, whether or not you pursue certification.

Worth naming explicitly: NIST's AI Risk Management Framework (AI RMF 1.0, January 2023) shows up constantly in generic AI governance content, and it's a genuinely good document. But it was not written with predicate rules, Part 11, or design history files in mind. If your program is built on NIST AI RMF alone, you have a governance framework for AI in general and a gap where your GxP-specific obligations should be.

The Core Components of a Life Sciences AI Governance Framework

Strip away the framework-specific language and most of these standards converge on the same five components. Here's how I structure them when building a program from scratch.

1. Inventory and Classification

You cannot govern what you haven't found. Every AI tool in use, including the ones procurement never saw because someone's team subscribed to a chatbot with a corporate card, needs to land in an inventory with its intended use, its data sources, and its risk tier. Classification should follow the highest applicable standard: if the tool touches an MDR-regulated device, EU AI Act Article 6(1) and Annex I, Section A govern; if it touches drug development data destined for a regulatory submission, the FDA credibility framework in FDA-2024-D-4488 governs.

2. Risk Assessment Tied to Intended Use

Risk assessment for AI can't be generic. A model's risk depends on what happens if it's wrong, not on how sophisticated the model is. A locked algorithm sorting incoming supplier certificates by document type carries a different risk profile than an adaptive algorithm adjusting sterilization cycle parameters in real time. ISO 42001 clause 6.1.2 requires this assessment be documented and tied to specific, identified consequences, which lines up well with the FMEA-style risk tools most quality teams already run under ISO 14971.

3. Human Oversight and Intervention

EU AI Act Article 14 is explicit that human oversight can't be theoretical. Someone identified by role needs the authority, the training, and the interface to interrupt the system. In practice, this means your governance policy has to name who can override an AI-generated deviation classification, who reviews AI-drafted CAPA root cause statements before they're finalized, and what the escalation path looks like when the model's confidence score drops below a defined threshold.

4. Data Governance and Provenance

Every AI framework in this space cares where the training and input data came from, because a model's outputs are only as defensible as the data behind them. For regulated records, this connects directly to 21 CFR Part 11: if an AI system generates or modifies a GxP record, that record needs the same audit trail, attribution, and data integrity controls you'd require of a human-entered one. A model that can't tell you which source documents informed a given output is not going to survive an inspection question about it.

5. Change Control and Post-Market Monitoring

Static validation is not enough for a system that can drift. Whether you're running a PCCP under FDA's framework or a continuous monitoring plan under EU AI Act Article 72, you need defined performance thresholds, a monitoring cadence, and a documented trigger for when drift requires re-validation rather than routine tuning. This is where a lot of programs fail quietly: the model was validated once, at launch, and nobody revisited it against production data eighteen months later.

Where This Gets Genuinely Contested: GxP AI vs. Discovery AI

Not every AI use case in a life sciences company carries the same governance weight, and treating them identically wastes effort in one direction and creates real risk in the other. I draw the line at whether the output feeds a GxP process or a regulatory submission.

Use Case Category Example Governing Obligation Typical Oversight Level
GxP-adjacent operations AI-assisted deviation triage, batch record review 21 CFR Part 11, internal SOP, validation per GAMP 5 High — documented human review required
Regulated device function AI-enabled diagnostic algorithm EU MDR/IVDR + AI Act Art. 6(1)/Annex I, FDA GMLP Highest — full design control lifecycle
Regulatory submission support AI model analyzing trial data for a filing FDA-2024-D-4488 credibility framework High — risk-based credibility assessment
Early discovery / literature synthesis LLM summarizing published research Internal governance policy, no predicate rule Moderate — quality review, not regulatory filing

The mistake I see most often runs in both directions. Some companies apply full design-control rigor to an internal literature-summarization tool that will never touch a submission, burning validation resources on something low-risk. Others let a chatbot quietly draft language that ends up, mostly unedited, in a CAPA effectiveness check, because nobody classified it as touching a GxP record in the first place. The governance framework's first job is sorting these cases correctly, before it does anything else.

Common Pitfalls I See in the Field

A few patterns show up repeatedly enough that they're worth naming directly.

Companies write an AI policy that reads well but was never mapped to a specific SOP, which means it has no enforcement mechanism when an auditor asks to see evidence of it in action. Teams treat vendor AI tools as outside their governance scope because "the vendor validated it," without confirming the vendor's validation covered their specific intended use. And quality teams underestimate how much AI-generated content is already flowing into their systems through general-purpose tools that were never intended for regulated recordkeeping in the first place — a risk I've written about in more detail in why general-purpose AI tools aren't safe for quality records.

A Starting Checklist for Building the Program

If you're building this from nothing, sequence matters:

  1. Build the inventory. You can't classify or assess risk on tools you haven't found.
  2. Classify each tool against the table above, tying it to the highest applicable framework.
  3. Write the risk assessment before you write the policy, not after, so the policy reflects actual risk rather than generic best practice.
  4. Define human oversight roles by name and by authority, not just by department.
  5. Build change control and a monitoring plan around measurable performance thresholds.
  6. Confirm your systems can hold the evidence. Make sure whatever platform holds your validation and audit trail records can actually document AI-specific decisions, not just legacy paper-replacement workflows. If your current QMS software was built before predetermined change control plans existed as a regulatory concept, it's worth checking whether it was designed to hold this kind of evidence at all — a question I explore in how to validate QMS software under 21 CFR Part 11 and Annex 11.

Where This Is Heading

The EU AI Act's high-risk obligations phase in through August 2026 for most Annex III use cases, but product-integrated systems — like the MDR/IVDR-embedded AI this guide spends the most time on — fall under Annex I and get an extra year, with compliance required by August 2027 under Article 113. FDA, meanwhile, has signaled it will keep issuing use-case-specific guidance rather than one omnibus AI regulation. That means the shape of AI governance in this industry is going to keep arriving in pieces: a device guidance here, a drug development framework there, an EU regulation with its own timeline running in parallel. The organizations that will handle this well aren't the ones waiting for a single unified standard to arrive. They're the ones who already built a governance structure flexible enough to absorb the next guidance document without a redesign.

Frequently Asked Questions

What is AI governance in the context of life sciences?

AI governance in life sciences is the set of policies, risk assessments, and controls an organization uses to manage AI systems that touch GxP processes, regulated device functions, or regulatory submissions. It extends existing quality systems rather than replacing them, connecting to design controls, change control, and 21 CFR Part 11 recordkeeping requirements.

Does the EU AI Act apply to pharmaceutical and medical device companies?

Yes, when the AI system is a safety component of a device already regulated under EU MDR 2017/745 or IVDR 2017/746. Under Article 6(1) of Regulation (EU) 2024/1689, such systems are automatically classified as high-risk, triggering obligations including a lifecycle risk management system under Article 9 and human oversight measures under Article 14.

What is FDA's Predetermined Change Control Plan (PCCP)?

A PCCP is a plan, submitted as part of a device's original marketing submission, that pre-specifies how an AI-enabled device's algorithm is permitted to change after clearance without requiring a new submission for each change. FDA finalized its guidance on PCCPs for AI-enabled device software functions in December 2024.

Is ISO 42001 certification required for life sciences AI systems?

No. ISO/IEC 42001:2023 is a voluntary AI management system standard. It is not a legal requirement in the way 21 CFR Part 11 or EU MDR are, but its structure, particularly the risk assessment process in clause 6.1.2, is increasingly used as the reference model auditors expect to see reflected in internal AI policy.

How does 21 CFR Part 11 apply to AI-generated GxP records?

If an AI system creates, modifies, or approves a record used in a GxP process, that record is subject to the same electronic record and electronic signature requirements as any human-generated record under 21 CFR Part 11, including audit trail, attribution, and data integrity controls. The fact that a model generated the content does not exempt it from these requirements.

Last updated: 2026-09-27

J

Jared Clark

Principal Consultant, Certify Consulting

Jared Clark is the founder of Certify Consulting, helping organizations achieve and maintain compliance with international standards and regulatory requirements.