HIPAA (USA)  ·  GDPR (EU)  ·  PHIPA (Ontario)  ·  PIPEDA (Canada)  ·  EU AI Act (EU)  ·  NIST AI RMF  ·  FDA AI/ML  ·  SOC 2  ·  ISO/IEC 42001  ·  ISO/IEC 27001  ·  OECD AI Principles

US: FDA AI/ML

Guidance for Software as a Medical Device (SaMD) in Healthcare

History and Overview

The U.S. Food and Drug Administration (FDA) regulates medical devices, including Software as a Medical Device (SaMD)
which incorporates artificial intelligence (AI) and machine learning (ML). AI/ML-based medical software is increasingly used in
diagnostics, imaging, predictive analytics, clinical decision support, and triage. Depending on its intended use and risk profile, AI/ML software may require FDA clearance or approval through pathways such as 510(k), De Novo, or Premarket Approval (PMA).

Guidance for AI/ML in medical devices is outlined in the FDA’s AI/ML SaMD framework, which emphasizes patient safety, clinical validity, transparency, and ongoing performance monitoring. This guidance complements other U.S. healthcare regulations, including HIPAA for privacy.

Scope of Application

FDA oversight applies to AI/ML systems that:

  • – Diagnose, treat, or mitigate medical conditions
  • – Provide clinical recommendations to healthcare professionals
  • – Perform medical image analysis or automated triage
  • – Monitor patient health in real-time or predict adverse events

Relevance to AI in Healthcare

Regulation ensures patient safety, clinical effectiveness, and trustworthy AI deployment. Specifically, the FDA evaluates:

  • – Clinical validity and patient benefit
  • – Algorithm performance, reproducibility, and consistency
  • – Cybersecurity, data integrity, and compliance with HIPAA
  • – Quality management aligned with ISO 13485

Key Obligations and Requirements

  • Premarket Submission: 510(k), De Novo, or PMA submissions depending on device risk and novelty

  • Good Machine Learning Practices (GMLP): Transparent development, model validation, and testing procedures

  • Real-World Performance Monitoring: Continuous post-market surveillance and data collection

  • Change Management Protocols: Documented processes for adaptive AI updates and retraining

Governance Requirements

  • – Model design specifications including architecture, inputs, and intended use
  • – Clinical evaluation reports demonstrating safety and efficacy
  • – Real-world performance logs, including outcomes, errors, and deviations
  • – Change logs and versioning for all model updates, especially adaptive AI systems

Risks and Challenges

  • – Lengthy and complex regulatory approval processes for high-risk AI systems
  • – Uncertainty for continuously learning or adaptive AI models under existing frameworks
  • – Need for interdisciplinary collaboration between clinicians, data scientists, and regulatory specialists
  • – Maintaining patient privacy and cybersecurity in real-world deployments

Best Practices for Compliance

  • – Engage early with regulators via the Pre-Submission (Pre-Sub) process to clarify requirements
  • – Use explainability and interpretability tools for clinical decision-support models
  • – Conduct pilot deployments under real-world conditions before full-scale rollout
  • – Document all design, validation, and post-market activities to demonstrate compliance with GMLP
  • – Integrate cybersecurity measures and HIPAA-compliant privacy controls

Future Developments

  • – Ongoing development of a regulatory framework for adaptive and continuously learning AI
  • – Draft guidance on predefined change control plans to manage AI/ML updates
  • – Alignment with other regulatory standards such as ISO 13485, ISO/IEC 42001, and FDA SaMD guidance updates
  • – Increasing focus on transparency, bias mitigation, and human oversight in AI systems used in healthcare

Alignment with Other Standards

  • ISO 13485 – Quality management systems for medical devices

  • HIPAA – Privacy and security of patient health information

  • NIST AI RMF – Risk management for trustworthy AI lifecycle

References