Artificial Intelligence in Medical Devices Questionnaire published by TEAM-NB
The Artificial Intelligence in Medical Devices Questionnaire, developed by Team-NB and IG-NB, is an essential guideline designed to align AI-based medical devices with the requirements of the MDR and IVDR. This document provides process-oriented recommendations for manufacturers, focusing on safety, performance, and regulatory compliance throughout the lifecycle of AI-based medical devices.
The Artificial Intelligence in Medical Devices Questionnaire, developed by Team-NB and IG-NB, emphasizes process-based requirements over specific product characteristics. It aims to:
- Cover all phases of the AI lifecycle, from development and validation to post-market surveillance.
- Highlight the importance of tailoring conformity assessments to the intended use of the device.
The MDR emphasizes conformity through General Safety and Performance Requirements (GSPRs) outlined in Annex I. The Questionnaire aligns by focusing on lifecycle management, ensuring devices maintain compliance across iterative processes like updates or retraining of AI models. Notably, the MDR (Annex II) demands extensive technical documentation, which the Questionnaire mirrors in its emphasis on procedural clarity and documentation rigor.
Collection, processing, storage and training of SaMD datasets
The Questionnaire outlines robust requirements for the collection, processing, and storage of training, validation, and test datasets:
- Procedures for anonymization and pseudonymization of sensitive data.
- Justification for dataset representativeness, addressing potential biases and imbalances.
- Documentation of technical inclusion/exclusion criteria.
MDR Annex I (Chapters 1 and 3) explicitly requires managing patient data responsibly, especially regarding privacy and accuracy in device performance. The Questionnaire aligns closely, integrating GDPR (Regulation (EU) 2016/679) principles for personal data protection while focusing on dataset adequacy to ensure clinical safety and effectiveness.
Validation of SaMD AI systems
Validation of SaMD AI systems should be performed to:
- Validation of models before market entry, ensuring compliance with intended use.
- Addressing risks associated with adaptive, self-learning models by limiting “certifiability” unless proper safeguards ensure safety.
MDR Annex XIV emphasizes clinical evaluations to verify the device achieves its intended purpose and benefits outweigh risks. The Questionnaire supplements these requirements by addressing AI-specific challenges like model drift and the need for iterative revalidation.
Risk Assessment in AI medical devices
The Questionnaire mandates comprehensive risk assessments that include:
- Risks from imbalanced datasets, demographic disparities, or biased algorithms.
- Cybersecurity threats, including adversarial attacks specific to AI algorithms.
Risk management in the MDR (Annex I, Section 3) requires manufacturers to reduce risks as far as possible. The Questionnaire expands on this by focusing on unique risks posed by AI, requiring specific mitigation plans that align with ISO 14971:2019 for risk management.
Clinical Evaluation in AI medical devices
The Questionnaire incorporates justification for the use of AI over conventional methods, with documented clinical benefits and evaluations of performance across different demographics and patient subgroups.
MDR Annex XIV requires robust clinical investigations, particularly for novel technologies. The Questionnaire strengthens this by advocating for evidence of AI model transferability across populations, ensuring equitable clinical performance.
Change Management and Iterative Updates
AI systems often evolve post-deployment. The Questionnaire highlights:
- Requirements for managing significant changes, such as retraining algorithms or updating datasets.
- Ensuring such changes do not compromise previously validated performance.
Under MDR Article 120(3), any significant change requires a new conformity assessment. The Questionnaire’s guidance on iterative updates ensures compliance while maintaining device safety and effectiveness.
Safety and Cybersecurity in SaMD with AI
The Questionnaire provides actionable steps to mitigate AI-specific risks:
Identifying threats like poisoning or evasion attacks and incorporating measures to harden AI algorithms and infrastructures against these vulnerabilities.
While MDR does not directly address cybersecurity in AI, it mandates general safety and performance requirements. The Questionnaire integrates MDR Annex I’s focus on safety by addressing emerging cybersecurity risks specific to AI.
Post-Market Surveillance
Manufacturers are required to develop comprehensive Post-Market Surveillance (PMS) Plans and monitor real-world data to ensure ongoing compliance with expected performance and safety.
MDR Article 83 outlines PMS requirements, including monitoring adverse events and ensuring continued safety and performance. The Questionnaire echoes these requirements, emphasizing iterative learning from real-world data to refine AI models post-market.
The Artificial Intelligence in Medical Devices Questionnaire serves as a practical extension of MDR requirements, providing tailored guidance for the complexities of AI in healthcare. Its process-oriented framework ensures manufacturers address regulatory challenges proactively, particularly in areas like data integrity, risk management, and cybersecurity. By aligning closely with MDR principles, the document equips manufacturers to develop safer, more effective AI-based medical devices that meet EU regulatory standards.
Link to the document: