Early Recalls and Clinical Validation Issues in AI-Enabled Medical Devices

Published Date : 22/08/2025 

This study examines the recalls of FDA-cleared AI-enabled medical devices (AIMDs) and their association with clinical validation and manufacturer status. Key findings include a higher rate of recalls for devices without clinical validation and those produced by publicly traded companies. 

Artificial intelligence–enabled medical devices (AIMDs) are increasingly prevalent in US clinical practice, with nearly all cleared through the US Food and Drug Administration (FDA) 510(k) pathway. Because the 510(k) clearance process does not require prospective human testing, many AIMDs enter the market with limited or no clinical evaluation. This can undermine clinician and patient confidence in their performance, especially when recalls occur. We conducted a study to examine whether a lack of reported clinical validation and the status of the manufacturer (publicly traded vs. privately held) were associated with AIMD recalls.

In this cross-sectional study, all FDA-cleared AIMDs were matched to recall entries in the Center for Devices and Radiological Health database from November 15 to November 30, 2024. Each recall number–device pair was counted as one event; devices could contribute multiple events. Validation status was classified as none, retrospective dataset, or prospective trial based on FDA summaries. AIMD commercialization models were determined using S&P Capital IQ NetAdvantage and PrivCo databases. Manufacturer-attributed recall causes were categorized as diagnostic or measurement errors, functionality delay or loss, physical hazards, biochemical hazards, and postmarket changes. We followed the STROBE reporting guideline.

Kaplan-Meier analyses were used to estimate recall-free survival, and group differences were assessed using log-rank tests. Multivariable logistic regression (GraphPad Prism 10; 2-sided α = .05) modeled the odds of any recall, adjusting for validation status, manufacturer type, clearance year (2020-2024 vs. earlier), and clinical specialty. The study used publicly available data and did not require institutional review board oversight or informed consent per the Common Rule (US 45 CFR 46).

Among 950 AIMDs, 60 (6.3%) were associated with 182 recall events (mean [SD] recalls per device, 3.0 [4.3]). Diagnostic or measurement errors accounted for 109 recalls encompassing 935,063 units, followed by functionality delay (44 recalls, 755,647 units), physical hazards (14 recalls, 8,192 units), and biochemical hazards (13 recalls, 76,257 units). Notably, 79 (43.4%) recalls occurred within the first 12 months of device clearance, approximately double the rate reported for all 510(k) devices. Additionally, 108 recalls (59.3%) remained unresolved at the study end date, with 20 recalls remaining unresolved for more than 3 years. Overall, recall-free survival was 96.6% at 1 year, 93.5% at 3 years, and 91.8% at 5 years.

Devices without reported validation had more recalls per device (mean [SD], 3.4 [5.1]) than those with retrospective (1.9 [1.5]) or prospective (2.0 [1.4]) validation, and were associated with larger recalls (mean [SD] units recalled, 12,193 [82,405] vs 6,523 [18,690]; P < .001). Public companies accounted for 53.2% of AIMDs but 91.8% of recalls and 98.7% of recalled units. Among recalled devices from privately traded companies, 40% lacked clinical validation compared with 77.7% of those from established public companies and 96.9% from smaller public companies. Multivariable analysis revealed that lack of clinical validation (odds ratio [OR], 2.8; 95% CI, 1.6-4.7) and public company status (OR, 5.9; 95% CI, 2.4-14.6) were independently associated with recall.

In this cross-sectional study, recalls of FDA-cleared AIMDs were uncommon but were concentrated early after clearance and predominantly involved products lacking clinical validation and manufactured by publicly traded companies. This suggests that the 510(k) process may overlook early performance failures of AI technologies. Requiring prospective evaluation or issuing time-limited clearances that lapse without confirmatory data may reduce these risks. Given that over 90% of recalled units were produced by public companies, heightened premarket clinical testing requirements and postmarket surveillance measures may improve identification and reduction of device errors, similar to risk-based strategies in pharmacovigilance. The association between public company status and higher recalls may reflect investor-driven pressure for faster launches, warranting further study.

Study limitations include reliance on publicly available validation reports, exclusion of software updates not labeled as recalls, and limited follow-up for devices cleared after 2022. Nevertheless, linking premarket evidence gaps and manufacturer type to postmarket actions offers practical guidance for regulators, clinicians, and health systems adopting AI-based tools. 

Frequently Asked Questions (FAQS): 

Q: What are AI-enabled medical devices (AIMDs)?

A: AI-enabled medical devices (AIMDs) are medical devices that incorporate artificial intelligence to assist in diagnosis, treatment, and patient care. These devices can range from software applications to physical devices with AI capabilities.

Q: Why are recalls of AIMDs important?

A: Recalls of AIMDs are important because they can indicate issues with the device's performance, which can affect patient safety and clinical outcomes. Recalls help ensure that only safe and effective devices are used in medical practice.

Q: What is the 510(k) clearance pathway?

A: The 510(k) clearance pathway is a regulatory process used by the FDA to evaluate medical devices. It allows devices to be marketed if they are found to be substantially equivalent to a legally marketed device, without requiring extensive premarket clinical testing.

Q: What were the key findings of the study?

A: The key findings of the study include a higher rate of recalls for AIMDs without clinical validation and those produced by publicly traded companies. The study also found that many recalls occurred within the first year of device clearance.

Q: How can the findings of this study impact future regulations?

A: The findings suggest that requiring more rigorous premarket clinical testing and postmarket surveillance for AIMDs, especially those produced by publicly traded companies, could help reduce the risk of recalls and improve patient safety. 

More Related Topics :