Abstract
With the explosion of innovation driven by generative and traditional artificial intelligence (AI), comes the necessity to understand and regulate products that often defy current regulatory classification. Tradition, and lack of regulatory expediency, imposes the notion of force-fitting novel innovations into pre-existing product classifications or into the essentially unregulated domains of wellness or consumer electronics. Further, regulatory requirements, levels of risk tolerance, and capabilities vary greatly across the spectrum of technology innovators. For example, currently unregulated information and consumer electronic suppliers set their own editorial and communication standards without extensive federal regulation. However, industries like biopharma companies are held to a higher standard in the same space, given current direct-to-consumer regulations like the Sunshine Act (also known as Open Payments), the federal Anti-Kickback Statute, the federal False Claims Act, and others. Clear and well-defined regulations not only reduce ambiguity but facilitate scale, showcasing the importance of regulatory clarity in fostering innovation and growth. To avoid highly regulated industries like health care and biopharma from being discouraged from developing AI to improve patient care, there is a need for a specialized framework to establish regulatory evidence for AI-based medical solutions. In this paper, we review the current regulatory environment considering current innovations but also pre-existing legal and regulatory responsibilities of the biopharma industry and propose a novel, hybridized approach for the assessment of novel AI-based patient solutions. Further, we will elaborate the proposed concepts via case studies. This paper explores the challenges posed by the current regulatory environment, emphasizing the need for a specialized framework for AI medical devices. By reviewing existing regulations and proposing a hybridized approach, we aim to ensure that the potential of AI in biopharmaceutical innovation is not hindered by uneven regulatory landscapes.
JMIR AI 2025;4:e57421doi:10.2196/57421
Keywords
Introduction
Background
The convergence of algorithms, artificial intelligence (AI), big data, and digital health technologies (DHTs) is a sea change not seen since the “dot.com” era, which has significantly changed the way we work, play, and learn [,]. However, the lack of comprehensive regulatory guidance has led to the force-fitting of novel innovations into existing categories, leading to ambiguous boundaries between medical devices and consumer electronics. This results in added ambiguity for innovators seeking to share valuable product concepts. What is lacking is a comprehensive approach to evaluating medical benefits, risks, and evidence that can be universally applied across different product categories, product types, and regulatory regimes []. Such an approach would be flexible, allowing for the distinctions between various products to be properly addressed. Clear regulations play an important role in enabling easier scaling, highlighting the mutually beneficial relationship between regulatory clarity and the acceleration of innovation. Moreover, the pharmaceutical industry’s comprehensive understanding of scaling and marketing extends beyond the confines of drug development, presenting a valuable paradigm for other sectors in the AI landscape. This paper addresses the ambiguity faced by innovators and proposes models for evidence strategies, particularly focusing on the distinct regulatory challenges faced by the biopharma industry.
At the time of this writing, Gartner [] has placed the increasingly popular generative AI technology at the peak of inflated expectations for emerging technologies in 2024. Whether the transformation that generative or other forms of AI and DHTs bring to health care occurs gradually or rapidly, there is widespread anticipation of both advancement and potential challenges [,]. The many use cases ranging across diagnostics, logistics, clerical improvements, and new treatment modalities in general medicine and across medical subspecialties have been described in detail within the literature [,]. Similarly, the use of these technologies holds equally compelling promise within biomedical product development [,].
Current Challenges
AI and DHTs represent a wide range of intricate and interconnected technologies, and the vast array of applications are equally diverse and complex. For example, most DHTs rely on proprietary algorithms trained from and across a mixture of private and public data sources. As multiple technologies are integrated into a tool, more information may be shared, and the capabilities and risks aggregate []. This additive complexity challenges traditional domain-based regulatory regimes. The US Food and Drug Administration (FDA) regulates medical devices, but not all algorithms and apps meet the regulatory definition of a medical device as defined in Section 201(h) of the Food, Drug, and Cosmetic Act. These apps often access and transmit data across the internet but do not fit neatly into the codified remit of the Federal Communications Commission or the Federal Trade Commission (FTC) []. Even within regulatory regimes, there are qualified gaps. The Office for Civil Rights enforces health care privacy, but only for covered entities, leaving a loophole and resulting in gaps in protection that are systematically being exploited []. Further, these overlapping, complex, and intricate interregulatory and intraregulatory regimes create confusion and inequities, hindering progress.
Objectives
While there have been efforts to establish standardized approaches to the regulatory assessment of DHT and AI medical products, many of these frameworks take the approach of a single regulator and regulatory regime versus approaches that inform regulatory decision-making across the spectrum of relevant regulators [-]. Further, these frameworks incorrectly assume that all innovators are alike. Consumer electronic companies and health technology startups, providing solutions that may overlap or compete with offerings from traditional medical device and pharmaceutical companies, often navigate a regulatory landscape that offers them more adaptability in their operations, which differs from the established health care regulations governing other sectors. The coexistence of unregulated and highly regulated makers in the same market can lead to various challenges, including issues related to safety, quality, and fair competition. Balancing innovation and oversight is crucial in this context. We need solutions that promote fair competition while maintaining a high standard of safety and product effectiveness, without creating a disparity between the heavily and lightly regulated entities. It is also helpful to level set on terminology. 
Textbox 1 provides definitions for common terms used in this space.
Digital health technologies: technologies consisting of hardware (eg, sensors or transmitters) or software (eg, connectivity software, algorithms, or artificial intelligence) components that are used for health care–related purposes.
Medical device software: term primarily used in the European Union to define software with a medical purpose that can be used either alone or in combination with a regulated medical device. This is not interchangeable with software as a medical device [].
Software as a medical device: software that is used for medical purposes and may do so independently of a hardware medical device as well as not being a required component of a hardware medical device [].
Mobile medical apps: mobile apps serving as medical devices, which integrate software functionality that aligns with the Food and Drug Administration definition of a device, as outlined in Section 201(h) of the Food, Drug, and Cosmetic Act. These apps may function as accessories to regulated medical devices or convert a mobile platform into a regulated medical device [].
Digital therapeutics: software-based interventions intended to prevent, manage, or treat medical conditions based on evidence of a demonstrable positive therapeutic impact on a patient’s health [].
Direct to consumer: marketing products or services directly to consumers without the involvement of a health care provider.
Regulatory Regimes and Industries in DHT
The key is that not all makers are subject to regulations in the same manner nor do they exhibit the same affinity for risk. For example, while direct-to-consumer advertising is highly regulated for pharmaceutical products, the oversight is less consistent for over-the-counter (OTC) “devices,” such as some medical tests not regulated by the FDA or FTC [,]. This not only results in a less than comprehensive regulatory coverage of AI medical devices and DHTs but also involuntarily creates an ecosystem where makers develop and market their products around the varying gaps in regulatory coverage. Certain products, such as OTC medical device algorithms to detect sleep apnea, may be subject to less regulation and thus have an advantage over established health care products. OTC sleep apnea devices represent a category of products that can fall in the “interstitial spaces” of regulatory oversight, as they are not always subject to the same level of scrutiny as prescription devices. These products often include wearable sensors, smartphone apps, or other consumer-grade devices that purport to detect symptoms of sleep apnea, such as disrupted breathing or low oxygen levels during sleep. Many OTC sleep apnea devices may fall into class I or II and thus may not require premarket approval, which is the most stringent type of device marketing application required by the FDA. Instead, they may only need to meet the requirements for 510(k) clearance, which is a less rigorous process and does not require clinical trials. However, there are also some OTC sleep apnea devices that do not fall under any FDA regulation because they are marketed as “wellness” or “lifestyle” products rather than medical devices. They do not detect signs of sleep apnea and are not marketed as a medical device but as a sleep improvement system; therefore, they do not fall under FDA regulation.
This lack of consistent regulation can create opportunities for companies to market products with less oversight and potentially greater profit margins, but it can also lead to consumer confusion and potential safety risks if the products do not perform as advertised. It is a clear example of how the existing regulatory framework may struggle to keep up with the rapid pace of innovation in DHT. Clear and well-defined regulations play a pivotal role, especially during the transition from exploratory phases to scaling products, enabling smoother, efficient scaling processes.
In many ways, the opposite situation exists for larger established health care organizations. A highly regulated pharmaceutical company that is already subject to the many previously discussed compliance regimes and other complex corporate regulatory obligations may find it too difficult or risky to attempt digital innovation, as the burden of reporting, evidence, and oversight are all greatly heightened compared to niche innovators. This scenario must be discouraged, as these organizations have deep expertise within the disease areas where they have successfully delivered drugs and devices. This expertise could lend itself to success beyond many health technology startups, which often fail due to a lack of market fit of their products []. Encouraging a balance between regulatory clarity and flexibility is paramount to fostering innovation across diverse players in the digital health landscape. Indeed, there is a cost to regulatory compliance, which is more readily absorbed by well-resourced companies. Smaller startups may not have sufficient funding to run the optimal size and scale validation study. They may have funding constrained by the need to showing promise to investors in order to survive to their next round of funding.
Evidence Requirements and Claims
While one side of the matrix is the nature of the products being developed and the types of makers, the evidence supporting these products is equally if not more diverse. Companies with very different sizes and areas of expertise may be competing openly within a range of product categories and evidence strategies with clinical development plans that seem lacking. Much of this may be attributed to the relative lack of maturation of the AI medical device and DHT spaces, which has led to a wide range of interpretation of the guidelines. This can pose challenges for companies with more rigid regulatory boundaries, which wish to participate in this evolving experimental domain and have substantial evidence strategies to support product development but are uncomfortable as the space is not mature. For most, the first step is to determine the type of product being developed. However, when dealing with products designed for medical purposes or functions, it is essential to ensure that it addresses an unmet medical need. In the United States, the product type can vary from a device software or algorithm that may be classified as mobile medical apps, software functions that are not medical devices, clinical decision support software, or software as a medical device (SaMD) [-]. Each of these product types needs different types and levels of evidence to support them in the market and may need regulatory approval. While the FDA offers guidance on how to determine the product type, significant judgment is required due to similarities within the categories as well as the severity of the disease indication. This necessitates an iterative thought process, considering multiple regulatory guidance alongside the evidence strategy and clinical development plan [].
To simplify this process, we developed the graphical consolidated regulatory decision framework shown in 
Figure 1 [-]. This framework builds on the approach in the FDA Guidance, Software as a Medical Device (SAMD): Clinical Evaluation []. Additional details supporting this framework are available in . A precursor to the workflow is determining whether the clinical association is well-established or novel. This can be nontrivial, as many SaMD products lack clinically established standards due to the novelty of the product. When there is a well-established clinical association, these SaMD have outputs with well-documented association as identified in sources such as clinical guidelines, clinical studies in peer-reviewed journals, consensus for the use of the SaMD, international reference materials, or other similar well-established comparators of previously marketed devices. When the clinical association is novel, these SaMD may involve new inputs, algorithms, outputs, new intended target population, or new intended use. An example may include the combination of nonstandard inputs (eg, mood or pollen count), with standard inputs (eg, blood pressure or other physiological signals), that uses novel algorithms to detect deterioration of health or diagnosis of a disease.
Figure 1). CDS: clinical decision support; IVD: in vitro diagnostic; SaMD: software as a medical device. The importance of objective consideration of these questions cannot be overstated. Frequently, innovators have embedded biases within their assumptions that cloud judgment in this assessment.
Any one or combination of these biases can threaten or derail the development of novel technology products including product integrity and patient safety. For example, the well-publicized case of the FDA warning letter that caused Owlet to cease selling their Smart Sock and copackaged products includes several of these biases []. The FDA determined that the product was a medical device but the maker had not reached the same determination []. According to the FDA, the apnea alarms had inadequate clinical evidence, and parents could potentially seek emergency care due to product alarms that had inadequately established clinical association (prestep), specifically dips in oxygen saturation as determined by pulse oximetry in infants during sleep []. This incident exemplifies how the misclassification of a product type and inadequate clinical evidence highlight the challenges in navigating ambiguous regulatory guidelines. Clear regulations not only mitigate the risk of inadequate clinical evidence, reducing the likelihood of triggering unnecessary care in this case, but also highlight the significance of aligning evidence strategies with robust clinical development plans to avoid such pitfalls.
In contrast, Apple’s approach to irregular heartbeat notification serves as an example that a technology company that operates outside the traditional health care sphere can diligently address product-type classification and evidence. Apple developed these FDA-cleared features via rigorous and traditional approaches using randomized controlled trials [,]. The resulting product label is considered FDA-regulated; therefore, the claims made about the product must not exceed the evidence produced, the specifics listed within the clearance letter, or the resulting label []. However, Apple is an exception within the technology industry in size, scope, and resourcing. The average medical device maker is much smaller and must build their product strategies around regulatory regimes in order to get their product to market. This is not solely about the avoidance of regulation. Many innovations pass the bar for regulatory clearance but not the bar for reimbursement, which can be a difficult and unprofitable market situation. Alternatively, developing a product that does not meet the bar for FDA regulation can result in a highly profitable “health” or “fitness” application that, while not regulated or reimbursed by health insurance, can be highly profitable by volume of sales even at very low-price points.
If the maker determines that the software product is not a medical device, the next step is to determine whether the product is a decision support tool, and this can be accomplished with the aid of the earlier-referenced FDA guidance document.
Continuing with the framework, if the software is a medical device, the next step is to determine whether the software is a SaMD. If not, the user is directed toward traditional medical device pathways. If so, they are guided through the SaMD categorization process. This is complicated by the requirement for a deep understanding of the medical indication and all possible outcomes from any chosen route. Determining whether a SaMD treats or diagnoses, whether it drives clinical management, or simply informs clinical care yields a level of interpretation that often varies by stakeholders. In addition, it must be simultaneously determined whether these actions occur in a critical, serious, or nonserious medical situation. The FDA has published guidance on when and whether independent review of these decisions should be included.
Once the SaMD is categorized, the next step is to determine whether the product is an in vitro diagnostic (IVD) or non-IVD SaMD. The rubric directs the user toward evidence generation in either case. When the SaMD is non-IVD and novel clinically, the evidence generation process requires the establishment of a valid clinical association, analytical validation, and clinical validation of the product. These steps have been elaborated in detail within the previously referenced work by Goldsack et al []. When the SaMD is an IVD, the evidence generation process is analogous to the non-IVD case and can follow the stages of the clinical evidence assessment process as outlined in the Global Harmonization Task Force’s Clinical Evidence for IVD Medical Devices [].
Product Labeling Standards as a Guide
One improbable solution to creating clear and concise regulations would be the reorganization of current regulatory regimes to produce a new agency focused directly on the regulation of health care technologies. However, we can gain some insights from the nutrition industry.
Today, there are 3 agencies responsible for the regulation of food and nutrition information, the FDA, the FTC, and the Food Safety and Inspection Service (FSIS) of the US Department of Agriculture []. This may appear logical, assuming that each agency shares part of an overall mission. However, the reality is that handoffs, overlapping, and gray areas decrease the regulatory effectiveness or, at minimum, create confusion of roles and responsibilities. Continuing the example of food labeling, the FTC regulates food advertising, while the other 2 agencies share responsibility for regulating labels: FSIS regulates meat, poultry, and egg labeling and FDA regulates labeling for all other foods and nonspecified red meat (game). The Nutrition Labeling and Education Act addressed FDA-regulated packages and FSIS-issued parallel regulations. As an example of a gap, there are no provisions in the regulatory authorities defined by Congress that allow the FDA to approve dietary supplements for safety before they reach the consumer []. This results in fragmented safety data and little ability to forecast or prevent harmful products from reaching consumers [].
There is a great deal that the digital health space can learn from nutrition and food labeling. Specifically, the FDA or Nutrition Labeling and Education Act and the FSIS oversee 3 elements of food package labeling: nutrient content, nutrient content claims, and “disease” claims. Further, the FDA has restricted health claims to a small number of permitted claims []. This type of strategic and comprehensive approach to labeling is a model that if applied to AI medical devices and DHTs would improve transparency and clarify their benefits and risks. Elements are starting to appear in relevant subdomains of digital health such as cybersecurity, computing hardware, clinical decision support, and medical devices [-]. Examples extending the nutrition-based health claims into the digital domain are shown in 
Figure 2. This figure shows health risk areas linked to nutrients having FDA-approved health claims. The figure shows those same health risk areas linked to digital health elements that could also have an impact on risk. Standardized AI medical device and DHT product labeling across prescription and nonprescription products could address the diversity of innovators and makers just as nutrition labeling levels the field between small farms and large, industrial food production corporations, as all are held to the same standards.
Different aspects of the creation, oversight, and enforcement of such labeling regulations would likely fall within the purview of existing regulatory bodies assessing medical devices and algorithm-driven DHTs. As 
Table 1 suggests, this is a current patchwork of different agencies without necessarily one central authority. The table indicates how the different agencies could each play a role in the oversight of AI medical devices and DHTs and the relevant product development.
| Federal agency | Relevant aspects of current role | Possible enhanced role |
| US Food and Drug Administration | Regulation of medical devices, MMAsa, SaMDsb | Develop specific guidelines for labeling AIc medical devices, including continuous learning algorithms, and establish a clear pathway for marketing authorizations and DTCd |
| Federal Trade Commission | Enforcement of federal laws that prevent fraud, deception, and unfair business practices (as in advertising) | Regulate use of labels in DTC advertising of AI medical devices and DHTs |
| Centers for Medicare & Medicaid Services | Administers the Open Payments program, enhancing transparency by collecting and publicly disclosing information about financial relationships between health care providers and pharmaceutical or medical device manufacturers | Outline required transparency elements in the label, capturing health care provider financial interactions with companies in the design, testing, and use of DHTs |
| US Department of Health and Human Services, Office for Civil Rights | Ensures the privacy and security of protected health information | Ensure that the AI medical device label provides transparency in how AI devices use patient data and enforce penalties for noncompliance |
| National Institute of Standards and Technology | Develops and maintains technical standards | Establish the benchmarks cited on the label for AI performance, safety, and interoperability with other medical systems |
| US Consumer Product Safety Commission | Protects the public from the risk of injury or death associated with consumer products | Issue product recalls for all other DHTs or AI-enabled health care devices not under the Food and Drug Administration’s purview |
aMMA: mobile medical app.
bSaMD: software as a medical device.
cAI: artificial intelligence.
dDTC: direct to consumer.
Al-Based DHT Package Labeling
Regardless of the type of maker, a significant unmet need that can inform innovators’ strategies is standardized package labeling. Based upon the known potential harms and known potential limitations in AI-based DHTs, the minimum content for analogous product labeling would include the type of algorithm, the framework used for evidence generation, qualification and quantification of reproducibility, the ethical framework used, a description of the data used to train the model including how it was collected and consented, a statement on how bias was minimized or quantified, the risk management framework used to aggregate these various elements, and performance metrics [-]. This would enable a DHT packaging label, similar to the example shown in Table 2.
| DHT or algorithm design element | Permitted health claim | Example |
| Type of evidence basis | Primary benefit or utility | Randomized controlled trials, real-world evidence, etc |
| Ethical framework | Population benefit-risk | Inclusion and exclusion criteria as well as interpretability and explainability |
| Reproducibility | Statistically quantified and qualified claims | Specific indications and efficacy, data lineage, model versioning |
| Training data description | Applicability and specificity to populations | Rationale for included and excluded populations, training and testing data split |
| Disclosure of bias | Limitations of use and contraindications | Phenotypic traits such as skin tone |
| Risk management framework | Product integrity | Cybersecurity resilience, prevention of AI poisoning, measures for protecting user data (such as differential privacy) |
| Performance | Primary benefit or utility | Sensitivity, specificity, negative predictive value, positive predictive value |
Labeling would directly counter the real and perceived black box problem and inform clinicians, researchers, patients, and caregivers in a manner that is equivalent to how they study, learn, and use new prescription and OTC drugs, diagnostics, and medical devices today [,].
New regulatory frameworks often face pushback from those they regulate, and labeling regulations are likely no exception. This resistance can be mitigated somewhat with guidance documents that incorporate feedback from various manufacturers, educational initiatives, and avenues for direct interaction between companies and regulatory bodies.
Conclusions
AI medical products and DHTs hold immense promise, but the diverse regulatory constraints among product makers necessitate a standardized approach. This is especially critical for smaller AI developers who operate in a different landscape than health care or larger industries. To create consistency, adopting minimum product labeling requirements, understanding claims, and having substantial evidence plans become essential. As innovation accelerates, ensuring equity in the ecosystem will allow both emerging and mature technology innovators to contribute meaningfully without being hindered by not only regulatory disparities but also ambiguities, such as the uncertain classification of certain AI applications and the lack of clear communication standards. Future research exploring the various reimbursement strategies and ethical implications of AI across product makers would be valuable to providing a more complete picture of this space. The perspectives from a wide range of digital health ecosystem stakeholders should be included to ensure that their diverse needs and expectations are being addressed.
Acknowledgments
This research was a collaboration between Duke University Clinical Research Institute and Pfizer Inc. This research was funded by Pfizer Inc.
Authors' Contributions
EP, KN, EF, and TT contributed to the conceptualization, writing of the original draft, and the reviewing and editing of the manuscript. EP, EF, and TT prepared and finalized all figures. All authors have read and approved the final manuscript.
Conflicts of Interest
KN is an employee and shareholder of Pfizer Inc. EP was affiliated with Duke Clinical Research Institute at the time of this initiative and is currently affiliated with Pluto Health.
References
- Geier B. What did we learn from the dotcom stock bubble of 2000? Time. Apr 14, 2015. URL: https://time.com/3741681/2000-dotcom-stock-bust [Accessed 2024-01-07]
- Dewey C. 36 ways the web has changed us. The Washington Post. Mar 12, 2014. URL: https://www.washingtonpost.com/news/arts-and-entertainment/wp/2014/03/12/36-ways-the-web-has-changed-us [Accessed 2024-01-07]
- What is digital health? US Food and Drug Administration. 2020. URL: https://www.fda.gov/medical-devices/digital-health-center-excellence/what-digital-health [Accessed 2024-01-07]
- What’s driving the Hype Cycle for generative AI, 2024. Gartner. Nov 14, 2024. URL: https://www.gartner.com/en/articles/hype-cycle-for-genai [Accessed 2024-01-07]
- Yu KH, Beam AL, Kohane IS. Artificial intelligence in healthcare. Nat Biomed Eng. Oct 2018;2(10):719-731. [CrossRef] [Medline]
- Haug CJ, Drazen JM. Artificial intelligence and machine learning in clinical medicine, 2023. N Engl J Med. Mar 30, 2023;388(13):1201-1208. [CrossRef] [Medline]
- Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med. Jan 2022;28(1):31-38. [CrossRef] [Medline]
- Jiang F, Jiang Y, Zhi H, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. Dec 2017;2(4):230-243. [CrossRef] [Medline]
- Paul D, Sanap G, Shenoy S, Kalyane D, Kalia K, Tekade RK. Artificial intelligence in drug discovery and development. Drug Discov Today. Jan 2021;26(1):80-93. [CrossRef] [Medline]
- Smith GF. Artificial intelligence in drug safety and metabolism. Methods Mol Biol. 2022;2390:483-501. [CrossRef] [Medline]
- Sivakumar CLV, Mone V, Abdumukhtor R. Addressing privacy concerns with wearable health monitoring technology. WIREs Data Min Knowl. May 2024;14(3):e1535. [CrossRef]
- McKinnon J, Kendall B. Is the FTC up to the task of internet regulation? The Wall Street Journal. Dec 15, 2017. URL: https://www.wsj.com/articles/is-ftc-up-to-the-task-of-internet-regulation-1513349967 [Accessed 2024-01-07]
- Mandl KD, Perakslis ED. HIPAA and the leak of “deidentified” EHR data. N Engl J Med. Jun 10, 2021;384(23):2171-2173. [CrossRef] [Medline]
- Silberman J, Wicks P, Patel S, et al. Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework. NPJ Digit Med. May 31, 2023;6(1):101. [CrossRef] [Medline]
- Coravos A, Doerr M, Goldsack J, et al. Modernizing and designing evaluation frameworks for connected sensor technologies in medicine. NPJ Digit Med. 2020;3:37. [CrossRef] [Medline]
- Goldsack JC, Coravos A, Bakker JP, et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). NPJ Digit Med. 2020;3:55. [CrossRef] [Medline]
- Software as a medical device: possible framework for risk categorization and corresponding considerations. International Medical Device Regulators Forum. Sep 18, 2014. URL: http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf [Accessed 2024-01-07]
- Guidance on qualification and classification of software in regulation (EU) 2017/745—MDR and regulation (EU) 2017/746—IVDR. Medical Device Coordination Group. URL: https://health.ec.europa.eu/system/files/2020-09/md_mdcg_2019_11_guidance_qualification_classification_software_en_0.pdf [Accessed 2025-03-24]
- Device software functions, including mobile medical applications. US Food and Drug Administration. URL: https://www.fda.gov/medical-devices/digital-health-center-excellence/device-software-functions-including-mobile-medical-applications [Accessed 2024-01-07]
- Digital therapeutics (DTx). European Data Protection Supervisor. URL: https://edps.europa.eu/press-publications/publications/techsonar/digital-therapeutics-dtx_en#:~:text=Digital%20Therapeutics%20(DTx)%20are%20evidence,a%20medical%20disorder%20or%20disease [Accessed 2024-01-07]
- Refuse to accept policy for 510(k)s guidance for industry and Food and Drug Administration staff. US Food and Drug Administration. 2022. URL: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/refuse-accept-policy-510ks [Accessed 2024-01-07]
- From the manufacturers’ mouth to your ears: direct to consumer advertising. US Food and Drug Administration. 2015. URL: https://www.fda.gov/drugs/special-features/manufacturers-mouth-your-ears-direct-consumer-advertising [Accessed 2024-01-07]
- Rigg K. The top 3 reasons health care startups fail. High Tech World. 2022. URL: https://www.htworld.co.uk/news/the-top-3-reasons-health-tech-startups-fail [Accessed 2025-03-24]
- Examples of software functions that are not medical devices. US Food and Drug Administration. 2022. URL: https://www.fda.gov/medical-devices/device-software-functions-including-mobile-medical-applications/examples-software-functions-are-not-medical-devices#:~:text=Software%20functions%20that%20are%20intended,are%20not%20intended%20for%20use [Accessed 2024-01-07]
- Clinical Decision Support Software. US Food and Drug Administration. 2022. URL: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software [Accessed 2024-01-07]
- Software as a Medical Device (SaMD). US Food and Drug Administration. 2018. URL: https://www.fda.gov/medical-devices/digital-health-center-excellence/software-medical-device-samd [Accessed 2024-01-07]
- Classify your medical device. US Food and Drug Administration. 2020. URL: https://www.fda.gov/medical-devices/overview-device-regulation/classify-your-medical-device [Accessed 2024-01-07]
- Guidance—MDCG endorsed documents and other guidance. European Commission—Public Health. URL: https://health.ec.europa.eu/medical-devices-sector/new-regulations/guidance-mdcg-endorsed-documents-and-other-guidance_en [Accessed 2024-01-07]
- Clinical Decision Support Software Frequently Asked Questions (FAQs). US Food and Drug Administration. 2022. URL: https://www.fda.gov/medical-devices/software-medical-device-samd/your-clinical-decision-support-software-it-medical-device [Accessed 2024-01-07]
- Is your software a medical device? European Commission. Mar 23, 2021. URL: https://health.ec.europa.eu/system/files/2021-03/md_mdcg_2021_mdsw_en_0.pdf [Accessed 2025-03-24]
- Software as a medical device (SAMD): clinical evaluation—guidance for industry and Food and Drug Administration staff. US Food and Drug Administration. Dec 8, 2017. URL: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/software-medical-device-samd-clinical-evaluation [Accessed 2024-01-07]
- Marchante M. Owlet Smart Sock stops U.S. sales after FDA warning. Miami Herald. Dec 2021. URL: https://www.miamiherald.com/news/recalls/article256291067.html [Accessed 2025-04-01]
- Warning letter to Owlet Baby Care, Inc. US Food and Drug Administration. URL: https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/warning-letters/owlet-baby-care-inc-616354-10052021 [Accessed 2025-03-24]
- Bonafide CP, Jamison DT, Foglia EE. The emerging market of smartphone-integrated infant physiologic monitors. JAMA. Jan 24, 2017;317(4):353-354. [CrossRef] [Medline]
- Direct-to-consumer tests. US Food and Drug Administration. 2018. URL: https://www.fda.gov/medical-devices/in-vitro-diagnostics/direct-consumer-tests [Accessed 2025-03-24]
- Perez MV, Mahaffey KW, Hedlin H, et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N Engl J Med. Nov 14, 2019;381(20):1909-1917. [CrossRef] [Medline]
- Statement from FDA Commissioner Scott Gottlieb, M.D., and Center for Devices and Radiological Health Director Jeff Shuren, M.D., J.D., on agency efforts to work with tech industry to spur innovation in digital health. US Food and Drug Administration. 2018. URL: https://www.fda.gov/news-events/press-announcements/statement-fda-commissioner-scott-gottlieb-md-and-center-devices-and-radiological-health-director [Accessed 2024-01-07]
- Clinical evidence for IVD medical devices. International Medical Device Regulators Forum. Nov 2012. URL: https://www.imdrf.org/sites/default/files/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n6-2012-clinical-evidence-ivd-medical-devices-121102.pdf [Accessed 2024-01-07]
- Consumer use of information: implications for food policy. US Department of Agriculture, Economic Research Service. URL: https://www.ers.usda.gov/webdocs/publications/41905/51665_ah715c.pdf?v=0 [Accessed 2025-04-01]
- Questions and answers on dietary supplements. US Food and Drug Administration. 2022. URL: https://www.fda.gov/food/information-consumers-using-dietary-supplements/questions-and-answers-dietary-supplements [Accessed 2024-01-07]
- Beach C. Report finds enormous increase in number of food items recalled in 2022. Food Safety News. Mar 15, 2023. URL: https://www.foodsafetynews.com/2023/03/report-finds-enormous-increase-in-number-of-food-items-recalled-in-2022/ [Accessed 2024-01-07]
- Label claims for conventional foods and dietary supplements. US Food and Drug Administration. 2022. URL: https://www.fda.gov/food/food-labeling-nutrition/label-claims-conventional-foods-and-dietary-supplements [Accessed 2024-01-07]
- Lemos R. Cybersecurity 'nutrition' labels still a work in progress. Dark Reading. Nov 11, 2022. URL: https://www.darkreading.com/dr-tech/cybersecurity-nutrition-labels-still-a-work-in-progress [Accessed 2024-01-07]
- Sendak MP, Gao M, Brajer N, Balu S. Presenting machine learning model information to clinical end users with model facts labels. NPJ Digit Med. 2020;3:41. [CrossRef] [Medline]
- A new role for the FDA in medical device regulation. The Center for Growth and Opportunity. Jun 17, 2019. URL: https://www.thecgo.org/research/a-new-role-for-the-fda-in-medical-device-regulation/ [Accessed 2024-01-07]
- Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. Jul 6, 2023;6(1):120. [CrossRef] [Medline]
- Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ, SPIRIT-AI and CONSORT-AI Working Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digit Health. Oct 2020;2(10):e549-e560. [CrossRef] [Medline]
- Beam AL, Manrai AK, Ghassemi M. Challenges to the reproducibility of machine learning models in health care. JAMA. Jan 28, 2020;323(4):305-306. [CrossRef] [Medline]
- Murphy K, Di Ruggiero E, Upshur R, et al. Artificial intelligence for good health: a scoping review of the ethics literature. BMC Med Ethics. Feb 15, 2021;22(1):14. [CrossRef] [Medline]
- Frenkel S, Thompson S. ‘Not for machines to harvest’: data revolts break out against A.I. The New York Times. Jul 15, 2023. URL: https://www.nytimes.com/2023/07/15/technology/artificial-intelligence-models-chat-data.html [Accessed 2024-01-07]
- Igoe K, Harvard T. Algorithmic bias in health care—how to prevent it. Chan School of Public Health. 2012. URL: https://www.hsph.harvard.edu/ecpe/how-to-prevent-algorithmic-bias-in-health-care/ [Accessed 2024-01-07]
- AI risk management framework. National Institute of Standards and Technology. URL: https://www.nist.gov/itl/ai-risk-management-framework [Accessed 2024-01-07]
- Blouin L. AI’s mysterious 'black box' problem, explained. Dearborn. 2023. URL: https://umdearborn.edu/news/ais-mysterious-black-box-problem-explained [Accessed 2024-01-07]
- Wadden JJ. Defining the undefinable: the black box problem in healthcare artificial intelligence. J Med Ethics. Jul 21, 2021:medethics-2021-107529. [CrossRef] [Medline]
Abbreviations
| AI: artificial intelligence |
| DHT: digital health technology |
| FDA: Food and Drug Administration |
| FSIS: Food Safety and Inspection Service |
| FTC: Federal Trade Commission |
| IVD: in vitro diagnostic |
| OTC: over-the-counter |
| SaMD: software as a medical device |
Edited by Bradley Malin, Khaled El Emam; submitted 16.02.24; peer-reviewed by Gro Berntsen, Mahshad Lotfinia, Ryan Jung, Usha Lokala; final revised version received 10.01.25; accepted 23.02.25; published 07.04.25.
Copyright© Eric Perakslis, Kimberly Nolen, Ethan Fricklas, Tracy Tubb. Originally published in JMIR AI (https://ai.jmir.org), 7.4.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR AI, is properly cited. The complete bibliographic information, a link to the original publication on https://www.ai.jmir.org/, as well as this copyright and license information must be included.