This review is a subanalysis of a larger umbrella review examining barriers to and facilitators of digital health tool (DHT) implementation in clinical settings. This review and the parent review followed established methods for conducting scoping reviews [15,16] and the PRIOR (Preferred Reporting Items for Overviews of Reviews) reporting guideline (Checklist 1) [17]. The parent review protocol is registered with PROSPERO [18]. This review deviates from the registered protocol as this is a subanalysis of the parent review. An umbrella review is an appropriate method given the rapid pace of AI implementation, the volume of reviews on AI implementation, and the need to comprehensively synthesize implementation determinants for AI in clinical settings [19].

The parent umbrella review search strategy was conducted in 3 databases with the help of a reference librarian: PubMed, Scopus, and CINAHL (Multimedia Appendix 1). The initial search was conducted to identify all types of reviews (eg, systematic, scoping, and narrative) published in English between September 29, 2013, and November 6, 2023. These search dates are for the reviews and not the primary studies included in the reviews. An updated search was conducted to include reviews published through July 10, 2025. The date range was determined based on the time frame during which DHTs became more prevalent in health care [20]. The search terms included both MeSH and free-text terms related to digital tools, clinical care settings, and implementation (Multimedia Appendix 1).

Reviews were included in the parent review if they met the following criteria: publication in English; description of determinants (barriers and facilitators) of implementing digital tools in clinical care settings (eg, inpatient hospital setting, outpatient routine care, and outpatient emergency care); publication between 2013 and 2025; and full-text availability. There was no geographic constraint. Reviews were included regardless of the geographic location in which the reviews’ included studies were conducted. Reviews were excluded if they focused on nonclinical settings, did not describe the implementation of digital tools, did not focus on health care providers (eg, physicians and nurses) as implementers of the DHTs, or did not provide detailed methods, such as search terms and dates and inclusion criteria, to be able to discern the origin of the results presented in each review (Multimedia Appendix 2).

The initial search returned 4180 reviews. The updated search returned 1659 reviews, for a total of 5839 reviews. Reviews were imported into Covidence (Veritas Health Innovation), where duplicates were removed. For title and abstract screening and full text review, blinded dual independent screening was conducted. Consensus meetings were held regularly throughout screening to resolve conflicts or disagreements for inclusion (research team members: A Gilbert, A Gacad, LP, MKK, DT, and OE).

The study team collectively developed and pilot tested 2 data extraction protocols. The first protocol abstracted review characteristic data from all reviews in the parent review sample and was used to identify subsets of reviews to explore implementation determinants with a second data extraction protocol. This was done due to the large number of included reviews and time and resource intensity to qualitatively abstract implementation determinants. This paper focuses on the subset review for AI DHTs. Data from both extraction protocols were abstracted using dual-blinded independent review with consensus meetings.

In the first extraction protocol, data included the type of DHT (eg, AI, telehealth, and clinical decision support tools); the health care setting the review focused on (eg, outpatient routine care, outpatient emergency care, and inpatient); health or health care outcomes the review focused on (eg, chronic disease prevention and management, patient engagement and satisfaction, and health care use); and the geographic setting the review focused on (eg, United States only, countries outside the United States, and global [United States and other countries]). AI tools were defined as any DHT that used AI or ML, which may have overlapped with other DHT types (eg, telehealth and clinical decision support tools). If no data existed, reviewers marked “none specified.” Data were extracted by reviewers in Covidence (research team members: A Gilbert, A Gacad, LP, MKK, and DT).

The second protocol was informed by CFIR, which outlines determinants of implementation across outer setting (eg, policies and law and partnerships), inner setting (eg, communications and available resources), implementer (eg, health care provider motivation and capability), and innovation (eg, AI complexity and design) levels. If a determinant did not fit within an existing construct, reviewers placed the text into “other” categories within the appropriate level (outer setting, inner setting, implementer, and innovation). If it was unclear whether a determinant was a barrier or facilitator, reviewers indicated it as both. Determinants were also indicated as both a barrier and a facilitator if the review presented them that way. Data were also extracted on whether a review used a theory, model, or framework (yes or no); the name of the theory, model, or framework; and how it was applied (eg, to inform research questions or aims, to support data extraction or analysis, and to interpret results or frame discussion).

The extraction template was built in Microsoft Excel and Qualtrics XM. Data extraction was conducted for all reviews identified through the first data extraction protocol that focused on AI DHTs. Extraction was conducted independently by 2 reviewers who copied and pasted review text into the Excel file (research team members: A Gilbert, A Gacad, LP, MKK, DT, ON, KT, WLT, BW, and PJ). Consensus meetings were held by the paired reviewers to discuss extraction and come to consensus for the final data. The final data were then entered into the Qualtrics form.

Data were downloaded from Qualtrics. Study team members (A Gilbert, A Gacad, MKK, and LP) reviewed and summarized the results for each CFIR construct and the “other” category. Study team members met to discuss the summarized results and emergent themes. The results are presented by CFIR domain or level by summarizing barriers and facilitators across the constructs using narrative synthesis.

The quality of reviews in this AI subanalysis review was assessed using the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Systematic Reviews and Research Syntheses through dual-blinded independent assessment and consensus meetings (research team members: A Gilbert, A Gacad, LP, and MKK) [21].

The study team used the Excel-based GROOVE (Graphical Representation of Overlap for Overviews) tool to calculate the corrected covered area (CCA) for reviews in this AI subanalysis review. The CCA is the prevalence of primary studies included in more than one of the included reviews [22]. The tool calculates the CCA for the sample, with a CCA of 0% to 5% representing a slight overlap, 6% to 10% a moderate overlap, 11% to 15% a high overlap, and above 15% a very high overlap.

Data extracted from the reviews included in this subanalysis were downloaded from Qualtrics into an Excel file for analysis. All text that was deductively extracted into CFIR constructs was summarized through consensus meetings. The “other” categories for each level were analyzed using an inductive approach to identify themes within CFIR domains (innovation, implementer, inner setting, and outer setting) that did not fit within existing CFIR constructs. Inductive analyses were conducted using dual-blinded independent review with consensus meetings (research team members: A Gilbert and MKK). These themes are referred to as emergent themes in the Results section. We calculated the number of reviews from which data were extracted for each CFIR construct.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram (Figure 1) shows the number of reviews at each stage of the review.

A total of 22 reviews were included in this AI subanalysis review, with publication years from 2021 to 2025 and covering studies conducted between 1990 and 2025 (Table 1). Reviews varied in type, including systematic reviews (n=16, 73%) [1-3,6,7,23-33], scoping reviews (n=2, 9%) [34,35], rapid reviews (n=1, 5%) [36], literature reviews (n=1, 5%) [37], and narrative reviews (n=2, 9%) [4,38]. Most reviews (n=18, 82%) [1,3,4,6,7,23-30,32,34-37] did not specify a geographic region, although several (n=3, 14%) [2,31,38] reported focusing solely on non-US contexts. Eight (36%) reviews focused on specific health care settings, including inpatient settings (n=4, 18%) [25,27,31,36], outpatient routine care (n=5, 23%) [27,29,31-33], and outpatient emergency care (n=3, 14%) [25,27,30]. Eleven (50%) reviews applied one or more theories, models, or frameworks (TMFs) [1,2,4,6,23,24,27,29,32,33,35]. One (5%) review used TMFs to inform the research question or aim [35], 9 (41%) reviews used TMFs to support data extraction or analysis [1,2,4,24,27,29,32,33,35], and 6 (27%) used TMFs to interpret results and frame the discussion [1,2,6,29,33,35].

Nine (41%) reviews had a quality assessment of 100% [1-3,6,24,25,29,30,32], 14 (64%) reviews had a quality assessment of 80% or higher [1-4,6,23-25,27,29,30,32,33,38], and 4 (18%) reviews had a quality assessment of 50% or less (Table 2) [7,26,31,36]. The CCA for the included reviews indicated slight overlap at 0.4%. Of the 706 primary studies, 655 (92.8%) studies were only included in 1 review, 43 (6.1%) studies were included in 2 reviews, and 2 (0.3%) studies were included in 3 reviews.

This umbrella review synthesized determinants of AI implementation in clinical settings across 22 reviews and revealed a complex, multilevel interplay of technical, individual, organizational, and policy-related factors. While enthusiasm for AI’s potential is high, persistent and interdependent barriers can hinder widespread implementation and sustained use. The findings illustrate that successful AI implementation requires addressing not only traditional determinants captured by CFIR but also numerous emergent constructs that reflect AI’s unique risks, complexities, and sociotechnical dimensions.

This review identified determinants of AI implementation in clinical settings. Many of these determinants have not been observed in other forms of DHTs [49]. Specifically, key determinants included concerns about AI model development and data input as they relate to bias, equity, and validity; AI model transparency and upkeep for supporting clinician trust in AI tool accuracy and reliability; and usefulness of AI models for generalizability across conditions and populations.

These findings connect to the broader literature base, which shows that considerable progress has been made in developing model interpretation methods that aim to explain model predictions by quantifying the contributions of individual input variables at both the individual and population levels [50-53]. These methods are increasingly incorporated into clinical decision support tools to enhance clinicians’ understanding of model outputs and to surface factors associated with predicted risks [50-52]. However, it is important to recognize that most existing interpretation techniques characterize statistical associations rather than causal relationships. As a result, identified “important” features may reflect incidental or confounded correlations rather than true drivers of clinical outcomes. Substantial challenges, therefore, remain in advancing model interpretation approaches that can distinguish causal, actionable risk factors from spurious associations, which is essential for reliable clinical decision-making and effective intervention design [54]. In parallel with efforts to improve model interpretability, substantial progress has also been made in estimating and communicating the uncertainty associated with model outputs [55-58]. Explicitly reporting predictive uncertainty can help clinicians assess the reliability of AI-generated recommendations and calibrate their reliance on these outputs accordingly [59].

Our findings are aligned with other research, which shows that AI models often face substantial challenges in generalizability and robustness over time, as their performance can degrade when deployed in hospital systems or patient populations that differ from the data on which the models were trained. Recent efforts to address this issue include training models on large, diverse datasets that better represent broader populations, as well as developing foundation models pretrained on extensive clinical data to support a wide range of downstream tasks [60-63]. However, important questions remain regarding the extent to which even large-scale models can maintain robust performance in the presence of significant distribution shifts across institutions, care settings, and patient populations. These generalizability and robustness challenges have important equity and fairness implications [64,65]. Models trained on data that underrepresent certain demographic groups, socioeconomic contexts, or care settings may perform systematically worse for these populations [66,67], thereby exacerbating existing health disparities over the long term.

This review found that monitoring and governance are important determinants across multiple levels for AI adoption and implementation. Reviews underscored the need for regulations, laws, and policies at a variety of levels (federal vs state regulation, tort liability to clarify norms, contracting over the use of technologies, statutes governing the regulation of certain technologies through the FDA, professional society policies, and regulating the practice of medicine) and through different mechanisms to support responsible and successful AI adoption and address concerns about accountability, data privacy and security, and ethics.

The literature base suggests that some uses of AI in clinical practice can be governed or at least influenced by existing policies, such as some AI tools that are incorporated into medical devices and may be regulated federally by the FDA. Other uses of AI, such as ambient scribes, may fall outside of existing federal monitoring and governance frameworks, raising uncertainty about whether existing legal and regulatory frameworks, such as those governing health data privacy, sufficiently protect implementing facilities against legal risk. Even where some AI tools may be regulated for their safety and efficacy, other aspects of the same AI tool, such as allocation of liability, may lack clarity, such that the risk of liability may discourage adoption of the tool. For example, it may be unclear whether an AI developer or the adopting physician or health care facility would be the appropriate defendant in a case involving a patient harmed in the course of treatment involving an AI tool. The growing interest in how AI can be applied to clinical care has been accompanied by ongoing discussions on whether the legal accountability for decisions made falls on only the clinical user [68,69] or whether AI developers should be held solely or share accountability [68]. Developing a widely accepted consensus on this will help guide the implementation of large language models for clinical use.

Recent evidence suggests that many different types of legal and regulatory responses could be used to reduce ambiguity and support AI adoption by health care providers or systems. Importantly, at least some of these responses would not require passing new legislation or creating new regulatory frameworks at either the state or federal level, an important but always difficult lever to use. Consider the ways existing legal approaches, such as contract law, may allow developers and implementing facilities to resolve current uncertainties [70,71]. For instance, licensing agreements between health care systems and AI developers could include terms regarding liability and indemnification, and require developers to facilitate risk assessment and safety monitoring. Other private actors, such as the Coalition for Health AI (as one illustrative example), may also play a convening role in the development of best practices around the use of AI in health care settings, such as through the creation of its Responsible AI Guide and Checklist.

Our findings underscore that efforts to build health care providers’ capability and opportunity to use AI DHTs must directly address pervasive concerns about existential threat, skill erosion, and the loss of the human element in clinical care. Across reviews, providers expressed fears that AI could displace clinical roles, undermine professional identity, or lead to deskilling through overreliance on automated recommendations, particularly when AI output conflicted with clinical judgment or was poorly explained. These concerns are not simply attitudinal barriers but reflect rational responses to opaque systems, unclear accountability, and limited guidance on how responsibility is shared between clinicians and AI tools. This review identifies that building trust in AI outputs to support clinician capability may be done through improving model interpretability and providing training and knowledge on AI processes and outputs.

Recent literature shows that current implementation efforts that focus narrowly on technical training may be insufficient. Instead, strategies that explicitly frame AI as an augmentative—not substitutive—DHT are needed, emphasizing the clinician’s role in contextualizing AI output, maintaining ultimate decision-making authority, and preserving clinical expertise [72]. Educational initiatives could incorporate case-based learning that addresses disagreement between AI and clinicians, ethical decision-making, and strategies to avoid automation bias, thereby supporting skill retention and critical appraisal rather than passive reliance on AI [73]. To protect the human interaction at the core of health care, implementation strategies could also prioritize workflow designs in which AI reduces administrative burden and cognitive load, allowing providers to spend more time on relational aspects of care, such as communication, empathy, and shared decision-making [72]. Organizational policies and governance structures that clarify accountability, support reflective practice, and legitimize clinician discretion to override AI recommendations may further mitigate fears of job loss and loss of professional autonomy. Together, these strategies highlight that supporting providers’ capability and opportunity for AI use requires coordinated investments across education, organizational infrastructure, and policy environments, rather than reliance on individual-level motivation alone [74].

The integration of AI into clinical care challenges existing implementation science paradigms. Our findings demonstrate the need to adapt or extend frameworks such as CFIR to capture AI-specific constructs, including AI transparency, model lifecycle dynamics, data governance, and emergent psychological determinants, such as existential threat and relational components. Implementation strategies for AI must therefore be multidimensional, addressing technical optimization, clinician engagement, organizational readiness, and regulatory context simultaneously. Of note, several CFIR constructs, including trialability (innovation); need and motivation (implementer); relational connections, communications, and relative priority (inner setting); and external pressure (outer setting), were not identified in any of the included reviews. This absence may reflect that these determinants are not as salient for AI digital tools as for other interventions; however, this may be an artifact of how determinants were extracted and the language with which these types of determinants may be described in the literature on AI tools.

This umbrella review has some important limitations. The review synthesized evidence from existing reviews rather than primary studies, which may have amplified commonly reported determinants while obscuring contextual nuance, temporal changes, or setting-specific dynamics present in individual implementation efforts. Furthermore, the quality, scope, and methodological rigor of the included reviews varied, and many reviews did not clearly specify health care settings, geographic contexts, or theoretical frameworks, limiting our ability to draw conclusions about how implementation determinants may differ across health systems, populations, or regulatory environments. For example, although we identified inner setting resources and geography as potential determinants, we cannot draw systematic conclusions about differences in determinants across settings or geographic locations. Furthermore, we were unable to extract the phase of implementation or assess the dynamic nature of determinants, which can evolve over time. We used vote counting for how many reviews described CFIR barriers and facilitators; however, this method does not account for varying size and quality of the included reviews. Five of our reviews had quality ratings of 60% or lower, which may limit the rigor of our findings. We conducted a sensitivity analysis, which showed that excluding these reviews resulted in a change of 12% to 16% count of reviews in each CFIR level but did not qualitatively change conclusions drawn as the CFIR levels and constructs continued to be represented by other reviews.

The review was limited to English-language publications and peer-reviewed reviews, potentially excluding relevant gray literature, industry reports, and non-English evidence that may capture more pragmatic or rapidly evolving implementation experiences, particularly in low- and middle-income or non-Western health system contexts. In this study, coding sought to capture what was explicitly stated in the articles and may not have captured relationships among constructs or implicit barriers or facilitators (eg, provider trust may lead to lower motivation among clinicians to use AI). Although we pilot tested the extraction protocol, used a dual-blinded review process, and held regular consensus meetings to resolve discrepancies between extractions, there may still be variation in how review results were interpreted through CFIR. On the basis of how results were presented in reviews, it was not always possible to discern if a determinant was a barrier, facilitator, or both. If it was not possible to determine whether the determinant was a barrier or a facilitator, we coded it as both. Some reviews indicated that a specific determinant was both a barrier and a facilitator. We also coded these determinants as both.

This study has several notable strengths. First, the use of an umbrella review design enabled a comprehensive and high-level synthesis of a rapidly expanding and heterogeneous body of literature on AI implementation in clinical settings, providing a consolidated evidence base that would be difficult to achieve through primary studies alone. Second, grounding the synthesis in CFIR allowed for systematic, theory-informed organization of multilevel implementation determinants, enhancing conceptual clarity and facilitating translation of findings into actionable implementation strategies. Third, the combination of deductive mapping to CFIR constructs with inductive identification of emergent, AI-specific determinants represents a key strength, as it both leverages established implementation science theory and advances it by identifying constructs—such as transparency, accountability, trust, model bias, and professional identity threat—that may not be fully captured in existing frameworks. Fourth, the review used rigorous and transparent methods, including protocol registration in PROSPERO, comprehensive database searches with an updated search window, blinded dual independent screening and data extraction, consensus procedures, and formal quality assessment using the JBI tool, which together strengthen the credibility and reproducibility of the findings.

Effective and equitable AI implementation requires acknowledging and addressing a broad constellation of determinants at the innovation, implementer, inner setting, and outer setting levels. AI introduces unique technical and ethical complexities that exceed the boundaries of existing implementation frameworks. As health systems increasingly adopt AI tools, tailored implementation approaches that emphasize transparency, trust-building, ongoing model monitoring, and policy alignment will be essential to ensure safe and sustainable use in diverse clinical environments.