Artificial intelligence (AI) technologies are being rapidly adopted and implemented in health care settings to augment clinical decisions and the delivery of patient-centered care [1]. The use of AI applications presents a paradigm shift in health care and serves as a positive enabler for achieving the quintuple aims of health care [2]. In particular, AI applications have the potential to further integrate health equity and patient activation to ameliorate siloed and biased care, as advocated by the National Academy of Medicine [2,3]. Fostering a patient-centered culture that considers health equity entails continued partnerships with patients and encourages them to be coleaders of change within the clinical ecosystem [2]. This shift must emerge from the leadership and organizational levels and should include both a commitment to and development of strategic priorities, which include patient and family engaged care [2]. For instance, the Canadian Institute for Advanced Research urges the need for a collaborative and integrative effort to establish an AI for Health strategy to accelerate the adoption and scaling of AI-enabled technologies to provide compassionate and safe care [4]. The Canadian Institute for Advanced Research highlights the importance of including patient perspectives in the development, implementation, and evaluation of AI initiatives [4]. A few studies have reported that a co-design approach engaging patients and the public during the development process could enhance the accuracy, equity, and transparency of AI models [5-7]. Patients are key beneficiaries in the adoption and implementation of AI technologies in clinical settings; thus, engaging patients allows for diversity in perspectives, and their values and needs are included [8,9].

Patient engagement is defined as an individual’s active involvement in the care decision-making process and collaboration with key stakeholders to build an equitable and sustainable health system [10,11]. Understanding patient perceptions is an initial step in fostering patient engagement and ensuring the responsible and safe use of these novel technologies in clinical care settings [8]. A recent survey conducted by the Biron Health Group in Quebec indicated that many residents were in favor of using AI technologies to address health system issues and optimize clinical innovations [12]. The study showed that 63% agreed that AI could prevent adverse outcomes, while 40% believed that it could be used to augment clinicians’ expertise and lead to profound changes in care [12]. Many papers focused on patient perspectives of AI in various medical specialties, such as cardiology, dermatology, and radiology, and how they conceptualize AI technology in health care [13-21]. Although there are several studies focused on understanding patient perspectives in relation to specific AI technologies, patients need to be engaged at different stages of the AI implementation process [9,22-24]. The long-term sustainability of AI technologies in clinical environments vastly relies on patient acceptance, which is influenced by their knowledge and perception of opportunities as well as risks associated with using AI solutions [15].

Despite the positive views on the potential of clinical applications of AI and the promise of AI, there are many fears and misconceptions that remain. A few studies have shown that patients expressed concern regarding the use of personal health records for profit or being distorted by hackers, as this could have an impact on their employment or insurance coverage [15,25]. Balthazar et al [25] contended that even when patients have an in-depth understanding and thoughts on the appropriate use of their personal health information, they may not be able to understand the foundational concepts of machine learning models to make predictions or discern the difference between terms such as privacy and confidentiality. Another significant concern noted in the literature is the systemic bias that can potentially be embedded in AI models and that can stigmatize or marginalize certain populations [7,8,25]. Patients’ perspectives on AI may differ based on their socioeconomic status, ethnicity, and vulnerability [25]. Furthermore, patient engagement helps to cocreate the health care system, address the underlying social determinants of health [2,26], and ultimately democratize access to AI innovations [5]. Thus, minimizing the consequences and concerns of AI technologies is pivotal in facilitating trust and ensuring the successful adoption of these tools in clinical practice.

Establishing patient trust becomes increasingly difficult in a rapidly evolving digital space with complex and less-transparent AI technologies [8]. Studies have asserted that even though AI can empower patients, the lack of transparency and explanation of processes owing to the black box phenomenon could diminish patients’ trust if the model is not reflective of current evidence, is biased, or is erroneous [27-30]. Notwithstanding the high accuracy and advancements in AI technologies, patients value human judgment when making care decisions [31]. Empathy, compassion, and trust play a significant role in forming the basis for augmenting patient-centered care and ensuring the sustainability of AI innovations [27,32]. It is vital for care providers to actively engage patients when making care decisions and foster a therapeutic relationship [32]. Kerasidou [32] highlighted that patients preferred to interact with health care professionals (HCPs) who both have clinical expertise and provide empathetic and compassionate care. An interpersonal care model allows HCPs to better understand and address individual needs and to build patients’ trust [7,32]. In addition, the literature emphasizes the importance of public perception and literacy in fostering trust and removing any potential misconceptions regarding AI [28]. Esmaeilzadeh et al [27] advocated for patient education to ensure that patients are prepared to make informed decisions and communicate effectively with their care providers. The authors underlined the importance of patients being active partners during the adoption and integration of AI innovations in their care [27]. Thus, patient engagement helps diminish the gap between patients’ expectations of AI technologies and their experiences with care providers [33].

Cutting-edge technologies such as AI are poised to transform the health care system, as we slowly shift to a new revolution in the next era [20]. This shift is facilitated through medical education; however, there are gaps in its implementation across all levels of medical education. This includes the lack of standardization, varying levels of AI literacy among faculty, and limited infrastructure for embedding AI concepts within existing curricula [34]. There is a need for medical education to go beyond medical informatics and machine learning, enabling HCPs to operationalize these novel tools at the point of care [34]. Despite the use of AI to accelerate innovations in patient care and the need for patient voices, there is limited literature on patient engagement and their perceptions of how AI will affect care delivery, thus ensuring AI technologies are aptly integrated within the clinical environment and cultivating patient trust [9]. To put the needs of patients first in creating a healthier world using AI, the objective of this study was to elucidate patients’ perceptions of what skills they believe HCPs should have in preparation for this AI-enabled future and how we can better engage patients when adopting and deploying AI technologies in health care.

A qualitative study design was used to elicit participants’ perceptions of the adoption and implementation of AI within the health care ecosystem.

This study was approved by the University Health Network Research Ethics Board (ID:20-6148.2).

A maximum purposive sampling approach was used to ensure that the participants represented various comfort levels with AI technology and contexts in which they received care. It was also used to gain insights into the diverse perspectives that should be considered when adopting and deploying AI technologies in clinical settings. Purposive sampling enables researchers to identify and select participants based on their ability to yield relevant information about a particular phenomenon [35,36]. Participants were recruited from a national group of approximately 25 patients via email invitations sent on behalf of the research team by education committee members of Canada Health Infoway. Participants who consented to participate in the interviews were asked to inform individuals within their networks via a snowball sampling approach [37]. The snowball sampling method was used to recruit additional participants, who may add valuable perspectives to the study and enable an in-depth understanding of the phenomenon. Individuals were eligible to participate if they were patients at any Canadian medical center (acute or long-term) and were able to provide informed consent.

Semistructured interviews were conducted with patients on the web via Microsoft Teams, in following COVID-19 pandemic social distancing measures. An instructional designer and research associates who have experience in qualitative research methods conducted the interviews. In addition, the interviewers have formal education in health informatics (TJ), public health (SY), educational technology (MC), and educational and counseling psychology (MZ). A semistructured interview guide consisting of 13 open-ended questions was used to guide discussions (Multimedia Appendix 1). The interviewers probed participants when necessary to further explore and understand salient ideas. The participants’ level of comfort in sharing their perceptions and experiences determined the length of the interview. The interviews lasted approximately 17 to 48 minutes. The interviews were conducted until the researchers felt that no new ideas emerged and data saturation was achieved. Participants were offered an honorarium of CAD $50 (US $37.32) in the form of e-gift cards. Verbal informed consent was obtained before conducting interviews. All interviews were digitally audio-recorded, professionally transcribed, and deidentified. The transcripts were reviewed for accuracy by a research associate.

Reflexivity is crucial in qualitative research, as it enables researchers to position themselves and reflect on the biases, values, and experiences that they bring [38,39]. Recognizing the researchers’ perspectives and positionality, research rigor was asserted by providing a reflexive stance in the research process, including different viewpoints from the team. Seven members of the core research team participated in the coding and analytic process, including 4 research associates from the digital education department at a large multisite academic health sciences center (TJ, SY, MZ, and SB), instructional designer (MC), 2 patient partners (JA and SO), and a senior investigator (DW, a PhD education researcher). This enabled a rigorous interpretation and analysis of the findings. A systematic process outlined by Braun and Clarke [40] was used to inductively analyze the data. Two research associates (TJ and SY) independently analyzed the first 3 transcripts from an exploratory lens and developed an initial coding structure. Each of the remaining transcripts were coded independently by two study team members (TJ and MC, MZ or SB). New data were constantly compared with the existing data, thus resulting in iterative refinement of the coding structure and the structuring of further data collection. Iterative discussions with the research team helped contextualize the overarching themes and resolve disagreements. The senior investigator (DW) on the team reviewed all themes and provided additional input when consensus could not be reached. Two patient partners who were part of the study team (JA and SO) reviewed the themes, which allowed for triangulation of the data from various perspectives. Data were analyzed for emerging themes using NVivo version 12 (QSR International), a qualitative data analysis software program. The rigor and quality of thematic analysis were evaluated using a 20-question evaluation tool [41]. The team also maintained a record of each team member’s coding, notes from meetings, and different versions of the coding structure. This review enhanced the credibility and trustworthiness of the findings. Furthermore, an intercoder agreement was established using NVivo 12 to ensure transparency and rigor of the data.

In total, 12 interviews were conducted between August 2021 and December 2021. Of the 12 participants, 10 (83%) were females, and 2 (17%) were males. Table 1 shows the characteristics of the study participants. The average length of the interviews was 30 minutes. Most participants were very comfortable with the technology and somewhat familiar with AI. Thematic analysis of the data yielded three major themes, each with several subthemes (Table 2): (1) cultivating patients’ trust, (2) fostering patient engagement, and (3) establishing data governance and validation of AI technologies.

As new technologies and AI solutions emerge within health care, it is crucial to ensure that patients are included in the delivery of their own care. Advancements in digital technologies have revolutionized the possibilities of delivering optimal and patient-centric care in this continuously evolving health care ecosystem. Despite the rapid penetration of innovative technologies in clinical care, little is known about the effectiveness of AI technologies. The efficacy and long-term adoption of these technologies depend greatly on patient engagement and adherence [15]. McMahon [42] contended that patient engagement as part of medical education and continuing professional development is crucial in providing an opportunity for HCPs to develop their patient-centric skills, increase sensitivity to patient needs and values, and foster interprofessional collaborative practice. Patient expertise is based on their unique experiences of receiving care and the impact of the social determinants of health. Therefore, it is important to acknowledge and appreciate the value of these diverse patient viewpoints [43,44]. In addition, patient participation is reported to improve care providers’ communication skills and empathy and increase their awareness of patients’ needs in marginalized communities [45-47].

This study aimed to understand patients’ perspectives on how to better foster patient engagement in the uptake of AI technologies and what competencies they believe are essential in preparing HCPs for digital care. Through semistructured interviews with patient partners, three predominant themes emerged: (1) cultivating patients’ trust, (2) fostering patient engagement, and (3) establishing data governance and validation of AI technologies. Participants in both urban and rural settings highlighted similar ideas with regard to AI adoption.

In a recent scoping review, Charow et al [34] identified key competencies that are currently taught as part of the AI curriculum and what programs should be taught. The authors used Bloom Learning Taxonomy to group curriculum topics [34]. Table 3 illustrates the overlap of competencies identified in the scoping review and highlighted by the participants in this study.

As technologies are being integrated within care settings, participants in this study emphasized that it is important for HCPs to acknowledge how data are acquired and processed and explain a rationale when making decisions. Interestingly, the psychomotor and affective domains of Bloom Learning Taxonomy were reiterated by participants. Critical appraisal, ethical and legal considerations, communication, interpersonal skills, empathy, compassion, and emotional responsiveness were highlighted as important competencies to minimize the negative implications of AI integration at the point of care.

This study highlights the importance of establishing trust and transparency as part of the patient-clinician relationship. Many participants stated that lack of transparency in data access and use could potentially erode their trust in using AI for care delivery. This was in line with a previous study [30], which suggested that physicians must have a thorough knowledge of the AI technologies used and be prepared to provide a coherent rationale when making clinical decisions. For instance, if a patient is diagnosed with cancer, they would want to understand how AI technology arrived at that decision [48]. What becomes a challenge, however, is that advanced AI technologies are often built using complex algorithms, which may be difficult to explain, even if clinicians have the technical expertise [48]. In a qualitative study that examined patient privacy perspectives on health information exchange, trust was identified as a key antecedent for establishing effective patient-clinician relationships [49]. Transparent communication regarding the use of AI technologies serves as an initial step toward cultivating trust [49]. The authors noted a significant association between patients’ trust in clinicians and their willingness to share personal health information [49].

Patients believe the clinician’s presence is important, particularly when discussing sensitive information regarding their care. AI technologies should support existing patient care and not replace physician interactions. Similar to our study, previous research indicated that patients valued the interaction with the clinician rather than with AI technology alone [29]. AI technologies can potentially diminish clinician-patient interactions and jeopardize the humanistic facet of patient care [15,50]. Patients who interacted only with AI technologies in their care reported a lack of compassion and empathy [19,21,22] and a limited opportunity for patients to ask follow-up questions, discuss treatment options, and receive emotional support [19,21]. Davenport and Kalakota [48] further reinforced this point, highlighting the importance of establishing an empathetic relationship between clinicians and patients. In other studies, patients specified that the AI output should be verified by the physician for accuracy [22] and be used as a second opinion to inform clinical decisions [9,19]. In the event of a disagreement between the physician and the AI technology, patients favor the physician’s judgment as the final decision [9,22]. Yang et al [21] reported that AI can serve as a copilot in automating tasks and optimizing the quality of care. More importantly, the literature emphasizes the role of providers in decision-making, as they need to adapt the AI results based on the uniqueness of each patient and their circumstances [9].

Engaging patients in proactive care leads to better patient experience and improved health system outcomes [48]. The findings from this study suggest that education on AI innovations helps to create awareness and foster confidence among patients. As a result, patients’ self-efficacy increases, enabling them to be knowledgeable and competent in safely navigating a digitized health care environment. This also contributes to the increased acceptance of AI technologies in practical settings to enhance the quality of care. Recent studies on patient perspectives on the use of AI in health care reported that it is critical for patients to be educated on the threats of AI technologies in an ever-increasing technology-enabled care environment [50,51]. Cultivating a strong culture of cybervigilance across this new digital space is vital for delivering care and ensuring that large amounts of sensitive and valuable data in vulnerable systems are protected. Moreover, Kovarik [52] reported that patients should be educated on the fundamentals of AI, which will be valuable when discussing diagnoses and treatment options.

Furthermore, the findings of this study underline the need for data stewards and regulations to ensure the protection and confidentiality of patient data. Consistent with previous literature, patients reported high levels of concern toward the misuse of their personal health information [15,48,51]. Patients in this study also expressed privacy concerns, such as how their health data would be used, how their data would be shared, and for what purposes. This ambivalence has resulted in increased fear among patients, and the need for choice and autonomy. Participants stated that it was important to have a choice in terms of consenting to what information they would prefer to opt-in or opt-out for secondary use of data. In a review article on the practical implementation of AI technologies, the authors asserted that cybersecurity measures need to be implemented to address concerns about the inappropriate use of patient data [53]. A few studies have reported that patients feared that their personal health information might be not anonymized or be used for profit by insurance and third-party companies [15,50]. In one study, patients perceived that insurance companies could use AI technologies to discern new information about their health and make changes to their premiums [9].

Oversight and regulatory measures are necessary to ensure the confidentiality of patient data and to protect against nefarious acts [9]. The AI implementation toolkit developed by Canada Health Infoway provides guidance on an AI governance framework [54]. This framework consists of 3 key constructs that oversee the responsible and ethical implementation of AI technologies: people, policies, and procedures [54]. The people construct consists of skillsets required to form a committee that provides procedural and practical guidance for AI implementation [54]. Policies focus on providing directions for risk considerations related to AI [54]. Procedures provide operational guidance on implementation aspects, including risk assessment, data testing, and monitoring [54]. Establishing governance structures is pivotal in monitoring ethical issues and mitigating any negative repercussions as a result of AI implementation in a milieu of increasing vulnerability to data breaches [48]. Matheny et al [3] delineated that it is imperative to involve patients and their families when developing regulatory and legislative solutions regarding the use of AI technologies in clinical contexts.

Finally, the participants noted the importance of examining implicit biases to ensure that AI technologies are inclusive and equitable. Biases in data sets may pose challenges in generalizing results and further exacerbate health inequities as well as discriminatory practices. This point was reinforced in a nominal group technique study that emphasized the negative implications of using homogenous data sets for developing algorithms [23]. One example of this is when AI models are developed based on data from a single health care institution, which may not be representative of a larger population [55]. The literature also reports that developers could inadvertently integrate their biases into the model development process [9]. Daneshjou et al [56] noted that there are no standards for describing data sets used for AI model development. Descriptions of data sets could aid in a better understanding of models and any underlying biases. Interestingly, our study also accentuated the notion of using AI technologies to reduce bias from a patient perspective. In health care, clinicians sometimes have preconceived notions about their patients; hence, a patient may not receive the same care as they would if they did not have that notation in their health records. Participants believed that AI technologies could remove some of the preconceived ideas and perceptions that contribute to the marginalization of specific populations when providing care, thus creating a more equitable and inclusive care environment.

The findings of this study should be examined in light of these limitations. A limitation of this study is that the study population included no individuals in the age range of 0 to 40 years. Despite the less frequent use of health care services in this age group, they may represent a more technology-savvy population. This study provides diverse perspectives from rural and urban settings in Canada, as context plays a pivotal role in influencing the uptake of technology. This study provides a nuanced understanding of patient perceptions in both settings and how their perceptions may be similar. The interviews were conducted until theoretical saturation was achieved (n=12). In addition, a rigorous analytical approach was adopted, including iterative discussions with the research team and patient partners to validate emerging themes. Another limitation of this study was the recruitment of predominantly female patients, contributing to an underrepresentation of male voices. Demographic data such as race, ethnicity, employment, disability, and language were not collected, as the purposive sampling attempted to recruit participants based on comfort with the technology and the contexts in which they received care.

This study revealed that to successfully adopt AI technologies in care settings, it is crucial to foster patient trust, build continued partnerships with patients, and establish data governance and validation of AI technologies. As we shift to a digital form of care, AI innovations are being rapidly adopted and implemented within the clinical ecosystem at a fast pace to advance the delivery of patient care and enhance efficiency at a systems level. Rather than AI becoming a replacement for humanistic care, AI and care providers play a synergetic role in the future of digital care. Understanding the needs and values of patients helps ensure the safe, effective, and responsible use of AI. Patient engagement helps to provide a real-world perspective and coconstruct knowledge from an end-user standpoint, thus ensuring that AI innovations are successfully integrated into practice settings. The findings of this study have implications for all stakeholders with accountability to ensure that patients are actively engaged in sustaining safe and high-quality care.