Current Landscape of Generative AI Use as a Search Engine Among Resident Physicians: Cross-Sectional Study

Introduction

In recent years, generative AI (GenAI) has gained significant attention and has been increasingly integrated into clinical care. GenAI is expected to serve as an “interactive encyclopedia,” supporting clinical decisions by providing disease information, listing differential diagnoses, and suggesting treatment options at a level comparable to those of physicians [1-4]. However, its inherent limitations, stemming from its stochastic nature and training data, include confabulations and biases [5]. These limitations are compounded by human cognitive bias toward GenAI outputs [2,6].

Concerns around knowledge collapse and inadequate skill development (eg, deskilling, never-skilling, and mis-skilling) among physicians who overrely on GenAI are also emerging [2,7,8]. They are theoretically caused by cognitive off-loading and have already been documented in endoscopists [9]. We hypothesized that a similar phenomenon may be observed in other competencies, including those that remain indispensable even when using GenAI. For example, clinical reasoning grounded in pathophysiological knowledge is essential for identifying confabulations [8,10]. Similarly, considering local disease epidemiology is crucial to counter biases embedded in large databases [6]. Moreover, thorough history taking and physical examination skills are unlikely to be replaced by GenAI [11,12]. The importance of these competencies is particularly true for infectious diseases (IDs), where bedside evaluation and awareness of local pathogen epidemiology and antimicrobial resistance are central to diagnosis and treatment but are insufficiently represented in current GenAI models [6].

Resident physicians’ recognition of these drawbacks is of paramount importance. First, as digital natives, they may be less critical of advanced technologies [13]. Second, with limited clinical experience, they may struggle to evaluate the accuracy of GenAI outputs [13]. Third, their clinical performance has enduring implications for patient care. These concerns extend to GenAI nonusers, given that the adoption of GenAI in medicine is unlikely to cease [2]. However, the preparedness of nonusers for its future integration remains unclear.

This study aimed to explore the current landscape of (1) the resident physicians’ use of GenAI in clinical practice, (2) awareness of GenAI’s limitations among users and nonusers, and (3) differences in bedside behaviors regarding the management of IDs between the two groups to assess knowledge collapse and inadequate skill development among GenAI users.

Methods

Study Setting, Study Participants, and Data Source

We conducted a cross-sectional study in January 2025 among examinees of the General Medicine In-Training Examination (GM-ITE) in Japan, where the academic year runs from April to March. The GM-ITE is a nationwide, annual computer-based test consisting of 80 multiple-choice clinical questions for postgraduate year (PGY) 1 and PGY 2 resident physicians. It has been administered by the Japan Institute for Advancement of Medical Education Program (JAMEP), a nonprofit organization that evaluates medical education in Japan, since 2011. In 2024, 9580 resident physicians—approximately half of all PGY 1 and PGY 2 residents in Japan—took the examination [14]. GM-ITE data have been widely used in medical education research [15-19].

We developed a 29-item multiple-choice electronic survey addressing GenAI use in daily life outside clinical work, GenAI use as a search engine, and bedside behaviors in ID management (Multimedia Appendix 1), which was disseminated concurrently with the examination. The variable regarding GenAI use in daily life was collected to provide contextual information on study participants. Participation was voluntary, and respondents received an online reference list of common pathogens and antimicrobials, which did not contain GenAI-related content, as an incentive. All GM-ITE examinees who responded to our survey and consented to the use of their responses for research purposes were included in our study. For the comparison between resident physicians with and without GenAI use as a search engine, we analyzed responses only from individuals who responded to the relevant item.

In Japan, 2-year residency programs start in April and end in March. They are mandatory for all medical school graduates seeking clinical positions, regardless of their desired specialty. An ID department rotation is elective, whereas an antimicrobial stewardship program is not included [15].

Questionnaire Development

GenAI use as a search engine in daily life and in clinical work was assessed using separate survey items. We designed the survey to evaluate respondents’ GenAI literacy using Bloom’s taxonomy of learning domains, a framework commonly applied in medical education [20]. The framework comprises the psychomotor (eg, practical skills), cognitive (eg, knowledge), and affective (eg, attitudes or medical ethics) domains. In the psychomotor domain, we investigated whether a respondent used GenAI to gain information about the disease of interest or to help enumerate differential diagnoses. The cognitive domain examined perceived GenAI literacy, including confabulation [6,21], biased output [6], automation bias [2,6], physicians’ knowledge collapse and deskilling [2,8], and uninformed use of GenAI in patient care [22], as a surrogate for true GenAI literacy. It also addressed competencies considered essential regardless of GenAI, such as history taking, physical examination [11,12], pathophysiological knowledge, and clinical reasoning [8,10]. The affective domain adopted the FAVES (fairness, appropriateness, validity, effectiveness, and safety) framework and transparency, an expansion of the Belmont principles (ie, beneficence, respect for people, and justice), to assess ethical considerations of GenAI [22].

Furthermore, we also included items on bedside behaviors crucial to ID management and likely to remain essential, including history taking, physical examination, clinical reasoning, recognition of local epidemiology (ie, epidemics and antibiogram data), and individualization of care goals [6,8,10-12]. We piloted the survey among resident physicians at the University of Tokyo Hospital to evaluate clarity and feasibility. After administration of the GM-ITE, demographic information and survey responses were collected.

Statistical Analysis

In statistical hypothesis testing, responses on a 5-point Likert scale were dichotomized: scores 1 to 3 (strongly disagree, disagree, and neutral) were categorized as “not applicable” and scores 4 to 5 (agree and strongly agree) as “applicable” and then summarized as proportions, including the items assessing GenAI use as a search engine and bedside behaviors. Associations between GenAI use and respondents’ demographics and cognitive and affective domains were assessed using univariable chi-square tests.

Our previous study suggested the potential independent effect of an ID department rotation and the presence of antimicrobial stewardship programs on the performance of resident physicians in managing IDs [15]. Therefore, this study used multivariable logistic regression to examine associations between GenAI use (independent variable) and bedside behaviors (dependent variables), adjusting for these two factors, alongside sex, PGY, institution type (ie, university or community hospital), and overall GM-ITE score.

We performed a sensitivity analysis on the cutoff of item responses, wherein 5-point Likert scale items were dichotomized into scores 1 to 2 and scores 3 to 5. This shift also applied to the cutoff used to distinguish between GenAI users and nonusers. Statistical significance was defined as a 2-tailed P<.05. Throughout the statistical testing, complete-case analyses were conducted because the proportion of missing values remained below 5% [23]. Analyses were conducted using Stata (version 16; StataCorp LLC).

Ethical Considerations

This study was approved by the Institutional Review Board of JAMEP (number 24‐20). All survey responses were examined after obtaining informed consent from the study participants. The study data were deidentified, and no identification of individual participants is possible in any portion of the manuscript or supplementary material. The study adhered to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines.

Results

Characteristics of Study Participants

Of 9179 GM-ITE participants, 2989 (32.6%) resident physicians from 546 hospitals completed the survey, including 1002 (33.5%) female respondents. Respondents comprised 1506 (50.4%) PGY 1 and 1483 (49.6%) PGY 2, with 2701 (90.4%) aged <30 years. Most respondents (n=2401, 80.3%) worked in community hospitals. Overall, 1948 (68.4%) reported using GenAI in daily life outside clinical work.

After excluding 139 (4.3%) resident physicians who did not respond to the item regarding GenAI use as a search engine, the remaining 2850 respondents were included in subsequent analyses. Differences between GenAI users (n=1124, 39.4%) and nonusers (n=1726, 60.6%) are summarized in Table 1. On the basis of the proportion of GenAI users among each sex, female respondents were less likely than male respondents to use GenAI as a search engine (350/963, 36.3% vs 774/1887, 41%; P=.02). GenAI users relied on self-study more frequently than nonusers (759/1118, 67.9% vs. 981/1706, 57.5%; P<.001). No statistically significant differences were observed in the proportion of PGY 1 residents (P=.72), individuals aged <30 years (P=.40), or individuals working at a university hospital (P=.84).

Psychomotor Domain of GenAI Use Among Resident Physicians

Among all respondents (n=2850), 344 (12.1%) reported receiving training in GenAI use as a search engine. GenAI was infrequently used as the primary educational resource: 13% (144/1110) of respondents used it for differential diagnoses, 10.5% (117/1112) for unfamiliar diseases, and 8.7% (97/1111) for familiar diseases. Even among GenAI users, printed or electronic textbooks remained the predominant educational resource: 54.1% (600/1110) for differential diagnoses, 53.3% (592/1111) for familiar diseases, and 37.7% (419/1112) for unfamiliar diseases.

Cognitive Domain of GenAI Use Among Resident Physicians

The distribution of responses for each item on 5-category scales was illustrated in Multimedia Appendix 2. GenAI’s potential for confabulation and bias was recognized by 64.6% (1808/2799) and 60.2% (1679/2790) of respondents, respectively, while only 48.8% (1356/2777) acknowledged patients’ right to be informed of GenAI use. Awareness was consistently higher among GenAI users than nonusers (Table 2 and Figure 1): confabulation (817/1103, 74.1% vs 991/1696, 58.4%; P<.001), biased output (769/1101, 69.8% vs 910/1689, 53.9%; P<.001), automation bias (743/1099, 67.6% vs 885/1690, 52.4%; P<.001), knowledge collapse (664/1099, 60.4% vs 905/1683, 53.8%; P=.001), and informing patients of GenAI use (578/1102, 52.5% vs 778/1675, 46.4%; P=.002).

‎

Similarly, users more often recognized essential competencies for ID management: history taking and physical examination (965/1106, 87.3% vs 1296/1706, 76%; P<.001), pathophysiological knowledge (958/1102, 86.9% vs 1286/1697, 75.8%; P<.001), and clinical reasoning (951/1103, 86.2% vs 1301/1692, 76.9%; P<.001).

Affective Domain of GenAI Use Among Resident Physicians

Overall, 57.3% (1584/2765) to 60.1% (1663/2768) of respondents assessed GenAI for effectiveness, safety, applicability, and validity, while 47.3% (1311/2770) and 49.8% (1381/2774) considered fairness and transparency, respectively. Across all domains, users showed greater ethical awareness than nonusers: effectiveness (764/1092, 70% vs 843/1680, 50.2%; P<.001), safety (755/1095, 68.9% vs 892/1681, 53.1%; P<.001), fairness (615/1092, 56.3% vs 696/1678, 41.5%, P<.001), applicability (740/1091, 67.8% vs 844/1674, 50.4%; P<.001), validity (772/1092, 70.7% vs 891/1676, 53.1%; P<.001), and transparency (645/1095, 58.9% vs 736/1679, 43.8%; P<.001).

Association Between GenAI Use and the Management of ID

GenAI use was independently associated with favorable behaviors (Table 3): thorough patient examination (adjusted odds ratio [aOR] 1.42, 95% CI 1.18‐1.70), appropriate antimicrobial selection (aOR 1.57, 95% CI 1.32‐1.87), familiarity with current epidemic information (aOR 1.46, 95% CI 1.22‐1.75), local antibiogram use (aOR 1.62, 95% CI 1.38‐1.89), individualized goal setting (aOR 1.64, 95% CI 1.40‐1.92), and individualized treatment strategy (aOR 1.45, 95% CI 1.23‐1.71).

Sensitivity Analysis

A sensitivity analysis using an alternative cutoff for dichotomization of item responses was performed, which largely demonstrated consistency with the primary analysis regarding the trends of demographics, the 3 domains of GenAI, and the association between GenAI use and bedside behaviors. Statistically significant differences diminished in only 3 items: concerns over potential knowledge collapse and inadequate skill development, awareness of the necessity for transparency, and acknowledgment of the importance of clinical reasoning ability (Multimedia Appendices 1, 3 and 4).

Discussion

Principal Findings

In this nationwide questionnaire survey of resident physicians in Japan conducted in January 2025, 39.4% (1124/2850) of respondents reported using GenAI as a search engine, although textbooks remained their primary educational resource. Perceived GenAI literacy and ethical considerations varied, with nonusers demonstrating lower familiarity. Fewer than half of respondents recognized the importance of disclosing GenAI use to patients. Notably, GenAI use was positively associated with preferred behaviors in the management of IDs.

Comparison to Prior Work

Despite its potential, the penetration of GenAI use as a search engine remained limited, both quantitatively and qualitatively, among resident physicians in Japan as of 2025. Consistent with earlier studies of physicians in training in the United States (2023) [24], Türkiye (2023‐2024) [25], and China (2024) [26], fewer than half of the respondents used GenAI in clinical practice. Furthermore, even among GenAI users, GenAI’s role as a search engine was peripheral, as most of them primarily used other tools. These findings may be at least partially explained by the unavailability of training for the appropriate application of GenAI [27]. Although GenAI offers opportunities for self-study, its output may be unsuitable for learners with differing levels of knowledge to interpret reliably [28]. Accordingly, structured education appears necessary to prepare resident physicians for the integration of GenAI into clinical practice.

Our study showed the systematic differences between GenAI users and nonusers. First, we found a lower likelihood of GenAI use among female resident physicians compared with their male counterparts. A study of physicians in training in the United States suggested a trend of less interest in GenAI among female than male respondents [24]. Probably, in the process of adopting new technology, female physicians prioritize social influence from their peers over its performance expectancy to determine whether to accept it [27]; they may defer their acceptance until GenAI is used by their peers. Second, nonusers consistently exhibited limited perceived GenAI literacy. According to the diffusion of innovations theory, the “early and late majority”—those likely to adopt GenAI in the future—tend to rely more on peer behavior (eg, colleagues’ GenAI use) than on theoretical understanding (eg, GenAI mechanisms), in contrast to “early adopters” (ie, current users) [29].

Our findings highlighted that transparency with patients was among the least considered aspects of GenAI use, although many respondents recognized the limitations of GenAI, such as confabulations and the possibility of biased outputs. The tendency of physicians to avoid disclosing GenAI use to patients has also been observed outside Japan [24] and may conflict with the principle that high-quality care should remain patient-centered [30]. From the patient’s perspective, ensuring transparency and explainability in the clinical decision-making process is critical [23], particularly as patients have begun voicing concerns about potential physician overreliance on GenAI [31,32].

In the present study, our hypothesis of knowledge collapse and inadequate skill development associated with GenAI use among resident physicians was not supported. Although this finding may be reassuring, the association may have been affected by several factors. First, unmeasured GenAI users’ traits, such as inquisitiveness, may have influenced the results. Second, other anticipated roles of GenAI, such as democratizing and accelerating access to context-specific medical knowledge, may have obscured the associations [33,34]. Nevertheless, empirical evidence regarding the long-term impact of GenAI on physicians’ performance remains limited, underscoring the need for further investigation.

Limitations

This study has several limitations. First, the cross-sectional design precludes assessment of causal relationships between GenAI use and physicians’ knowledge, attitudes, or behaviors. The associations among these factors may also have been affected by unmeasured variables, such as learners’ inquisitiveness and diligence, although we tried to mitigate their impact by including the GM-ITE score in our model. Longitudinal studies at multiple time points are required to capture changes and inform the timely development of medical education. Second, nonresponse bias may have influenced the findings, potentially leading to the overrepresentation of GenAI-familiar resident physicians. Although we suggested that GenAI literacy was insufficient among resident physicians, we may have underestimated the severity of the issue. Third, the validity and reliability of our original survey items have yet to be explored, although we conducted a pilot test to enhance face validity. Specifically, we measured perceived GenAI literacy via a survey as a surrogate for actual GenAI literacy, which requires direct observation in clinical settings. Given the possibility that some respondents who misunderstood GenAI literacy reported their awareness, our study findings may have overestimated the level of GenAI literacy among resident physicians. Finally, we analyzed most 5-point Likert scale items after dichotomizing them to improve interpretability. Therefore, we may have overlooked nuanced differences.

Future Directions

Our study findings underscore the need for residency programs to provide structured education on GenAI use in clinical contexts [6]. Such training should address not only effective prompt construction [35] but also the cultivation of critical appraisal skills to avoid premature closure [7,36] and the promotion of ethical considerations, including transparency with patients during patient care, all of which may need to be incorporated into future residency competency frameworks. The approach aligns with the recently proposed DEFT (diagnosis, evidence, feedback, and teaching)–AI framework for improving the quality of bedside GenAI education [7].

Conclusions

Resident physicians in Japan currently use GenAI as a search engine to a limited extent. Their literacy regarding GenAI remains inadequate, particularly among nonusers.