Artificial intelligence (AI) is transforming health care, particularly in the realm of clinical documentation and workflow optimization. AI-powered tools that transcribe patient interactions and generate structured medical notes have demonstrated potential for reducing administrative burden among health care providers [1,2]. Physicians currently spend an average of 2 hours daily on computer-based documentation, a major contributor to burnout [3].

While AI documentation tools have been studied among residents and attending physicians, little is known about their adoption, usability, and perceived value among medical students. This represents a key knowledge gap, as students’ documentation needs, educational priorities, and workflow constraints differ significantly from those of practicing clinicians. We hypothesized that medical students would report positive experiences with AI documentation assistance, particularly regarding perceived documentation burden.

The medical field has long incorporated dictation systems to reduce manual note-writing time, and AI now enables conversion of dictated content into structured, physician-style documentation. Some tools, such as CarePilot, also predict medical coding from recorded encounters. Although research suggests AI will likely become a routine part of multiple medical specialties [4], the extent to which medical students will adopt and benefit from these systems remains unclear. Existing literature includes a mix of opinion pieces, preliminary studies, and peer-reviewed trials [5-7], but few have systematically examined the student population in a controlled setting.

Recent work has shown that AI-assisted clinical documentation can improve efficiency, accuracy, and even suggest clinical phenotypes and diagnoses [4]. However, generalizing these findings to medical students requires caution. Students operate in supervised educational contexts with lower patient loads and different performance expectations, making their experiences with AI documentation tools potentially distinct from those of fully licensed physicians.

This study aims to address this gap by assessing initial usability and satisfaction with an AI documentation tool among third- and fourth-year medical students in a controlled, simulation-based environment. This exploratory evaluation is not intended to determine the tool’s effectiveness in real clinical practice but rather to establish a baseline understanding of student perceptions and identify potential facilitators and barriers to adoption.

In this simulation-based exploratory study, medical students reported overall positive perceptions of the CarePilot system, particularly regarding ease of use, learnability, and documentation efficiency. However, some respondents noted challenges related to workflow disturbance and functional expectations, highlighting areas for potential improvement. While these findings suggest that CarePilot can support efficient note generation and reduce cognitive effort in an educational simulation, the extent to which these benefits will translate to real-world practice remains uncertain and should be investigated further rather than interpreted as definitive outcomes.

Several studies have shown that AI-powered documentation tools can improve workflow efficiency and reduce administrative burden in active clinical settings. Previous studies have reported significant reductions in documentation time and after-hours charting in multisite clinical trials [3]. Others found that ambient AI scribes shortened note completion times without compromising quality [9]. Shah et al [10] and Alboksmaty et al [11] further demonstrated improvements in clinician satisfaction and reductions in burnout scores in primary care and outpatient contexts. Our results are consistent with these observations in that users reported high satisfaction with efficiency and usability. However, unlike those real-world studies, our work was conducted in a simulation with medical students who do not face the same time pressures, billing requirements, or patient volumes as practicing clinicians. Because medical students do not shoulder full clinical responsibility, their perceptions may not generalize to practicing clinicians. This difference in user profile and environment should temper direct comparisons, though the strong usability signals we observed suggest potential for similar benefits if implemented with appropriate adaptation.

Not all evaluations of AI documentation systems have been uniformly positive. This pattern is consistent with prior work showing that although clinicians value generative AI for improving documentation efficiency, they remain cautious about reliability, automation bias, medico-legal risk, and the need for continued clinician supervision [12]. Even when efficiency gains are realized, challenges such as workflow interruptions, system lag, and imperfect transcription accuracy can undermine adoption [13]. Similar concerns were raised by participants in our study, with approximately 30% reporting neutral or negative overall satisfaction and identifying workflow interruptions as a key limitation. These disturbances may have stemmed from a combination of factors, including occasional system latency, the need for mid-encounter corrections, and an interface design that required shifting attention away from the patient. Identifying and addressing such sources could improve the fit into the natural flow of a clinical encounter. The alignment between these reported barriers in both simulated and real clinical contexts reinforces the importance of optimizing speed, accuracy, and EHR integration before widespread implementation.

Our findings also align with the broader literature on clinician burnout and documentation burden. While AI systems can meaningfully reduce charting time, their success depends on high usability, minimal cognitive load, and robust technical support [2]. In our cohort, the intuitive interface and ease of learning likely contributed to positive perceptions, mirroring the importance of these factors in real-world adoption. However, claims about enhancing the learning experience, reducing cognitive load, or mitigating burnout should be framed as potential benefits that merit further research rather than as direct conclusions from the current study.

In addition to efficiency, AI-assisted documentation has implications for clinical communication and patient engagement. When well-integrated, such systems can free clinicians to spend more time face-to-face with patients [11]. Conversely, poor integration risks distracting from patient interactions. Our results point to a similar tension: while many participants valued the time savings, some noted that the need to monitor and edit the system’s output could detract from patient focus. Future iterations of CarePilot could incorporate features such as real-time summarization prompts or patient-facing summaries to strengthen this aspect while maintaining accuracy under clinician supervision.

Successful adoption will also require targeted training and adaptation to the specific needs of different clinical environments. Studies across diverse specialties have shown that familiarity with the system, iterative user feedback, and integration into authentic workflows are critical for sustained use. Our results, though limited to an educational simulation, underscore these same principles. Additionally, the potential influence of self-selection bias and the Hawthorne effect should be acknowledged, as participants who chose to enroll and knew they were being observed may have reported more favorable impressions.

While our survey was designed through a targeted literature review and expert consultation, it was not directly adapted from established, validated usability instruments such as the System Usability Scale or the technology acceptance model. Instead, items were tailored to capture domains relevant to medical student experiences with AI-assisted documentation, including perceived efficiency, ease of use, and workflow integration. The instrument underwent face and content validation through review by faculty with expertise in medical education and health technology, and was pilot-tested with a small cohort of medical students before implementation. Although this process supported clarity and relevance of items for our specific context, the lack of use of a formally validated instrument may limit comparability with other studies. Future work could incorporate or adapt validated frameworks to enhance psychometric rigor and facilitate cross-study benchmarking.

Additionally, our findings highlight overall positive perceptions of CarePilot’s usability, efficiency, and organization. The interpretation of these results must be considered in light of alternative explanations. Factors such as prior exposure to EHR systems, individual comfort with technology, and simulation-specific conditions (eg, absence of real clinical time pressures, controlled patient presentations) may have influenced participants’ responses. Additionally, we did not collect demographic variables such as gender, age, or socioeconomic status, which could reveal important subgroup differences in perceptions of usability and satisfaction. Incorporating these variables in future research would allow for a more nuanced understanding of how personal and contextual factors influence technology adoption among medical trainees. Addressing these considerations will help ensure that implementation strategies are tailored to diverse user needs and training environments.

This study has several limitations that should be considered when interpreting the findings. It was conducted in a simulated clinical setting, which may not fully reflect the complexities and pressures of real-world environments. The controlled nature of the simulation and the use of medical students may limit generalizability to practicing clinicians. The sample size was relatively small and drawn from a single institution, further constraining external validity. Moreover, self-selection bias and the Hawthorne effect may have influenced responses.

Despite these limitations, our findings suggest that AI-assisted documentation tools such as CarePilot hold promise for improving usability and efficiency, with potential downstream benefits for clinician workflow, education, and patient engagement. These potential benefits should be validated in larger, more diverse, real-world studies. Future research should explore integration with existing EHR platforms, enhance real-time data capture, minimize workflow disturbances, and assess long-term effects on clinician efficiency, burnout, and patient outcomes. Addressing these challenges will be essential to ensuring that AI-driven documentation solutions contribute positively to clinical practice while preserving the quality of physician-patient interactions.