TY - JOUR AU - Perakslis, Eric AU - Nolen, Kimberly AU - Fricklas, Ethan AU - Tubb, Tracy PY - 2025/4/7 TI - Striking a Balance: Innovation, Equity, and Consistency in AI Health Technologies JO - JMIR AI SP - e57421 VL - 4 KW - artificial intelligence KW - algorithm KW - regulatory landscape KW - predictive model KW - predictive analytics KW - predictive system KW - practical model KW - machine learning KW - large language model KW - natural language processing KW - deep learning KW - digital health KW - regulatory KW - health technology UR - https://ai.jmir.org/2025/1/e57421 UR - http://dx.doi.org/10.2196/57421 ID - info:doi/10.2196/57421 ER - TY - JOUR AU - Levites Strekalova, A. Yulia AU - Liu-Galvin, Rachel AU - Border, Samuel AU - Midence, Sara AU - Khan, Mishal AU - VanZanten, Maya AU - Tomaszewski, John AU - Jain, Sanjay AU - Sarder, Pinaki PY - 2025/2/3 TI - Summer Research Internship Curriculum to Promote Self-Efficacy, Researcher Identity, and Peer-to-Peer Learning: Retrospective Cohort Study JO - JMIR Form Res SP - e54167 VL - 9 KW - artificial intelligence KW - biomedical research KW - curriculum KW - training programs KW - workforce N2 - Background: Common barriers to students? persistence in research include experiencing feelings of exclusion and a lack of belonging, difficulties developing a robust researcher identity, perceptions of racial and social stigma directed toward them, and perceived gaps in research skills, which are particularly pronounced among trainees from groups traditionally underrepresented in research. To address these known barriers, summer research programs have been shown to increase the participation and retention of undergraduate students in research. However, previous programs have focused predominantly on technical knowledge and skills, without integrating an academic enrichment curriculum that promotes professional development by improving students? academic and research communication skills. Objective: This retrospective pre-then-post study aimed to evaluate changes in self-reported ratings of research abilities among a cohort of undergraduate students who participated in a summer research program. Methods: The Human BioMolecular Atlas Program (HuBMAP) piloted the implementation of a web-based academic enrichment curriculum for the Summer 2023 Research Internship cohort, which was comprised of students from groups underrepresented in biomedical artificial intelligence research. HuBMAP, a 400-member research consortium funded by the Common Fund at the National Institutes of Health, offered a 10-week summer research internship that included an academic enrichment curriculum delivered synchronously via the web to all students across multiple sites. The curriculum is intended to support intern self-efficacy, researcher identity development, and peer-to-peer learning. At the end of the internship, students were invited to participate in a web-based survey in which they were asked to rate their academic and research abilities before the internship and as a result of the internship using a modified Entering Research Learning Assessment instrument. Wilcoxon matched-pairs signed rank test was performed to assess the difference in the mean scores per respondent before and after participating in the internship. Results: A total of 14 of the 22 undergraduate students who participated in the internship responded to the survey. The results of the retrospective pre-then-post survey indicated that there was a significant increase in students? self-rated research abilities, evidenced by a significant improvement in the mean scores of the respondents when comparing reported skills self-assessment before and after the internship (improvement: median 1.09, IQR 0.88-1.65; W=52.5, P<.001). After participating in the HuBMAP web-based academic enrichment curriculum, students? self-reported research abilities, including their confidence, their communication and collaboration skills, their self-efficacy in research, and their abilities to set research career goals, increased. Conclusions: Summer internship programs can incorporate an academic enrichment curriculum with small-group peer learning in addition to a laboratory-based experience to facilitate increased student engagement, self-efficacy, and a sense of belonging in the research community. Future research should investigate the impact of academic enrichment curricula and peer mentoring on the long-term retention of students in biomedical research careers, particularly retention of students underrepresented in biomedical fields. UR - https://formative.jmir.org/2025/1/e54167 UR - http://dx.doi.org/10.2196/54167 ID - info:doi/10.2196/54167 ER - TY - JOUR AU - Cresswell, Kathrin AU - de Keizer, Nicolette AU - Magrabi, Farah AU - Williams, Robin AU - Rigby, Michael AU - Prgomet, Mirela AU - Kukhareva, Polina AU - Wong, Shui-Yee Zoie AU - Scott, Philip AU - Craven, K. Catherine AU - Georgiou, Andrew AU - Medlock, Stephanie AU - Brender McNair, Jytte AU - Ammenwerth, Elske PY - 2024/8/7 TI - Evaluating Artificial Intelligence in Clinical Settings?Let Us Not Reinvent the Wheel JO - J Med Internet Res SP - e46407 VL - 26 KW - artificial intelligence KW - evaluation KW - theory KW - patient safety KW - optimisation KW - health care KW - optimization UR - https://www.jmir.org/2024/1/e46407 UR - http://dx.doi.org/10.2196/46407 UR - http://www.ncbi.nlm.nih.gov/pubmed/39110494 ID - info:doi/10.2196/46407 ER - TY - JOUR AU - Burns, Christina AU - Bakaj, Angela AU - Berishaj, Amonda AU - Hristidis, Vagelis AU - Deak, Pamela AU - Equils, Ozlem PY - 2024/8/6 TI - Use of Generative AI for Improving Health Literacy in Reproductive Health: Case Study JO - JMIR Form Res SP - e59434 VL - 8 KW - ChatGPT KW - chat-GPT KW - chatbots KW - chat-bot KW - chat-bots KW - artificial intelligence KW - AI KW - machine learning KW - ML KW - large language model KW - large language models KW - LLM KW - LLMs KW - natural language processing KW - NLP KW - deep learning KW - chatbot KW - Google Search KW - internet KW - communication KW - English proficiency KW - readability KW - health literacy KW - health information KW - health education KW - health related questions KW - health information seeking KW - health access KW - reproductive health KW - oral contraceptive KW - birth control KW - emergency contraceptive KW - comparison KW - clinical KW - patients N2 - Background: Patients find technology tools to be more approachable for seeking sensitive health-related information, such as reproductive health information. The inventive conversational ability of artificial intelligence (AI) chatbots, such as ChatGPT (OpenAI Inc), offers a potential means for patients to effectively locate answers to their health-related questions digitally. Objective: A pilot study was conducted to compare the novel ChatGPT with the existing Google Search technology for their ability to offer accurate, effective, and current information regarding proceeding action after missing a dose of oral contraceptive pill. Methods: A sequence of 11 questions, mimicking a patient inquiring about the action to take after missing a dose of an oral contraceptive pill, were input into ChatGPT as a cascade, given the conversational ability of ChatGPT. The questions were input into 4 different ChatGPT accounts, with the account holders being of various demographics, to evaluate potential differences and biases in the responses given to different account holders. The leading question, ?what should I do if I missed a day of my oral contraception birth control?? alone was then input into Google Search, given its nonconversational nature. The results from the ChatGPT questions and the Google Search results for the leading question were evaluated on their readability, accuracy, and effective delivery of information. Results: The ChatGPT results were determined to be at an overall higher-grade reading level, with a longer reading duration, less accurate, less current, and with a less effective delivery of information. In contrast, the Google Search resulting answer box and snippets were at a lower-grade reading level, shorter reading duration, more current, able to reference the origin of the information (transparent), and provided the information in various formats in addition to text. Conclusions: ChatGPT has room for improvement in accuracy, transparency, recency, and reliability before it can equitably be implemented into health care information delivery and provide the potential benefits it poses. However, AI may be used as a tool for providers to educate their patients in preferred, creative, and efficient ways, such as using AI to generate accessible short educational videos from health care provider-vetted information. Larger studies representing a diverse group of users are needed. UR - https://formative.jmir.org/2024/1/e59434 UR - http://dx.doi.org/10.2196/59434 UR - http://www.ncbi.nlm.nih.gov/pubmed/38986153 ID - info:doi/10.2196/59434 ER - TY - JOUR AU - Narang, Gaurav AU - Chen, J. Yaozhu AU - Wedel, Nicole AU - Wu, Melody AU - Luo, Michelle AU - Atreja, Ashish PY - 2024/6/6 TI - Development of a Digital Patient Assistant for the Management of Cyclic Vomiting Syndrome: Patient-Centric Design Study JO - JMIR Form Res SP - e52251 VL - 8 KW - cyclic vomiting syndrome KW - vomiting KW - vomit KW - emetic KW - emesis KW - gut KW - GI KW - gastrointestinal KW - internal medicine KW - prototype KW - prototypes KW - iterative KW - self-management KW - disease management KW - gut-brain interaction KW - gut-brain KW - artificial intelligence KW - digital patient assistant KW - assistant KW - assistants KW - design thinking KW - design KW - patient-centric KW - patient centred KW - patient centered KW - patient-centric approach KW - System Usability Scale KW - symptom tracking KW - digital health solution KW - user experience KW - usability KW - symptom KW - symptoms KW - tracking KW - monitoring KW - participatory KW - co-design digital health technology KW - patient assistance KW - patient experience KW - mobile phone N2 - Background: Cyclic vomiting syndrome (CVS) is an enigmatic and debilitating disorder of gut-brain interaction that is characterized by recurrent episodes of severe vomiting and nausea. It significantly impairs patients? quality of life and can lead to frequent medical visits and substantial health care costs. The diagnosis for CVS is often protracted and complex, primarily due to its exclusionary diagnosis nature and the lack of specific biomarkers. This typically leads to a considerable delay in accurate diagnosis, contributing to increased patient morbidity. Additionally, the absence of approved therapies for CVS worsens patient hardship and reflects the urgent need for innovative, patient-centric solutions to improve CVS management. Objective: We aim to develop a digital patient assistant (DPA) for patients with CVS to address their unique needs, and iteratively enhance the technical features and user experience on the initial DPA versions. Methods: The development of the DPA for CVS used a design thinking approach, prioritizing user needs. A literature review and Patient Advisory Board shaped the initial prototype, focusing on diagnostic support and symptom tracking. Iterative development, informed by the design thinking approach and feedback from patients with CVS and caregivers through interviews and smartphone testing, led to significant enhancements in user interaction and artificial intelligence integration. The final DPA?s effectiveness was validated using the System Usability Scale and feedback questions, ensuring it met the specific needs of the CVS community. Results: The DPA developed for CVS integrates an introductory bot, daily and weekly check-in bots, and a knowledge hub, all accessible via a patient dashboard. This multicomponent solution effectively addresses key unmet needs in CVS management: efficient symptom and impacts tracking, access to comprehensive disease information, and a digital health platform for disease management. Significant improvements, based on user feedback, include the implementation of artificial intelligence features like intent recognition and data syncing, enhancing the bot interaction and reducing the burden on patients. The inclusion of the knowledge hub provides educational resources, contributing to better disease understanding and management. The DPA achieved a System Usability Scale score of 80 out of 100, indicating high ease of use and relevance. Patient feedback highlighted the DPA?s potential in disease management and suggested further applications, such as integration into health care provider recommendations for patients with suspected or confirmed CVS. This positive response underscores the DPA?s role in enhancing patient engagement and disease management through a patient-centered digital solution. Conclusions: The development of this DPA for patients with CVS, via an iterative design thinking approach, offers a patient-centric solution for disease management. The DPA development framework may also serve to guide future patient digital support and research scenarios. UR - https://formative.jmir.org/2024/1/e52251 UR - http://dx.doi.org/10.2196/52251 UR - http://www.ncbi.nlm.nih.gov/pubmed/38842924 ID - info:doi/10.2196/52251 ER - TY - JOUR AU - Herter, Ernst Willem AU - Khuc, Janine AU - Cinà, Giovanni AU - Knottnerus, J. Bart AU - Numans, E. Mattijs AU - Wiewel, A. Maryse AU - Bonten, N. Tobias AU - de Bruin, P. Daan AU - van Esch, Thamar AU - Chavannes, H. Niels AU - Verheij, A. Robert PY - 2022/5/4 TI - Impact of a Machine Learning?Based Decision Support System for Urinary Tract Infections: Prospective Observational Study in 36 Primary Care Practices JO - JMIR Med Inform SP - e27795 VL - 10 IS - 5 KW - machine learning KW - ML KW - artificial intelligence KW - clinical decision support system KW - implementation study KW - information technology KW - urinary tract infections N2 - Background: There is increasing attention on machine learning (ML)-based clinical decision support systems (CDSS), but their added value and pitfalls are very rarely evaluated in clinical practice. We implemented a CDSS to aid general practitioners (GPs) in treating patients with urinary tract infections (UTIs), which are a significant health burden worldwide. Objective: This study aims to prospectively assess the impact of this CDSS on treatment success and change in antibiotic prescription behavior of the physician. In doing so, we hope to identify drivers and obstacles that positively impact the quality of health care practice with ML. Methods: The CDSS was developed by Pacmed, Nivel, and Leiden University Medical Center (LUMC). The CDSS presents the expected outcomes of treatments, using interpretable decision trees as ML classifiers. Treatment success was defined as a subsequent period of 28 days during which no new antibiotic treatment for UTI was needed. In this prospective observational study, 36 primary care practices used the software for 4 months. Furthermore, 29 control practices were identified using propensity score-matching. All analyses were performed using electronic health records from the Nivel Primary Care Database. Patients for whom the software was used were identified in the Nivel database by sequential matching using CDSS use data. We compared the proportion of successful treatments before and during the study within the treatment arm. The same analysis was performed for the control practices and the patient subgroup the software was definitely used for. All analyses, including that of physicians? prescription behavior, were statistically tested using 2-sided z tests with an ? level of .05. Results: In the treatment practices, 4998 observations were included before and 3422 observations (of 2423 unique patients) were included during the implementation period. In the control practices, 5044 observations were included before and 3360 observations were included during the implementation period. The proportion of successful treatments increased significantly from 75% to 80% in treatment practices (z=5.47, P<.001). No significant difference was detected in control practices (76% before and 76% during the pilot, z=0.02; P=.98). Of the 2423 patients, we identified 734 (30.29%) in the CDSS use database in the Nivel database. For these patients, the proportion of successful treatments during the study was 83%?a statistically significant difference, with 75% of successful treatments before the study in the treatment practices (z=4.95; P<.001). Conclusions: The introduction of the CDSS as an intervention in the 36 treatment practices was associated with a statistically significant improvement in treatment success. We excluded temporal effects and validated the results with the subgroup analysis in patients for whom we were certain that the software was used. This study shows important strengths and points of attention for the development and implementation of an ML-based CDSS in clinical practice. Trial Registration: ClinicalTrials.gov NCT04408976; https://clinicaltrials.gov/ct2/show/NCT04408976 UR - https://medinform.jmir.org/2022/5/e27795 UR - http://dx.doi.org/10.2196/27795 UR - http://www.ncbi.nlm.nih.gov/pubmed/35507396 ID - info:doi/10.2196/27795 ER -