TY  - JOUR
AU  - Khademi, Sedigh
AU  - Palmer, Christopher
AU  - Javed, Muhammad
AU  - Dimaguila, Gerardo Luis
AU  - Clothier, Hazel
AU  - Buttery, Jim
AU  - Black, Jim
PY  - 2024
DA  - 2024/8/30
TI  - Near Real-Time Syndromic Surveillance of Emergency Department Triage Texts Using Natural Language Processing: Case Study in Febrile Convulsion Detection
JO  - JMIR AI
SP  - e54449
VL  - 3
KW  - vaccine safety
KW  - immunization
KW  - febrile convulsion
KW  - syndromic surveillance
KW  - emergency department
KW  - natural language processing
AB  - Background: Collecting information on adverse events following immunization from as many sources as possible is critical for promptly identifying potential safety concerns and taking appropriate actions. Febrile convulsions are recognized as an important potential reaction to vaccination in children aged <6 years. Objective: The primary aim of this study was to evaluate the performance of natural language processing techniques and machine learning (ML) models for the rapid detection of febrile convulsion presentations in emergency departments (EDs), especially with respect to the minimum training data requirements to obtain optimum model performance. In addition, we examined the deployment requirements for a ML model to perform real-time monitoring of ED triage notes. Methods: We developed a pattern matching approach as a baseline and evaluated ML models for the classification of febrile convulsions in ED triage notes to determine both their training requirements and their effectiveness in detecting febrile convulsions. We measured their performance during training and then compared the deployed models’ result on new incoming ED data. Results: Although the best standard neural networks had acceptable performance and were low-resource models, transformer-based models outperformed them substantially, justifying their ongoing deployment. Conclusions: Using natural language processing, particularly with the use of large language models, offers significant advantages in syndromic surveillance. Large language models make highly effective classifiers, and their text generation capacity can be used to enhance the quality and diversity of training data. 
SN  - 2817-1705
UR  - https://ai.jmir.org/2024/1/e54449
UR  - https://doi.org/10.2196/54449
DO  - 10.2196/54449
ID  - info:doi/10.2196/54449
ER  -