Critical Assessment of Large Language Models’ (ChatGPT) Performance in Data Extraction for Systematic Reviews: Exploratory Study

Introduction

Systematic literature reviews (SLRs) are indispensable across various fields, synthesizing evidence to inform decision-making in areas as diverse as public health, policy, and biomedical sciences, where rigor and comprehensiveness are paramount. With the rapid expansion of the literature, SLRs are more important than ever to help not only synthesize evidence but also identify areas in which the literature is robust or deficient [1]. However, conducting SLRs is resource intensive, involving manual and careful screening of potentially relevant studies [2]. In particular, SLRs that assess and report on analytical methods and key findings require more domain-specific expertise and multiple researchers for coding, making them more challenging.

Given the recent rapid advancement of large language models (LLMs), researchers have proposed their potential utility in conducting SLRs [3,4]. Several reviews have found that artificial intelligence (AI)–enabled methods exhibit reasonable performance and improved efficiency in literature screening [5-9], an integral component of SLRs. In particular, the role of SLRs in health and biomedical sciences underscores the need for reliable, accurate data extraction tools that maintain the rigorous standards expected in these fields. However, studies that have tested the capabilities of AI for data extraction have identified challenges necessitating human intervention for completion [6,8].

To understand whether recently developed LLMs can overcome this barrier, many studies have evaluated the performance of various LLMs in automating SLR tasks [10-12]. Their findings reveal LLMs’ potential in extracting data that are relatively easily retrievable (ie, study design, participant characteristics, and primary outcomes). However, these studies have not explored the ability of LLMs to extract more complex data (eg, methods used to obtain study outcomes), which may pose greater challenges as extracting such complex information is often subject to individual researchers’ perspectives [13,14].

LLMs currently demonstrate advantages in assisting researchers, such as having rapid response times and providing high-level summaries of results [15]; however, a recent study revealed the shortcomings in ChatGPT’s depth of knowledge and contextual understanding for conducting SLRs in comparison to researchers [16]. Yet, there is growing potential for ChatGPT’s utility as an assistant in complex qualitative content analysis, as shown by a recent study that assessed its ability to categorize strategies and behaviors in forum posts about reducing sugar intake [17]. Thus, although LLMs show potential in data extraction tasks for SLRs, there is more to be explored on the current LLMs’ capability to undertake a comprehensive approach that involves extracting not only basic study characteristics but also information critical for interpreting the results of studies within SLRs.

In this study, we aimed to evaluate an LLM’s (ChatGPT) ability to extract more complex, nuanced data from scientific studies, representing a novel approach that goes beyond the simpler tasks of extracting descriptive information, such as study design or participant characteristics, which have been the primary focus of prior research. Our study applied a structured series of prompts and validations to systematically gauge where LLMs excel and where human oversight remains essential.

We used ChatGPT (GPT-4) for its accessibility and ease of use, recognizing that most individuals conducting SLRs may not be proficient with more customizable, technical platforms, such as the GPT application programming interface (API). Sophisticated LLM frameworks (eg, retrieval-augmented generation) offer technical capabilities that may better address needs for SLRs, but these features generally require AI-specific expertise. Consequently, our study prioritized a more common usage scenario in which a widely accessible GPT model is used for data extraction tasks.

Methods

Overview

We focused on COVID-19 simulation modeling studies as a case study, leveraging a large collection of study reports that our team recently assembled for a SLR that aimed to assess the incorporation of human behavior dynamics in COVID-19 simulation models [18]. For this study, the data elements selected for extraction were aligned with the specific objectives of our review: to evaluate ChatGPT’s capacity to manage both explicit and nuanced data in SLRs. To determine whether LLMs can effectively screen papers and extract information, we randomly chose 10 of the papers and extracted data both manually and using ChatGPT. We selected a sample of 10 studies [19-28] as this number was manageable for an in-depth exploration of each paper, allowing us to conduct detailed comparisons of extraction accuracy and making it feasible to perform multiple iterations of prompt engineering and assessment throughout the process.

Our examination of COVID-19 modeling studies leveraged our team’s expertise and enabled us to confidently determine correct answers for meaningful comparisons with ChatGPT’s outputs. Although COVID-19 may no longer be at the forefront of global attention, these studies remain a methodologically rich test case rather than a focus of current clinical relevance. They combine straightforward features (eg, study design, interventions) with more abstract elements of human behavior (eg, compliance with public health measures, mobility changes), allowing us to systematically assess ChatGPT’s performance across varying levels of complexity. Furthermore, the pandemic’s urgency previously highlighted the value of accurate, rapid synthesis of research findings during health emergencies, making it an ideal context for evaluating the practical utility of LLMs in accelerating SLRs.

Many researchers who conduct SLRs may not necessarily have the expertise required to implement LLMs through advanced tools. Thus, we conducted our analysis using ChatGPT, given that it is one of the most widely adopted models, attracting 393 million users each month, as shown by October 2024 data [29]. We used the web browser interface for its user-friendly design, as opposed to an API. Although using the API might have offered more controllable responses, our study prioritized accessibility and usability, reflecting the real-world context in which many researchers engage with ChatGPT. We selected the GPT-4 model specifically, as at the time of our analysis, it was one of the few that directly analyzed full text as a PDF file. Although our approach could also be applied to studies available in other formats, our focus on PDF files reflects their frequent use in academic publishing and SLRs.

Ethical Considerations

This study did not involve human participants, identifiable human data, or interaction with individuals. As such, it did not fall under the scope of research requiring review by an institutional review board, and ethical approval was not required.

Data Elements

We defined the sets of measures to be extracted as (1) study settings (ie, analysis location, modeling approach, and analyzed interventions) and (2) behavioral components (ie, changes in travel and mobility, perception of risk and severity, and compliance and resistance to public health measures). Therefore, for each of the 10 studies [19-28], we extracted 6 distinct data elements, resulting in 60 data elements.

We distinguished these measures to reflect their nature: study settings are straightforward, whereas extracting information on behavioral components in COVID-19 models is influenced by researchers’ perspectives [13]. We confined our study settings to information explicitly stated in the text. For behavioral components, we categorized them into no mention (A), mentioned but not modeled (B), modeled exogenously but not analyzed (C), modeled exogenously and analyzed (D), modeled endogenously but not analyzed (E), and modeled endogenously and analyzed (F). Endogenous modeling incorporated human behavior as an internal part of the model, where it both influenced and was influenced by the spread of COVID-19. Exogenous modeling indicated behavioral changes were external factors impacting the spread of COVID-19 without being influenced by it. This classification ranged from minimal (A) to comprehensive (F) incorporation into COVID-19 models.

Data Extraction Process and Comparison

Two trained researchers independently extracted data from the 10 studies [19-28] and then reconciled discrepancies in their findings—6 related to study settings and 11 to behavioral components—for convergence. The researchers discussed any unresolved discrepancies with a third senior researcher to reach a consensus. Subsequently, we initiated a dedicated session for every study and prompt, uploading the individual files (in PDF) into the GPT-4 model using ChatGPT’s user interface (accessed in January-April 2024) and documented the responses. To gain confidence in our manual screening, wherever ChatGPT, consistently through iterations of prompts, provided answers that disagreed with our manual coding, we reassessed our original codings. Following this reassessment and after making necessary adjustments, we finalized the manual screening results and considered them as the correct responses. We calculated individual researchers’ average accuracy rates (the percentage of their correct responses before any consensus was reached), allowing us to directly compare ChatGPT’s performance against the individual researchers’ average accuracy for each measure throughout prompt iterations [8].

Prompt Engineering

We started by providing ChatGPT with a general prompt to extract each desired data element. Due to initially unsatisfactory results, we iteratively engineered prompts based on the initial responses and our manual coding. This process involved altering the wording of prompts for clarity and concisely adding specific descriptions of our objective, approach, and definitions of key terms [30]. Particularly, we often guided ChatGPT to base any interpretations strictly on what was explicitly stated in the text, as it often made incorrect inferences. Furthermore, we followed up with ChatGPT regarding its incorrect responses by inquiring about potential improvements to the prompts after clarifying the desired answers [30].

We continued to refine the prompts until we achieved complete alignment with our manual screening results, reached saturation in improvement, or explored viable avenues for prompt enhancement to the best of our capabilities. Given the exploratory nature of this study and the fact that LLMs are designed to interact with users in real time, prompt refinement without a formal training phase reflects a common use case for this technology. Recent tutorials and case studies have demonstrated ChatGPT’s feasibility in domain-specific and rapid literature reviews, reinforcing its relevance as a practical, real-world tool that benefits from iterative, user-guided prompt refinement [31,32]. Table 1 illustrates an example of the iterative process used to prompt ChatGPT for extracting data elements in this study. We applied a similar method to extract additional data elements, with the processes reported in Multimedia Appendix 1.

Results

Table 2 summarizes ChatGPT’s responses across 6 measures for the last iteration of prompts. These measures were divided into study settings and behavioral components.

Through the course of iterations, we identified 4 instances where ChatGPT consistently disagreed with our manual coding but was determined to have provided the correct answer upon reassessment. These instances were among the prompts related exclusively to study settings: 3 pertained to answers generated for prompts about interventions analyzed, and 1 addressed the correct location of analysis.

As a result of iterative prompt engineering, the average accuracy of ChatGPT’s responses showed a marked improvement of 33% and 23% between the initial and final iterations for extracting study settings and behavioral components, respectively (Figure 1). Specifically, in our initial prompts, 26 (43.3%) of 60 ChatGPT responses were correct. However, the latest prompt version yielded 43 (71.6%) correct answers. The iterative responses from ChatGPT and a comparison with our manual screening are detailed in Multimedia Appendix 1. By the fourth iteration, ChatGPT outperformed the individual screeners’ average accuracy in identifying study settings (Figure 1). However, ChatGPT consistently could not achieve a level of precision comparable to that of manual screeners (Figure 1) when extracting behavioral components.

‎

Specifically, ChatGPT provided correct responses for all 10 papers [19-28] when prompted to identify the analyzed location and model type used by the second and seventh iterations of prompts, respectively. At most, ChatGPT correctly identified 8 of 10 interventions analyzed within the studies, achieving a peak accuracy of 80% by the sixth iteration (Figure 2). Conversely, it took 6 and 7 iterations to reach a peak accuracy of 80% (ie, 8 of 10 correct classifications) for classifying how each study assessed changes in travel and mobility and the perception of risk and severity, respectively. Our alignment with ChatGPT’s responses for coding compliance and resistance to public health measures only achieved a maximum consistency rate of 3 (30%) correct answers across the 10 studies [19-28] (Figure 2).

Finally, the contrast between the distribution of the manual coding and ChatGPT is presented in Figure S1 in Multimedia Appendix 1, and the distributions of ChatGPT’s responses are presented in Figure S2 in Multimedia Appendix 1.

‎

Discussion

Principal Findings

Our analysis underscores the finding that ChatGPT’s assistance in full-text screening of study reports is particularly useful when handling simple inquiries, specifically general study settings, for which details are typically explicit in the text. However, the task of assessing nuances that necessitates drawing inferences (eg, the integration of human behavior in COVID-19 models), where the study may or may not have an explicit discussion, presents a significant challenge for ChatGPT.

Rather than prolonging the iterative process to elicit correct responses, we adopted a strategy of structuring prompts based on ChatGPT’s previous responses to ensure explicitness over the course of 7 iterations. Although it may have been possible to continue iterating to improve response accuracy, such a strategy is impractical in real-world scenarios. In addition, we acknowledge the potential risk of overfitting due to iterative prompt engineering on the same set of studies; however, our primary objective was to explore the utility of ChatGPT in real-world use cases where prompt refinement is common practice. This utility is especially relevant, given the broad reliance on SLRs to guide evidence-based practices across fields, with particular importance in health and biomedical sciences, where decisions impact health outcomes [31]. Hence, we limited this study to a manageable number of papers to better understand the limitations of prompt engineering itself. Importantly, the framework we used for evaluating ChatGPT (GPT-4) in COVID-19 modeling studies can be applied similarly to other SLRs regardless of the specific topic.

We highlighted that ChatGPT’s performance is influenced by the explicitness of information within the text, not just by the clarity or objectivity of the prompts. This underscores a nuanced limitation: the technology’s current dependency on explicit textual evidence for accurate data extraction. This limitation is notable, even with the use of straightforward prompts, underscoring a significant barrier in LLMs’ application to literature analysis. For instance, despite clear prompts, ChatGPT often struggled to correctly identify the model type in studies—expected to be straightforward—unless explicitly mentioned. Even when instructed to label model types as “unspecified” in the absence of clear documentation, early iterations often resulted in incorrect answers rather than adherence to the “unspecified” directive. This illustrates that ChatGPT’s accuracy is dependent not only on the prompt structure but also on the presence of explicitly detailed textual information. Hence, a central insight from this analysis is the significant obstacle that LLMs encounter when navigating ambiguity, further complicating the tasks of engineering effective prompts.

Furthermore, among ChatGPT’s responses that did not correspond to our manual coding, we observed a tendency for ChatGPT to extrapolate beyond the presented data. Despite instructions to confine responses to the explicit content of each study, ChatGPT often listed additional interventions not stated by the authors. This pattern of overreporting was also evident when categorizing studies based on the extent of their integration of behavioral components into the model. ChatGPT frequently assigned a higher integration level than that supported by the studies’ text (Multimedia Appendix 1). These skewed errors align with ChatGPT’s tendency to “hallucinate,” or provide confidently articulated yet factually unsupported responses [10].

Despite these challenges, LLM tools may be useful for SLRs. For 3 (30%) studies, ChatGPT correctly identified and analyzed interventions that we initially overlooked. For example, in Wong et al’s study [23], ChatGPT identified COVID-19 testing as an intervention in 6 of 7 prompt iterations—a detail our original manual assessment missed. This led us to reevaluate the study, and upon confirming ChatGPT’s accuracy, we modified our assessment accordingly. Conversely, we revisited our categorization of behavioral components in 6 incidences with which ChatGPT consistently disagreed, but we confirmed that our original manual coding was correct.

The main contribution of this study is to extend the understanding of current LLMs’ capabilities and limitations in handling complex data, providing valuable insights for those who conduct SLRs and are exploring the use of LLM platforms. Our observation that ChatGPT outperformed the average accuracy of individual reviewers when identifying study settings underscores its utility as an assistant or a second reviewer in extracting basic measures for SLRs. These results support previous research, which indicates the potential of LLMs for handling basic data extraction tasks effectively [10,11] and their use as a collaborator [32] or a second rater [33] in SLRs. However, for ChatGPT to be effectively used in this role, there remains a need for researchers to provide clear and detailed prompts that provide the relevant context. This approach requires researchers to have a thorough understanding of the context relevant to their inquiries and have access to reliable, coded data elements to directly compare against ChatGPT’s responses.

In terms of extracting complex components, ChatGPT failed to achieve comparable accuracy to that of individual screeners, highlighting the continued necessity of manual data extraction and additional research to overcome the limitations of this technology. Although further testing with a test sample may yield different results, the insufficient performance observed in our training data alone suggests that current LLMs remain unreliable for handling complex data extraction tasks. These findings align with other studies that discuss LLM performance, which similarly conclude that although automation is advancing, prevailing errors emphasize that structured oversight remains critical [15,34].

Limitations

This study is subject to several limitations. First, this study focused on one aspect of the SLR process, given that other steps (eg, writing Boolean query formulations [35] and screening titles and abstracts [10,36]) have already been examined in greater detail. Second, we evaluated ChatGPT with only one review topic, which limits the generalizability of our results. Third, we often noticed inconsistencies in ChatGPT’s responses for the same prompts, but we did not formally assess reproducibility. Fourth, although accurate data extraction highly depends on the LLMs’ capabilities in accurately parsing PDF files [12], our paper does not quantify the impact of any errors in converting PDF files to text. Nonetheless, our results still highlight the stark differences in LLMs’ capabilities in accurately extracting simple versus complex data elements. Fifth, we selected ChatGPT for this study due to its wide accessibility and usability. We recognize that our exclusive focus is a limitation, as other models may have provided different insights or comparative performance benchmarks. Future research should explore the capabilities of multiple models to provide a more comprehensive understanding of LLM performance in this context. For example, certain advanced approaches (eg, retrieval-augmented generation) exist to better handle evidence or references from external sources. Since our focus was on a common, practical scenario in which typical reviewers use a publicly accessible GPT model, these specialized methods fall outside the scope of this study. Finally, due to the rapidly changing nature of LLMs, our findings may not hold over time.

Conclusion

Overall, LLMs, and ChatGPT in particular, show promising performance in assisting the extraction of explicitly stated information from the full text of study reports, particularly when limited scientific reasoning is required. However, ChatGPT currently exhibits limited potential for fully automating data extraction across more complex, subjective measures. Our findings emphasize the ongoing necessity of human oversight in handling complex, nuanced data extraction tasks, even as LLMs continue to improve. This position is consistent with broader calls in the literature to adopt a cautious, well-evaluated approach to integrating LLMs into evidence synthesis workflows [15]. We highlight an important contribution to human-AI collaboration research, demonstrating the need to integrate AI tools with human oversight in SLRs, particularly in areas where current models fall short.