Transparent Reporting of AI in Systematic Literature Reviews: Development of the PRISMA-trAIce Checklist

Introduction

Evidence-based practice is the cornerstone of modern science and decision-making across a wide range of disciplines, from clinical medicine and social sciences to engineering and computer science. One method of generating this evidence is a systematic literature review (SLR) [1,2]. In contrast to narrative reviews, SLRs follow a rigorous and predefined methodology to identify, appraise, and synthesize the totality of relevant research on a specific question. The goal is to minimize author bias and provide a reliable answer to the research question through a comprehensive and reproducible process [2].

Despite its widely recognized value, conducting an SLR presents considerable challenges. The requirement for completeness necessitates extensive searches across various scientific databases, often yielding thousands of potentially relevant publications. Each of these must then be manually screened for relevance, typically by at least 2 independent reviewers. This is followed by detailed data extraction and quality assessment of the included studies. The process is not only methodologically demanding but also extremely time-consuming and labor intensive, which significantly hinders the broad application of SLR methodology outside well-funded research groups [3].

Recent breakthroughs in natural language processing, particularly the development and wide availability of large language models such as GPT-4 or Claude 3, offer new possibilities to overcome these challenges. Today, artificial intelligence (AI)–supported tools can assist in almost all phases of the SLR process [4,5], from generating and optimizing search strategies to semiautomating the screening of titles and abstracts and extracting structured data from full texts [5,6]. Studies indicate that these technologies can reduce the manual workload by 50% to 75% [7], bringing the promise of democratizing SLR methodology within reach [8].

However, the integration of AI tools into the scientific process poses a new, fundamental challenge: ensuring transparency, traceability, and trust in the results [9]. The “black box” nature of many LLMs and their known weaknesses, such as “hallucinations” [10,11] or the reproduction of bias from training data [12,13], stand in direct contradiction to the core principles of SLRs. To ensure trust in traditional SLRs, the scientific community relies on established reporting guidelines, most notably the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 statement, which mandates detailed documentation of each step in the SLR process [14].

An analogous guideline including the use of AI in SLRs is clearly needed. Although a corresponding extension, the PRISMA-AI (PRISMA–Artificial Intelligence Extension), was announced in 2022, it has not yet been published. More importantly, according to the planning document, its conceptual focus is primarily on reporting SLRs that investigate AI as a subject of research in clinical medicine [15]. There is still a complete lack of a guideline for using AI as a tool within the research process applicable across all disciplines and an urgent need to formulate globally recognized guidelines for the use of AI tools [16]. This gap creates the risk of AI-supported SLRs becoming a nonstandardized and nontransparent method and therefore scientifically questionable.

To close this reporting gap and, thereby, mitigate the associated risks, we propose PRISMA-trAIce (PRISMA–Transparent Reporting of Artificial Intelligence in Comprehensive Evidence Synthesis). This checklist synthesizes key principles from established AI reporting guidelines to provide a structured and well-founded starting point. We offer this framework as a catalyst for community discussion, aiming to accelerate the collaborative development of a formal, consensus-based standard for transparent AI reporting in SLRs.

Methods

Formulation of the PRISMA-trAIce Checklist

We developed the PRISMA-trAIce checklist through a systematic, multistage process.

First, our targeted literature review focused on established, consensus-based AI reporting guidelines, primarily sourced from the EQUATOR (Enhancing the Quality and Transparency of Health Research) Network, a recognized authority on such standards [17-22]. Our search included keywords such as “artificial intelligence,” “machine learning,” and “reporting guideline.” The methodological approach was designed to build on robust, existing scientific consensus established by leading reporting guidelines. Many of these foundational guidelines, such as the CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) and TRIPOD-AI (Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis–Artificial Intelligence), were chosen as they represent the most widely adopted, cross-domain standards for research involving AI, thus providing the most robust and consensus-driven foundation for our synthesis. From this search, the following core guidelines were selected for adaptation: CONSORT-AI [18], SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence) [19], TRIPOD-AI and TRIPOD-LLM (Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis–Large Language Model) [20,23], DECIDE-AI (Developmental and Exploratory Clinical Investigations of Decision Support Systems Driven by Artificial Intelligence) [21], and the GAMER (Generative Artificial Intelligence Tools in Medical Research) guideline [24]. Leveraging this foundation is scientifically sound and resource efficient. Given the rapid pace of technological development in AI, a multiyear, de novo consensus process would risk producing an obsolete guideline upon publication. Therefore, we synthesized and adapted relevant elements from already existing, established frameworks, creating a focused, cross-disciplinary, and applicable checklist for AI-assisted SLRs.

Second, we extracted all individual checklist items from the selected guidelines for a qualitative content analysis. The guiding question for this analysis was as follows: is this reporting principle relevant and applicable to the use of AI as a methodological tool in an SLR? Each item was evaluated for its relevance to reproducibility, feasibility for authors, and adaptability to the SLR context. Items not considered relevant (eg, those concerning the patient safety of an AI intervention) were excluded. The analysis focused on principles such as the detailed disclosure of AI models, data origins, and the explicit description of human-AI interaction and oversight.

Third, the resulting relevant items were thematically synthesized to identify recurring core concepts (eg, “AI tool identification” and “human-AI interaction”). These thematic clusters were then mapped to the existing structure of the PRISMA 2020 statement to ensure seamless integration and user-friendliness.

Limitations

We recognize that the proposed checklist has limitations as it results from a systematic synthesis and adaptation, not a formal, broad-based Delphi study or consensus meeting. While its foundation is built on prior consensus, the checklist has not yet undergone this process. Its feasibility was assessed based on expert knowledge, but it has not been validated through a formal user study. Therefore, this work is explicitly intended as a well-founded proposal that provides an immediate solution and can serve as the foundation for a subsequent, community-driven formal consensus process.

Ethical Considerations

This paper presents a methodological proposal for a reporting guideline and does not involve research on human participants or animals. Therefore, approval from an institutional review board or an ethics committee was not required. The development of this work adheres to the ethical principles for medical research outlined in the World Medical Association Declaration of Helsinki and follows the guidelines on publication ethics of the Committee on Publication Ethics. The use of generative AI during the preparation of this manuscript is disclosed in the Acknowledgments section. The authors retain full responsibility for the scientific integrity and content of this work.

Results

The previously described method resulted in the PRISMA-trAIce checklist and a corresponding adapted PRISMA flow diagram, which we present in the following sections.

The PRISMA-trAIce Checklist

The PRISMA-trAIce checklist is an extension to the PRISMA 2020 statement designed to guide the transparent reporting of AI use in SLRs. It comprises 14 items that are categorized according to the structure of an SLR (title, abstract, introduction, methods, results, and discussion) and addresses critical aspects of AI use, from tool identification and human-AI interaction to performance evaluation and limitations.

The complete checklist is presented in Table 1. A detailed explanation of each checklist item, including its rationale, elaboration, sources, and application examples, is provided in Multimedia Appendix 1.

The PRISMA-trAIce Flow Diagram

To enhance the transparency of the study selection process, the standard PRISMA 2020 flow diagram was adapted (Figure 1). While the original diagram already includes “automation tools,” our proposed version sharpens this concept by distinguishing between rule-based administrative tools (eg, for deduplication) and evaluative AI systems. Therefore, the adapted diagram provides specific fields to report the number of records screened separately by AI systems versus human reviewers and the reasons for exclusion. This visual guide offers a more granular and immediate overview of the AI’s role in the screening process.

‎

Discussion

Principal Results

This work’s primary contribution is the PRISMA-trAIce proposal, a checklist designed as an extension to the PRISMA 2020 statement. Its purpose is to provide a standardized framework for the transparent reporting of AI as a methodological tool in SLRs while maintaining ease of use for everyone who is already familiar with the original PRISMA 2020 statement. By systematically synthesizing core principles from established, consensus-based AI reporting guidelines, we offer a robust starting point to address a critical gap in the current landscape of scientific reporting.

Rationale for a “Living Guideline”

The central challenge in creating reporting guidelines for a field as dynamic as AI is the tension between methodological rigor and timely relevance. A formal consensus process, while it is the standard, risks delivering a meticulously crafted guideline for an already outdated technological landscape. We argue that the immediate risk posed by the nontransparent use of AI tools outweighs the risk of introducing a well-reasoned albeit preliminary guideline. Therefore, our approach is pragmatic, positioning PRISMA-trAIce as a solution that the research community can immediately adopt and refine.

Limitations

The primary limitation of this proposal is inherent in its nature: it is the result of a systematic adaptation, not a formal, large-scale consensus-building exercise, such as a Delphi study. While grounded in established principles, its items have not yet been empirically validated across diverse research contexts. We do not see this as a shortcoming to be rectified later through traditional means but rather as justification for a more modern, agile approach to standard setting.

Therefore, we propose a community-driven, 2-tiered framework to guide the evolution of PRISMA-trAIce into a living standard. This includes (1) a central, stable anchor on GitHub [25] to serve as the single source of truth for the checklist, enabling transparent, version-controlled development; and (2) a dynamic community hub on Discord to foster rapid and low-friction collaboration. To provide a predictable development cycle, we envision a schedule of annual reviews to incorporate community feedback into future versions. This strategy applies the principles of open-source development to scientific standard setting, ensuring that PRISMA-trAIce can evolve in step with the technology it seeks to describe. We invite experts in evidence synthesis, methodology, and AI to contribute to this initiative. Our long-term vision is to foster a self-sustaining ecosystem. To this end, we are committed to facilitating the establishment of a formal steering committee constituted from dedicated members of the expert community, to which stewardship of the standard will ultimately be transferred.

Comparison With Prior Work

PRISMA-trAIce occupies a unique and critical niche within the landscape of AI reporting guidelines by providing the first framework specifically designed for AI as a methodological tool in evidence synthesis, a gap not addressed by existing standards. We position our contribution by highlighting 2 key distinctions.

On the one hand, it focuses on AI as a tool, not the subject. Guidelines such as CONSORT-AI [18] and the TRIPOD (Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis) statements (including TRIPOD-LLM) [20,23] are indispensable for studies in which AI is the subject of the research, for instance, as a diagnostic intervention or a clinical prediction model. These frameworks guide the reporting of the AI’s performance and impact as a study outcome. In contrast, PRISMA-trAIce is designed for the scenario in which AI is part of the research process itself. It guides the reporting on how this tool was used to produce the final systematic review.

On the other hand, it is a discipline-agnostic and SLR-specific framework. While some guidelines offer advice on reporting the use of AI tools, their scope is often domain specific. The GAMER [24] guideline, for example, provides valuable recommendations but is explicitly tailored to the field of medicine. PRISMA-trAIce, by design, is an extension of the universally adopted PRISMA 2020 statement. This makes it inherently discipline agnostic and directly integrable into the established workflow of systematic reviews across any field, from engineering to social sciences. Its structure is mapped directly onto the sections of an SLR, a level of specificity for the evidence synthesis process that other AI reporting guidelines do not provide.

Therefore, by focusing on AI as a methodological tool in a discipline-agnostic and SLR-specific manner, PRISMA-trAIce complements existing guidelines and addresses a critical, unmet need for transparency in the next generation of evidence syntheses.

Conclusions

The integration of AI into systematic reviews marks a disruptive moment in evidence synthesis, promising efficiency but demanding a renewed commitment to rigor. The current lack of a standardized reporting framework represents a critical threat to the transparency and trustworthiness of these AI-assisted reviews. The proposed PRISMA-trAIce checklist provides a foundational tool to address this challenge, aiming to immediately improve transparency while paving the way for a formal, community-endorsed standard.