ChatDRex streamlines complex, network-based drug repurposing analyses with a conversational interface. This approach is designed to address potential biases and hallucinations in LLMs by integrating directly with validated biomedical databases and specialized bioinformatics tools. It ensures data accuracy and enables sophisticated analyses without requiring specialized computational expertise.

The implemented multi-agent architecture is inspired by the chain-of-thought (CoT) paradigm [33] but adapted to the modularity of complex systems. In contrast to the original CoT, in which a single model sequentially externalizes the intermediate steps of reasoning [34], our approach delegates reasoning to a set of specialized agents, each of which has a clearly defined role in the drug repurposing workflow (Figure 2). A central planning agent functions as a meta-controller, breaking down user requests into subtasks, iteratively assigning them to the appropriate expert agents, and merging their results into a coherent outcome [35]. Consequently, the architecture enables task planning and orchestration, allowing each agent to focus on a small subtask using CoT [34].

ChatDRex is a multi-agent system (Figure 2), where all agents are constructed based on the same architectural principles (Figure 3). Each agent integrates a unified control loop consisting of few-shot prompting, which is part of ICL, short-term and long-term memory access, internal planning routines, and external tool integration. Our lightweight summary agent maintains contextual continuity by compressing prior interaction states (Section S1 in Multimedia Appendix 1). This reduces token usage and enables controlled memory resets. Prompt-injection safeguards are applied at this stage, which is particularly relevant given that LLMs are systematically vulnerable to jailbreaks and instruction-overriding attacks [37-39]. In addition, a sanity check is performed here to verify, based on user feedback, whether the LLM has correctly understood the request (details and an evaluation of the input guardrails can be found in Multimedia Appendix 2). According to the available summary and the new user prompt, the coordination of tasks is the responsibility of a planning agent. This agent uses an LLM-based decision process to dynamically direct user input to the most suitable specialized agents (Section S2 in Multimedia Appendix 1). The selection of agents is formulated as a multistep problem, ensuring efficient task decomposition and specialization [35]. If biological validation or interpretation is requested by the user, this is provided by the DIGEST (In Silico Validation of Disease and Gene Sets, Clusterings or Subnetworks) agent (more detailed algorithm descriptions can be found in Section S3 in Multimedia Appendix 1). It uses DIGEST to evaluate the functional coherence of gene sets and disease-associated subnetworks. Conventional enrichment approaches, which enumerate overrepresented terms, are contrasted with DIGEST, which quantifies internal functional coherence through comparison with randomized reference sets. This process provides empirical significance estimates [36]. Within ChatDRex, DIGEST is used to assess the biological plausibility of disease modules derived from network analyses, as well as user-provided gene sets in the absence of network context.

Network-driven analyses are conducted by the NeDRex agent, which provides algorithmic access to the integrated NeDRex KG (more detailed algorithm descriptions can be found in Section S4 in Multimedia Appendix 1). This agent orchestrates established network-medicine tools, including DIAMOnD for disease module expansion from seed genes (Section S4.1 in Multimedia Appendix 1) [15]. TrustRank is a methodology that facilitates the prioritization of drug targets based on their proximity to trusted disease nodes (Section S4.2 in Multimedia Appendix 1) [43]. Closeness centrality is used to rank candidates according to their global network distances (Section S4.3 in Multimedia Appendix 1) [7]. The integration of these tools facilitates the systematic identification of disease-relevant subnetworks and candidate therapeutics within a unified interaction graph, thereby constituting the computational core of ChatDRex’s network-based drug repurposing capabilities. The KG agent is responsible for the structured knowledge retrieval process (Section S5 in Multimedia Appendix 1). This agent translates natural language queries into schema-constrained graph queries over the NeDRex KG (Figure 4). The agent enhances its resilience, traceability, and resistance to hallucinations by integrating semantic query decomposition, embedding-based node matching, and deterministic Cypher query generation. Additionally, it uses fallback strategies that are guided by graph-based retrieval.

In order to achieve this, user queries are first semantically mapped to the fixed schema of the NeDRex KG and decomposed into a list of typed subquestions, rather than an explicit question graph as used in prior work [44]. Each subquestion is processed via an LLM call. This call identifies and annotates the corresponding KG node type, generates the query, and determines whether subsequent filtering is required. Subquestions that have been filtered are enriched using an embedding-based similarity search with the top 5 most similar KG nodes. Based on this representation, the agent generates Cypher queries deterministically for execution on the Neo4j-backed NeDRex KG. Query generation is guided by the known NeDRex schema and predefined type and edge constraints, reducing hallucinations and improving traceability [45,46]. The prior node enrichment enables robust filtering using resolved KG identifiers, which mitigates errors caused by spelling variations or lexical mismatches. To prevent over-constraining queries, filtering is applied only to elements that are explicitly marked. If Cypher query generation fails after a fixed number of retries, the agent switches to a GraphRAG-based strategy, in which graph-guided retrieval results are condensed and given to the LLM to generate a response [47]. In instances where NeDRex knowledge is inadequate, the research agent initiates a literature-based retrieval process by decomposing the query into focused subquestions and querying external scholarly resources (Semantic Scholar; Section S6 in Multimedia Appendix 1) [35,48]. This process serves to complement the KG with the most recent scientific evidence [35,48]. Consequently, the finalize agent synthesizes intermediate results into a coherent, citation-aware response (Section S7 in Multimedia Appendix 1) [49,50]. A secondary verification step evaluates generated content against the planning state, which contains the raw results from the other agents to mitigate hallucinations and ensure factual consistency [51,52]. If a graphical representation is required, the user has the option of controlling the Drugst.One graph representation via the network agent [6].

Evaluating LLM-based agent systems is challenging because, unlike deterministic programs, their outputs are inherently interpretive and nondeterministic, meaning there is often no single objective ground truth beyond simple accuracy metrics. Unlike deterministic systems, LLMs generate responses that encompass generation processes, complex interpretation, and decision-making mechanisms [53]. Additionally, stochastic LLM outputs can lead to variability across runs. The aforementioned problems result in a multifaceted evaluation challenge [54]. First, it must be verified whether the model uses external tools correctly and appropriately. The next step is to check that the resulting information has been processed and communicated correctly. Furthermore, LLMs have the capacity to generate statements that, while superficially plausible, are factually inaccurate, thereby introducing a degree of complexity to the evaluation process [55]. Quantitative assessment is only feasible for well-defined subtasks with an explicit ground truth (eg, exact database queries). Conversely, higher-level outputs, such as literature support, scientific interpretation, and argument completeness, lack a unique ground truth and are therefore only partially and qualitatively assessable [55]. A robust evaluation, therefore, requires a combined approach of automated metrics and manual validation [53]. To evaluate the accuracy of the answers created by an LLM, an LLM-as-a-judge model is used. An LLM-as-a-judge model evaluates the quality, accuracy, or relevance of answers using an LLM to simulate human judgment in tasks such as evaluation, ranking, or criticism [56]. All LLM-as-a-judge evaluations were performed by a single fixed judge model, gpt-oss:20b, regardless of which model generated the answers under assessment. The judge was run with a temperature of 1 and top-P of 0.9 (Multimedia Appendix 3). Given the potential for such distortions to be influenced by hallucinations, the process was closely monitored by experts and underwent corrective adjustments as needed. All evaluations were run thrice with gpt-oss:20b, nemotron-3-nano:4b, qwen3.6:27b, and gemma4:e4b. All model parameters can be found in Multimedia Appendix 3. Individual reported numbers represent the average of the best-performing model.

Tool accuracy is defined as the ability of an agent to identify and select the correct tool in response to a user’s input. For each tool, 100 evaluation questions were generated, resulting in a total of 400 queries. To assess tool selection under different prompting styles, 50 questions per tool explicitly contained tool-related keywords, whereas the remaining 50 questions described the desired task in natural language without mentioning the tool itself. This setup allowed us to evaluate both keyword-based and semantic tool selection capabilities. To further assess robustness to imperfect user input, 10 questions within each tool category were modified to include realistic human typing errors, such as misspellings, omitted characters, and typographical mistakes. The evaluation questions were generated synthetically to reduce manual workload, ensure a consistent structure, and maintain linguistic variation. A ground-truth dataset was constructed manually by selecting NeDRex diseases with associated genes available as gene symbols, UniProt identifiers, and Entrez Gene identifiers. These identifiers were incorporated into the questions and a dedicated extraction field, and the corresponding correct translations were recorded as ground truth. During testing, only the translation endpoint of the NeDRex API was executed live. The evaluation of all NeDRex tools and the network agent was conducted using a precomputed set of results associated with each question in the test set. This approach was implemented to optimize the evaluation runtime. The evaluation followed a 2-stage procedure. Call accuracy measures correct tool and API invocation, while answer accuracy evaluates the correct reflection of returned results in generated answers. This setup isolates tool-selection errors from errors in result interpretation and enables a focused assessment of ChatDRex’s tool accuracy.

To evaluate the NeDRex KG tool, we created 174 questions to determine whether the result set returned by the agent represents a superset of the silver standard annotation dataset (SSAD) created by human experts. The SSAD for the NeDRex KG evaluation was created using a self-generating and self-evaluating mechanism that leverages several language models and targeted validation steps (Section S8 in Multimedia Appendix 1). We manually reviewed the generated question-Cypher pairs, rephrasing queries where necessary to ensure they were biologically meaningful and logically sound. The end result was a manually corrected dataset. For evaluation purposes, each returned element must contain all properties defined in the SSAD, such as the biological meaning of the natural language question and the semantic relatedness between the natural language question and the corresponding Cypher query. In the comparison between the ground truth and generated answer, additional attributes were allowed provided that no expected elements were missing and no additional semantically inappropriate entries were introduced. A matching procedure was implemented to calculate the F_₁-score. This procedure identifies a possible hit in the LLM result for each manually curated question. Such a hit is only counted if all values of the SSAD result are contained in the corresponding LLM entry. We compared the results with GraphRAG and a classic CypherQA chain [57]. To contextualize the performance of the NeDRex KG tool, we compared its final answer outputs with those produced by a GraphRAG baseline and a conventional CypherQA chain [47,58]. For these baseline comparisons, we evaluated only the final returned results and did not assess intermediate reasoning steps, tool calls, or query-generation trajectories.

To quantitatively evaluate the research agent, we developed a benchmark for answering biomedical literature questions based on the publicly available PubMedQA dataset [59,60]. We assessed answer correctness with an LLM-as-a-judge procedure, using the PubMedQA question, the research agent’s response, and the reference answer as context. Responses were marked correct if they were semantically consistent with the reference answer. This included cases with additional relevant, literature-based context that did not contradict the expected conclusion. This semantic evaluation was appropriate as the agent produces free-text evidence syntheses rather than categorical yes/no/maybe labels.

ChatDRex is built on LangChain4j and Quarkus Support to streamline complex, network-based drug repurposing analyses through a conversational interface [61-63]. The frontend (Angular) connects to Quarkus servers, which communicate via WebSocket [64]. In addition to the WebSocket server, the backend offers a Representational State Transfer (REST) API and a model context protocol (MCP) server [65]. These provide the most important tools for external integration. These features make the frontend interchangeable and integrable into other LLM workflows [65]. ChatDRex utilizes, for example, gpt-oss:20b [66], a highly capable LLM developed by OpenAI and optimized for helpful responses (Multimedia Appendix 3). It is hosted through a self-hosted Ollama instance [67]. We use Ollama behind a self-hosted OpenWebUI that allows access control [68]. Furthermore, ChatDRex’s multi-agent architecture is designed to be flexible, allowing the integration of new and improved LLMs at the individual agent level as they become available. This modularity ensures that developers can add new algorithms, external APIs, or domain-specific applications through dedicated tool wrappers and service connectors without altering the core dialogue and orchestration logic. Consequently, ChatDRex remains adaptable to advances in LLMs and biomedical software. Presently, this extensibility is developer-mediated rather than fully self-service; arbitrary user-provided tools are not yet dynamically plugged in at runtime without a wrapper implementation.

This was a small-scale user study in which participants tested the ChatDRex application. In accordance with the screening procedure of the Ethics Commission of the Department of Informatics, University of Hamburg, the completed basic questionnaire (“Basisfragebogen”) indicated that the study posed no more than minimal risk to participants and involved no deception, no vulnerable populations, and no processing of sensitive personal data. Formal review by the Ethics Commission was therefore not required. The study was conducted in accordance with the Declaration of Helsinki and the data protection requirements of the University of Hamburg. All participants provided informed consent prior to participation.

ChatDRex provides end-to-end conversational analysis for network-based drug repurposing. It accomplishes this by following a workflow proposed, amongst others, in NeDRex [7], but within a chatbot-like environment. ChatDRex combines schema-constrained KG querying, network module expansion (DIAMOnD), drug prioritization (TrustRank or closeness centrality), and in silico validation based on functional coherence (DIGEST). In addition, literature research is integrated into the workflow. These findings are presented as interactive networks in-chat with a consistent visual legend across analyses using Drugst.One for visualization [6]. Examples can be found in Multimedia Appendix 4.

Beyond qualitative use cases, we quantitatively evaluated the agent tool interactions underlying ChatDRex (Table 1). Network-based tools, such as DIAMOnD, TrustRank, and closeness centrality, achieved near-perfect tool accuracy ranging from 0.97 (SD 0.00) to 0.99 (SD 0.00), indicating robust recognition of intention and correct tool selection by the agent. In contrast, the DIGEST-based analysis showed slightly lower tool and call accuracies (mean 0.85, SD 0.01 and mean 0.77, SD 0.01, respectively), reflecting the increased complexity of mapping closely related analytical tasks from user prompts.

For network expansion and prioritization tools (DIAMOnD, TrustRank, and closeness centrality), the final outcome presented to the user is determined directly by the underlying API outputs (Figure 5A, B, and D). The LLM does not perform additional interpretation in these cases but returns the deterministic results produced by NeDRex, such that tool and call accuracies fully characterize the system performance. In contrast, DIGEST-based functional enrichment requires the LLM to interpret structured enrichment results and summarize them in natural language, introducing an additional reasoning layer that was explicitly evaluated (Figure 5F). Accordingly, DIGEST showed reduced answer accuracy and was strongly penalized by the automated LLM-as-a-judge, which achieved only 0.3 (SD 0.01) accuracy for this task, despite the underlying analytical information being factually correct. Manual expert evaluation of the same DIGEST outputs yielded a substantially higher answer accuracy of 0.66 (SD 0.01), highlighting a pronounced mismatch between automatic LLM-based judging and human assessment. This discrepancy raised the question of whether LLM-based judging can be trusted more broadly. To address this, 20 randomly sampled answers from each evaluation area were independently assessed by human raters. Across all other components, human and automated judgments showed no significant differences, confirming the reliability of the LLM-as-a-judge in general. The mismatch is therefore specific to DIGEST. Here, outputs require interpreting complex enrichment results, and these results are expressed in natural language. This task is not well suited for current automatic evaluation schemes.

To further investigate this discrepancy, all DIGEST responses were manually re-evaluated by a human annotator using the NeDRex API as an external reference for verifying the correctness of the generated answers. This reassessment confirmed that several responses originally marked as incorrect by the automatic evaluator were in fact correct, indicating that the observed disagreement was primarily caused by limitations of the evaluation procedure rather than by errors in the generated analyses. Consequently, manual verification supported by domain-specific resources remains necessary for reliably assessing such outputs. These findings highlight the need for more specialized evaluation frameworks for biomedical LLM systems. Generic answer-similarity metrics or broad LLM-as-a-judge scores may insufficiently capture whether an answer is biologically plausible, mechanistically grounded, supported by retrieved evidence, and safe for downstream biomedical interpretation. Future work should therefore prioritize the development of domain-specific evaluation metrics that jointly assess factual correctness; evidence grounding; biomedical reasoning quality; uncertainty handling; and the distinction among partially correct, unsupported, and biologically misleading outputs.

Importantly, this evaluation artifact does not reflect a flaw in ChatDRex’s analytical pipeline. The consistently high tool and call accuracies across all components demonstrate that the underlying results are methodologically sound and can be trusted. Taken together, these findings show that ChatDRex provides reliable and reproducible tool outputs, while improving the automatic evaluation of complex natural language summaries remains the primary bottleneck for further development.

The evaluation of schema-constrained NeDRex KG queries showed that the proposed agent improves automatic Cypher query generation for accessing biomedical KGs while also highlighting remaining limitations in translating complex natural language queries into complete Neo4j statements. The strongest performance was observed for qwen3.6:27b, which achieved the best overall results with an average accuracy of 0.76 (SD 0.03) and a call accuracy (F_₁-score) of 0.82 (SD 0.01; Figure 5C). For this model, the fallback was used once, and in 5% of cases, more than one run was required to generate a correct Cypher query. The gpt-oss:20b model demonstrated the second-best performance, achieving an average accuracy of 0.66 (SD 0.01) and a call accuracy (F_₁-score) of 0.70 (SD 0.02). In contrast, smaller models demonstrated lower and less stable performance, with nemotron-3-nano:4b achieving an average accuracy of 0.59 (SD 0.06) and an F_₁-score of 0.58 (SD 0.02) and gemma4:e4b achieving an accuracy of 0.64 (SD 0.01) and an F_₁-score of 0.35 (SD 0.02).

The CypherQA baseline achieved values of 0.46 (SD 0.00) for gpt-oss:20b, 0.34 (SD 0.03) for nemotron-3-nano:4b, 0.42 (SD 0.01) for qwen3.6:27b, and 0.24 (SD 0.02) for gemma4:e4b. On the other hand, GraphRAG’s performance was lower on structure-specific KG questions, achieving values of 0.24 (SD 0.00), 0.17 (SD 0.04), 0.34 (SD 0.02), and 0.27 (SD 0.01), respectively. Overall, these results demonstrate that the KG agent benefits from the proposed architecture.

For the evaluated PubMedQA questions, all tested local models achieved high semantic answer accuracy. The best-performing configuration was qwen3.6:27b, with an answer accuracy of 0.93 (SD 0.01), closely followed by nemotron-3-nano:4b (accuracy of 0.92, SD 0.01) and gpt-oss:20b (accuracy of 0.90, SD 0.00). The gemma4:e4b model achieved a lower but still solid value of approximately 0.81 (SD 0.01; Figure 5E). As expected, a larger model was associated with better results in most cases. For comparison, the GPT-4 (Medprompt) baseline achieved a value of 0.82 [60,69]. Thus, the best ChatDRex research agent configurations in this evaluation outperformed the GPT-4 (Medprompt) baseline and performed significantly better than the originally reported accuracy of the fine-tuned BioBERT PubMedQA [69].

Importantly, the research agent was not limited to the single summary associated with each PubMedQA entry. Instead, it accessed an average of more than 30 literature sources per question, allowing the system to synthesize insights from multiple publications. This explains why the evaluation should not be interpreted as a standard PubMedQA ranking comparison but rather as an end-to-end benchmark for the retrieval and synthesis of biomedical literature using PubMedQA questions as controlled reference tasks [59]. The results show that ChatDRex can leverage external literature sources to generate semantically correct biomedical answers and that its literature-enhanced research workflow offers a clear advantage over purely fine-tuned, closed-context models for this type of biomedical QA. Drugst.One was excluded from the quantitative agent evaluation (see the Limitations section).

To evaluate how hallucinations are handled, we created a set of 10 curated biomedical multistep reasoning cases. These cases were designed to test the detection and correction of hallucinations under controlled conditions (Figure 6). Each case included an unsupported conclusion. For instance, therapeutic, causal, drug-target, drug-disease, pathway, or variant-disease claims that were not directly entailed by the retrieved evidence could be introduced.

We evaluated ChatDRex to determine if its grounding and validation mechanisms could detect unsupported answers, trigger corrections, and produce final answers consistent with NeDRex and the literature. The detection and correction score reflects how often unsupported drafts were successfully identified and corrected. The hallucinated draft score quantifies the grounding risk of the initial unsupported answer, while the corrected answer score measures the remaining unsupported content after correction. Across the evaluated models and repeated runs, ChatDRex achieved high detection and correction scores, with complete correction in most runs (Figure 6). The mean hallucinated draft score remained high across models, whereas the corrected answer score decreased substantially, indicating improved grounding after correction. A representative case demonstrated that an unsupported claim that imatinib targets EGFR was rejected and revised to reflect that imatinib is only supported for targeting ABL1, KIT, and PDGFRA. Thus, EGFR-based repurposing is not supported by the retrieved evidence.

To evaluate the usability of ChatDRex, we conducted a pilot study aimed at obtaining a user-centered assessment of ChatDRex’s usability (Figure 5G). The study participants included 7 bioinformaticians and 1 clinician (3 male and 5 female experts). To assess the predictions and their comparability, these participants were asked to review and evaluate the examples (Multimedia Appendix 4). The questionnaire can be found in Multimedia Appendix 5. In the questionnaire, participants reported good comprehensibility and user friendliness. Participants expressed confidence in the accuracy of ChatDRex’s predictions, as sources and specific links between KG nodes were included.

Through all the previously presented steps, ChatDRex provides researchers with an interface to access and analyze biomedical data while maintaining conversation coherence and system reliability. The system implicitly translates between gene and protein identifiers and efficiently handles secure connections to the NeDRex database through a custom API client (Tables S1 and S3 in Multimedia Appendix 4), facilitating the comprehensive exploration of genomic data relationships. To demonstrate the platform’s capabilities, we present concrete use cases that align with common workflows in drug repurposing research (Figure 7; Table S5 in Multimedia Appendix 4).

The platform interacts with NeDRex, which enables schema-constrained Cypher queries to retrieve disease indications and related associations directly from the KG. It also uses DIAMOnD for disease module detection and DIGEST to estimate these modules’ functional coherence based on enriched pathways and functional annotations (Tables S1 and S4 in Multimedia Appendix 4). TrustRank is applied to prioritize proteins and drugs for assessing network reliability (Table S5 in Multimedia Appendix 4). Finally, ChatDRex uses the research agent to facilitate targeted literature searches and summarize relevant findings about drugs and diseases (Table S2 in Multimedia Appendix 4). Multimedia Appendix 4 provides step-by-step workflows, intermediate outputs, and user interface snapshots for each use case.

ChatDRex has the ability to identify disease-related gene modules in biological networks using its integrated tools. Starting with the disease term colorectal cancer, ChatDRex retrieved a curated seed gene set of 38 canonical colorectal cancer genes from NeDRex, including genes involved in Wnt signaling, such as APC, and in DNA repair and damage response, such as MLH1, MSH2, and TP53. These genes served as disease-associated seeds for network expansion.

The DIAMOnD algorithm expanded the seed set by prioritizing genes with significant connectivity to the seed genes within the NeDRex protein-protein interaction network, yielding a disease module of 138 nodes and 2129 edges, which comprised the 38 seed genes and 100 genes added by DIAMOnD. Functional interpretation of the top 10 genes was executed via the ChatDRex DIGEST agent, which evaluated functional coherence by comparing observed annotation consistency against randomized gene sets. The returned DIGEST coherence scores indicated significant signals for cellular compartments (cytoplasm, nucleus, nucleoplasm, cytosol, and chromatin) and for cancer-related signaling pathways (pathways in cancer, Ras signaling, and PI3K-Akt signaling). The empirical P values were .000999 for both the Gene Ontology (GO) Cellular Component and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway annotation layers and .002 for GO Biological Process, which had been derived from permutation-based reference distributions. Given the broad nature of the most frequent cellular component terms and the exploratory nature of permutation-based coherence testing, these results can be interpreted as indicative rather than confirmatory, but they are consistent with biological processes previously implicated in colorectal cancer.

The top 10 genes (AKT1, SMARCA4, FBXW7, SMAD1, FANCC, PTPN11, FGF23, SMAD4, FGF6, and PTEN) emerged as highly connected genes within the expanded module, collectively spanning PI3K-Akt signaling, growth factor signaling, TGF-β/SMAD signaling, chromatin remodeling, and DNA repair. These genes represent candidates for further investigation, as their network positions suggest they may integrate multiple layers of colorectal cancer pathogenesis. However, direct interactions among them could only be partially verified in the NeDRex KG and the literature.

ChatDRex demonstrates that a conversational, multi-agent architecture can reliably orchestrate established network medicine tools for drug repurposing without requiring users to interact with the underlying pipelines directly. Across all evaluated components, network expansion, drug prioritization, and KG querying, ChatDRex achieved consistently high tool and call accuracies, indicating that the system correctly identifies and invokes the appropriate backend methods in response to user prompts. The resulting outputs are largely determined by validated, deterministic algorithms rather than by unconstrained model generation, which accounts for the stability and reproducibility observed across repeated runs.

Reliability, however, differs across pipeline stages. For network expansion and prioritization, performance reflects correct orchestration of deterministic backend methods, and the outcome is directly determined by the returned tool outputs. For DIGEST-based functional analysis, we observed comparatively weaker performance. This task requires the model to abstract structured coherence and enrichment statistics into a natural language summary, a genuine generative step that the other tools do not involve. This could impact the results.

ChatDRex sits at the intersection of 2 lines of work that have so far developed independently. Network medicine platforms, such as NeDRex, provide validated, deterministic methods for disease module identification and drug prioritization but presuppose that the user can appropriately use the interface, assemble a parameterized pipeline, and interpret raw network output. Their expressive power is therefore accessible primarily to bioinformaticians rather than clinicians and experimental biologists who generate the underlying hypotheses.

Conversely, general-purpose LLMs and biomedical LLM assistants offer a natural language interface but lack grounding in curated biomedical KGs. Without schema-constrained query generation and validated analytical backends, such systems are prone to unverifiable output and cannot guarantee that a returned drug candidate derives from an established prioritization method. Recent agentic systems in biomedicine have begun to couple LLMs with external tools but typically evaluate end-to-end answer quality without isolating tool selection from result interpretation.

ChatDRex differs in coupling both components explicitly. User prompts are translated into schema-constrained Cypher queries to retrieve relevant information from the NeDRex KG, while downstream analyses are performed by established tools, such as DIAMOnD, TrustRank, closeness centrality, and DIGEST, rather than by the model.

There are some limitations. First, a key limitation is that our evaluation establishes only that ChatDRex reliably selects and invokes the underlying methods and faithfully reports their outputs—not that the resulting candidates are clinically valid. We performed no retrospective validation against known drug-disease indications and tested no candidates experimentally. ChatDRex’s predictive validity for actual clinical drug repurposing therefore remains unverified, and its outputs should be regarded as hypothesis-generating rather than clinically actionable.

Second, the NeDRex KG benchmark may be affected by generation bias within the model family. The SSAD used to evaluate natural language-to-Cypher query generation was created with the help of several language models, including qwen2.5-coder:7b. It was then manually reviewed and corrected. Since qwen3.6:27b was the top performer in the NeDRex KG evaluation, its strong performance may be partly due to its alignment with query structures, wording patterns, or Cypher syntax produced by a related model family. This potential confounding factor should be considered when interpreting the relative model ranking.

Third, Drugst.One was not tested as part of the quantitative agent evaluation because it primarily provides a visual representation layer without adding contextual or analytical functions directly relevant to the agent’s reasoning and decision-making capabilities.

Fourth, all automated assessments of response quality were performed by a single judge model (gpt-oss:20b), which was also among the evaluated systems, and thus, self-preference bias cannot be excluded for that configuration. Independent models could provide a more reliable assessment. We took this into account by manually reviewing the responses and correcting them where necessary.

Fifth, the pilot cohort consisted of 7 bioinformaticians and a single clinician, most with intermediate to advanced programming experience. The results reflect usability for computationally experienced users and do not yet validate no-code usability for novice or noncomputational users.

Several directions follow directly from the limitations above. First, prospective validation is required: candidate drugs prioritized by ChatDRex for a given disease module should be assessed against experimental readouts or, where available, retrospective clinical evidence, in order to establish predictive rather than procedural validity.

Second, the contribution of the visualization layer warrants quantitative assessment. Although Drugst.One does not participate in the agent’s reasoning, its effect on user comprehension and hypothesis formation can be measured through an extended usability study.

Finally, the step where the LLM turns DIGEST’s statistical results into an explanation offers the clearest room for improvement, as this is where the most errors appear. Approaches that better structure or constrain this interpretation step, potentially through task-specific fine-tuning, could reduce these errors while preserving the reliable and reproducible results delivered by the deterministic DIGEST tool.

By automating query formulation, network construction, prioritization, visualization, and literature grounding, ChatDRex can shoulder a substantial portion of analytical work, allowing scientists to focus on interpretation, hypothesis refinement, and experimental design. It thereby lowers the technical barrier to network-based drug repurposing without displacing the validated methods on which such analyses depend.