Clinical Laboratory Parameter–Driven Machine Learning for Participant Selection in Bioequivalence Studies Among Patients With Gastric Cancer: Framework Development and Validation Study

Introduction

Inadequate participant recruitment may lead to failure in a clinical trial, ultimately delaying new drug development and increasing costs [1-7]. Tasks comprising the recruitment process, such as prescreening, consent, screening, and communication between the participant and the study staff for the clinical trial, are inevitably labor intensive [2,3,8-10]. For instance, researchers manually review large volumes of electronic medical records (EMRs) during the prescreening process to identify potential candidates. Despite advances in the methodologies for each process in clinical trials, a systematic approach for enhancing the efficacy of participant enrollment is lacking.

Artificial intelligence (AI) has various applications in every section of the industry, and a machine learning (ML)–based optimal design of clinical trials has entered the sphere of pharmaceuticals [11]. Several AI techniques for identifying, screening, and enrolling appropriate participants for clinical trials have been introduced, and some have been employed commercially [1,4,8].

Corporates such as Mendel, Deep 6 AI, and Antidote have developed and provided AI solutions for clinical trial recruitment [12-16]. These commercial services use massive data such as demographics, laboratory, imaging, and multiomics data to facilitate faster recruitment and provide full-service recruitment [14,15]. Jin et al [17] reported that a result from a user study using large language model framework, TrialGPT, to support patient matching resulted in a 42.6% decrease in the screening time [18]. Another AI approach is clinical trial digital twin technology [18-21]. Digital twin technology creates virtual patients that replicate individual characteristics, enabling the prediction of clinical responses [18,19,21]. By utilizing digital twins, the required sample sizes for clinical trials can be reduced [18,19,21]. These informatics-based tools are expected to improve recruitment efficiency, directly influencing the success rate of clinical trials [1,4,5].

Eligibility criteria specify the qualification of participants in clinical trials, and this component comprises structured information, including diagnoses and laboratory results, and unstructured information, such as clinical free text. Previous AI studies have primarily focused on developing advanced natural language processing and optical character recognition techniques for extracting unstructured data [2,3,6,8-10]. As AI technology is rapidly changing, most research and services have developed AI tools using both structured and unstructured data [12,14-16,22]. However, complicated preprocesses (eg, annotation), discrepancies between EMR systems and databases of institutions (eg, noninteroperable algorithms), and high costs limit their application. Moreover, highly sophisticated eligibility criteria can render the generalization of algorithms challenging.

In contrast, clinical laboratory test data and typical structured data can be readily incorporated into an EMR-utilizing algorithm, given that these data are comparatively more objective than other unstructured data [23,24]. Mohammad et al [11] have suggested that disclosing the pretest probability of a specific test result can be clinically meaningful for some laboratory tests.

In bioequivalence studies of drugs, clinical laboratory tests such as hematology and blood chemistry tests are included as eligibility criteria. Participants considered suitable by the investigator based on laboratory results are allowed to take part in the bioequivalence study. Therefore, prescreening through clinical laboratory tests is considered to be effective in quickly selecting potential candidates.

Accordingly, we formulated an ML-based framework to identify participants using the clinical laboratory test values of candidates. In this study, we chose to compare the ML-based method with a random selection method, which we considered representative of the common practice in clinical settings where patient lists are screened sequentially. Collectively, the objective of this study was to develop a simple and rapidly applicable ML algorithm that could assist in participant identification for bioequivalence study enrollment.

Methods

Study Design

In total, records of 11,592 patients with gastric cancer were acquired from the EMRs of Kyungpook National University Hospital in Korea from 2011 to 2019. Clinical laboratory parameters (including complete blood count and liver and kidney function tests) and acquisition dates were acquired to develop the ML-based model to predict relevant laboratory data of patients on a future date. The laboratory parameters selected were those used in the eligibility criteria for bioequivalence testing of anticancer drugs in patients with gastric cancer. We developed the model using 8 basic and straightforward parameters such as complete blood count (hemoglobin, neutrophil count, platelet count), liver function tests (total bilirubin, aspartate aminotransferase, alanine aminotransferase, alkaline phosphatase), and kidney function tests (creatinine), which are minimal tests capable of assessing health status. These parameters enable a simple and efficient preliminary assessment of potential screening candidates.

“Label 1” was assigned when the candidate’s clinical laboratory data fell within the valid range, while “label 0” indicated data outside the valid range (Table 1). A patient was considered a valid candidate for a clinical trial only if all predicted laboratory data met the eligibility criteria. To design an ML-based candidate selection method, the dataset was divided into training and test sets; data collected from 2011 to 2018 were used for training, while data from 2019 were reserved for testing (Table 1). This time-based split ensured a fair evaluation of the model’s performance by avoiding any temporal overlap, thereby reflecting real-world scenarios where future data (test data) would not be available during model training. Additionally, this approach allowed us to assess the model’s ability to generalize unseen data while accounting for potential temporal effects, such as advances in medical technology or shifts in population health trends. By adopting this methodology, we aimed to provide a realistic evaluation of the model’s performance in clinical settings.

Ethical Considerations

This study was approved by the Kyungpook National University Hospital Institutional Review Board (KNUH 2020-04-023), and a waiver of consent was authorized. All research was performed in accordance with the guidelines of the Declaration of Helsinki, and only deidentified data were used and analyzed in our retrospective study.

Data Preprocessing

We attempted to resolve the issues of aperiodicity and imbalance in the training dataset by applying the combination method and principal component analysis (PCA). First, to analyze the trend of laboratory data changes, at least 3 sequential data points of laboratory data and acquisition dates were required. However, given the nonperiodical patient visits to the hospital, the intervals between each data point were inconsistent. To compensate for the aperiodicity of the data, the distribution of data points was increased by computing a simple combination method as follows:

${\displaystyle C(n,k)=\frac{P\left(n,k\right)}{k!}=\frac{n!}{\left(n-k\right)!k!}}$ (1)

where P indicates the permutation function, n is the number of data points, and k is the number of selected sequential data points (4 in this study). Therefore, 4 sequential data points with 8 values, comprising 6 initial values to analyze the trend of data changes and 2 values as target data to verify the analysis model, were generated as one data group. For example, if a patient had 10 laboratory data points, 210 data groups, including 8 values, were augmented. Each laboratory data value was normalized by the minimum and maximum values of data distribution, and the acquisition date values are represented by |acquisition date − a screening date| / N, where N is a normalization factor. Finally, each data group was encoded by considering whether the last 2 values were within the valid range. Table S1 in Multimedia Appendix 1 presents the number of training data points obtained using the combination method.

The augmented training dataset was intuitively visualized by applying the first principal component, as shown in Figure 1. Although Equation 1 helps resolve aperiodicity, the problem of nonuniform distribution in the dataset persisted in both valid and invalid classes. Thus, to compensate for the nonuniform distribution of training data groups, we normalized the imbalanced data into 100,000 even training data points of each laboratory parameter by dividing the data distribution into 100 regions and randomly extracting 5000 data points from each region (Figure 1). Moreover, we attempted to correct the data imbalance by selecting the same number of data points for each label class. Table S2 in Multimedia Appendix 1 lists the numbers of training and test datasets.

‎

ML-Based Selection Method

There are various models in ML, and the performance of each model can vary greatly depending on how the elements of the process, such as data preprocessing, feature engineering, model selection, hyperparameter tuning, and ensembling, are configured. Therefore, specialized knowledge and experience are required, making it difficult for nonexperts to use the same skills as experts. To solve this problem, automated ML (AutoML) simplifies and automates the mentioned complex processes, so that nonexperts can easily use ML and expect the same performance as experts.

AutoGluon-Tabular is an AutoML and an open-source library developed by Amazon Web Services [25]. Figure 2 shows the architecture in this study that utilizes it. The input data is applied with various ML models (KNeighbors, random forest, Extra Trees, Light Gradient-Boosting Machine [LightGBM], Extreme Gradient Boosting [XGBoost], CatBoost, and NeuralNet) and a stacked ensemble technology that combines them, and the output is the probability value that predicts whether the patient is valid.

‎

Results

The present study consisted of two parts: (1) the generalization performance of the ML-based model was confirmed with various evaluation metrics, including accuracy, sensitivity, specificity, precision, F₁-score, and area under the curve (AUC); and (2) the proposed model was compared using a random selection method to evaluate its efficacy in participant recruitment.

The training results for each model were evaluated as F₁-scores (Table 2). The weighted ensemble model resulted in the highest average F₁-score of 0.91 (SD 0.076). The weighted ensemble model exhibited the best performance in terms of precision and recall. Among the same ML models, detailed models were classified according to parameters such as the size and calculation method. As shown in Table 2, we observed that the F₁-score for the weighted ensemble model was over 0.8 during the training process, indicating strong performance. An F₁-score above 0.8 is generally considered strong, as it reflects a good balance between precision and recall, which is crucial in clinical applications where both false positives and false negatives can have significant consequences. Precision measures how many of the predicted valid candidates are truly valid, while recall reflects how many of the actual valid candidates were correctly identified by the model. A high F₁-score ensures that the model is not only accurate in predicting valid candidates but also minimizes the risk of misclassifying candidates who could otherwise be eligible for clinical trials.

We evaluated the performance of the classification task of the ML algorithm (Table 3). The receiver operating characteristic curves for each clinical parameter are shown in Figure S1 in Multimedia Appendix 1. In Table 3, the overall AUC value exceeded 0.8, demonstrating high performance. An AUC value exceeding 0.8 is considered high, indicating that the model has a strong ability to discriminate between valid and invalid candidates. The AUC measures the model’s ability to distinguish between eligible and noneligible patients, and an AUC value closer to 1 signifies excellent performance. This is particularly important in clinical settings, where the model’s ability to prioritize the right candidates can improve the efficiency and accuracy of the screening process. What we found particularly noteworthy in the results shown in Table 3 was the high sensitivity. A higher sensitivity means that the probability of correctly identifying patients who fall within the valid range is increased. We can prioritize patients with a higher probability of being in the valid range based on model predictions, and by sequentially conducting additional tests, quickly screen for an appropriate and statistically significant sample size for clinical trials, ultimately contributing to a more efficient and successful trial.

For the application test, we evaluated the effectiveness of the proposed model for identifying eligible patients from the test dataset. First, assuming prescreening dates from December 1, 2019, to December 31, 2019, laboratory data were collected from 2019. One test dataset was constructed by retrieving the last 3 laboratory data points per patient during the period. Table 4 presents the number of clinical trial participants used for the application test. In the actual hospital EMR, no alkaline phosphatase test data were available for patients with gastric cancer during this period. Cases were classified according to their valid ratio to the total. We designated the parameters as case 1 when the number of valid data was <50%, case 2 when it ranged between 50% and 60%, and case 3 when it was >60%.

Figure 3 shows the number of valid and invalid patients by probability from 0 to 1. The number of invalid patients by probability was first drawn as a histogram, and then the valid patients were drawn on top of it by accumulating them. As can be seen in the results, there were fewer valid patients and more invalid patients on the side where the predicted probability was close to 0, while there were more valid patients and fewer invalid patients on the side where the probability was close to 1. Through this, we can see that the results predicted by ML are reliable, and it is advantageous to select patients with high probability values for patient screening.

Finally, valid candidates were identified by extracting patients in the order of highest predicted probability from the proposed model, and the results were compared with random findings. Figure 4 shows the results concerning the identification of valid patients for clinical trials. For example, in the case of hemoglobin, we aimed to identify 150 clinically suitable patients out of 673 total patients, where 203 patients met the eligibility criteria. As shown in Figure 4, our proposed model identified 150 valid patients by screening only 209 high-probability candidates. In contrast, a random selection process required screening 485 patients to find 150 valid patients. This demonstrates an approximate 57% reduction in workload.

Figure 5 shows a comparison of clinical trial participants in the order of probability of the proposed model and random method. Compared with the random selection method, more valid patients were distributed at the top of the proposed result, and more invalid patients were distributed at the bottom. When selecting patients for clinical trials, if the patients sorted by the proposed method are assigned from the top, it is possible to determine suitable target patients faster than with the random method.

Overall, the proposed ML model identified valid participants for clinical trials faster than the random selection method. In particular, as seen in the case of hemoglobin and creatinine, in case 1 and case 2, that is, when the valid rate was <60%, the proposed model identified valid participants considerably faster.

‎

‎

‎

Discussion

Principal Findings

We developed an ML-based framework to support the prescreening of clinical research using clinical laboratory tests to reduce the workload of reviewing structured data of eligibility criteria. We developed a protypical but practical framework using clinical laboratory data and achieved high accuracy and a 57% workload reduction compared to random selection when identifying 150 valid patients. Based on our results, we expect that the framework will potentially reduce investigators’ burden and shorten the recruitment period by assisting with conventional manual prescreening.

Prescreening involves the process of selecting candidates for screening, a cumbersome manual task that requires a heavy burden of research personnel at this stage [2,3,8-10]. The most efficient known method for determining eligibility criteria–matching candidates is as follows: candidates are randomly prioritized by research staff, and one staff member is selected to review and narrow the potential list for screening. Although several AI-based tools for assisting the recruitment of participants have been introduced, there are a few tools to support prescreening [1], and relevant studies using only structured information, such as laboratory results, are limited. Recently developed AI-based recruitment services offer a significant advantage in performing precise and rapid clinical trial participant matching by incorporating both structured and unstructured data. However, these services can be costly to implement. In contrast, our study suggests a practical alternative to more comprehensive but potentially cost-prohibitive AI-based recruitment services by showing that efficient prescreening can be conducted using laboratory data alone. This approach provides a more accessible and cost-effective method for individual researchers to apply in their bioequivalence studies.

Considering the framework of this study, the primary contribution is the establishment of a ranked list of potential participants for trial enrollment in an intuitive manner. A simple list of potential candidates can be valuable in informing the decision to review additional eligibility criteria, thereby alleviating the burden of manual prescreening and preventing underestimation due to manual review. Advantages of the framework include simple and time-saving characteristics, the requirement for only a few laboratory data among numerous EMR data, and effortless implementation through AutoGluon-Tabular. The utilization of programs such as AutoGluon-Tabular makes it easy for beginners to employ ML. In addition, ML with techniques tuned for performance improvement can be utilized by experts with minimal time and effort, making our approach more accessible.

We speculate that the irregularity of patient visiting schedules and the imbalance of the clinical laboratory parameters remain major driving limitations in the present ML-based framework. To overcome these defects, we carefully designed the algorithm using the PCA method (Table S2 in Multimedia Appendix 1 and Figure 1). However, further research is required to refine the solutions to combat aperiodicity and imbalanced data.

The proposed system can be integrated into clinical trial recruitment workflows as a decision-support tool for the prescreening process. Currently, prescreening candidates for clinical trials often involves manually reviewing large volumes of patient data to determine eligibility. By automating the selection process, our system can generate a ranked list of potential participants based on clinical laboratory parameters, allowing research staff to focus their manual review efforts on high-priority candidates. This integration could be achieved by embedding the system into existing EMR systems. The framework can process patient data directly from the EMR, analyze the eligibility criteria, and provide a prioritized candidate list in real time. Additionally, the ranked list can be updated dynamically as new patient data becomes available, ensuring that the recruitment process is both efficient and adaptive to changing conditions.

The clinical data used in this study were collected from a specific institution (Kyungpook National University Hospital) and from patients with a specific condition, which may limit the generalizability of the findings. Therefore, further diversification of the data and additional validation are necessary. Moreover, in terms of ethics, AI models should not replace human judgment in the clinical trial participant selection process, but rather serve as an assistive tool for initial screening. In addition to the limitations of generalizability due to the data source, integrating this system into clinical workflows may encounter challenges such as compatibility with diverse EMR platforms, requiring standardized data formats and robust interoperability. Training research staff to effectively use the system and ensuring transparency in AI decision-making processes will be critical for fostering trust and adoption. Moreover, ethical and regulatory compliance regarding patient data usage must be carefully managed to address privacy concerns and uphold clinical trial standards

Our future work will focus on improving the generalization of the framework. First, we plan to apply this method to screen patients for bioequivalence studies involving cancers other than gastric cancer. Bioequivalence studies of anticancer drugs are mostly conducted on patients under follow-up after curative treatment and are performed to demonstrate the equivalence of bioavailability. Therefore, compared to clinical trials aimed at evaluating efficacy, the eligibility criteria are not quite complicated. Additionally, the presence of similar criteria makes the framework developed in this study likely to be easily generalizable.

Second, while researchers typically utilize both structured and unstructured data in the screening process, this study is limited to the use of structured data. However, we expect that screening with an ML-based framework, followed by incorporating unstructured data, could help reduce the overall workload. We aim to enhance the framework to integrate unstructured data for a more comprehensive screening process. Third, we recognize the importance of comparing our approach to existing methods, and plan to include such comparisons in future research. In this study, we targeted structured information from patients; considering unstructured and structured information may therefore enhance the generalization of the framework, covering comprehensive eligibility criteria. In addition, we plan to conduct a prospective study to compare the proposed framework with conventional methods. Collectively, we anticipate that the framework will cover a wide disease spectrum and expand its applicability to clinical trials from other institutions.

Conclusion

We proposed an AI support framework utilizing structured information on eligibility criteria to select appropriate candidates for clinical trial enrollment. This method could accelerate the efficiency of prescreening processes and can be applied to various clinical trials.