Childhood mortality rates have declined globally. However, the risk of a child dying before the age of one year is highest in Africa at 51 per 1000 live births compared to 8 per 1000 births in the European region [1]. Currently, the probability of a child dying between birth and 5 years in Uganda is 53 per 1000 live births [2], while Kenya reported 41 deaths per 1000 live births [3]. Vaccine-preventable diseases (VPDs) account for 29% of deaths among children aged 1‐59 months and an estimated 3 million deaths every year, mainly in Africa and Asia [4].

Measles vaccination marks a great milestone as it is the last administered among the basic antigens [3] (see Table S1 in Multimedia Appendix 1). Measles is an acute viral VPD whose only reservoir is humans [5]. It is transmitted through the air or direct contact with an infected person [5]. With an incubation period of 7‐21 days [5], the World Health Organization indicates that up to 3 million measles cases are recorded annually [6]. Measles is one of the main causes of death among infants in the prevaccine period due to complications from the respiratory tract (pneumonia), the central nervous system (meningitis, encephalitis, meningoencephalitis, and encephalomyelitis), and the gastrointestinal tract (diarrhea) [6]. Despite free routine immunization (RI), many infants are zero-dose or drop out of the immunization schedule [7]. This situation was compounded by the COVID-19 pandemic, which impaired RI services in African countries, in that only 3 countries met the target of 95% first measles vaccination coverage in 2020 [5]. From January to March 2022, about 17,500 cases of measles were recorded in Africa, while 20 African countries reported measles outbreaks in the first quarter of 2022 [5].

Uganda experienced measles outbreaks in 76 districts in 2018 [8], January 2020 in Nakaseke [5], and February-June 2023 in Bundibugyo [9], among others. These outbreaks reflect the existence of communities with partially vaccinated or zero-dose infants whose herd immunity is not high enough to stall disease transmission [10]. Similarly, in Kenya, 2% of children aged 12‐23 months received no vaccinations [3].

Real-time monitoring of RI enables identification of underserved communities; zero-dose infants are likely to drop out of the immunization schedule [11]. Conventional defaulter identification methods, like resource-based approach (RBA) [12], rapid coverage monitoring (RCM) [13], administrative estimates [11], and surveys, are expensive to implement regularly [11,14], propagate data falsification [15], overestimate uptake [11,12,14], and are not proactive.

Several studies [16-18] have used individual models to predict RI defaulters, predominantly leveraging child and maternal characteristics as predictors, yet overlooking community, socioeconomic, and demographic factors. Some never applied resampling techniques like cross-validation to reduce overfitting or data balancing during training of an imbalanced dataset distribution. In addition, the studies lacked the application of interpretable deep learning methods, significant feature quantifying, and recommending remedial intervention strategies at a national level in Uganda and Kenya concerning measles vaccination defaulting. We claim that using advanced ensemble learning to predict immunization defaulters may reduce the risk of outbreaks and infant mortality.

Nantongo et al established that immunization defaulting increases as the child’s age increases and worsens with measles [19]. The study also indicated the reception of preceding vaccines among predictors for measles in Uganda. However, the model performance for measles was the worst among the four antigens. Nantongo et al [19] did not explore an interpretable deep neural network approach and an advanced ensemble technique; yet, interpretability was key in public health. Neither did it establish if the same predictors would influence measles defaulting in another geographical area other than Uganda, for the models’ generalizability. Accordingly, this study leverages literature to (1) increase predictors’ diversity, (2) explore a comparative study between Uganda and Kenya, (3) experiment with custom soft voting and stacking architectures, (4) experiment performance using interpretable deep learning methods, specifically TabNet, and (5) evaluate performance using a confusion matrix, Shapley additive explanations (SHAP), in addition to other metrics to improve interpretability, to guide public health officers in both countries to mitigate measles defaulting.

Traditional machine learning models for tabular data, like tree-based ensembles such as random forest (RF) and gradient boosting (GB), among others, are considered the top-performing models [20]. However, deep learning models like multilayer perceptron (MLP) and Tabular Network (TabNet) have been presented to perform better. MLP is a general-purpose “black-box” neural network characterized by multiple layers where every neuron in one layer is connected to every neuron in the next layer, and is applicable to many data types, but traditionally structured for tabular data due to its simple linear flow.

TabNet is a high-performance interpretable deep neural network for tabular data with feature selection capabilities [20]. Unlike traditional neural networks that process features all at once, TabNet has a sequential attention mechanism that learns to select a subset of the most relevant features to reason from at every decision step, providing local and global interpretability [20] (Figure 1). The TabNet architecture consists of an encoder that consists of a feature transformer, an attentive transformer, and feature masking, and a decoder for self-supervised learning [20]. TabNet has illustrated superior performance over traditional machine learning models [20,21], hence its application in this study.

In the medical field, a comparative performance assessment of a multilevel ensemble learning model to detect diabetes by [22] achieved a maximum accuracy of 88.3% compared to individual models experimented with, and the study suggested the use of synthetic minority over-sampling technique (SMOTE) on imbalanced classification datasets for future work [22]. Similarly, a study [23] concluded that the ensemble learning method was more suitable for handling the classification problems in the bioinformatics domain since it gave better accuracy with a lower root-mean square error.

Correspondingly, another study [24] experimented with stacking linear models, support vector regression (SVR) with a linear kernel, and SVR with a Gaussian kernel. Their results revealed that stack ensemble learning effectively predicted vaccination uptake using web search queries and clinical data for Denmark with a 4.7 root-mean-squared error. In addition, a study forecasting monkeypox transmission in different continents proposed stacking ensemble learning of adaptive boosting (AB) regression, GB regression, RF regression, ordinary least square regression, least absolute shrinkage selection operator regression, and ridge regression. Experimental results revealed that the stacking ensemble learning outperformed other ML approaches [25].

The study by Nantongo et al [19] used demographic, socioeconomic, maternal, community, and child characteristics as predictors. It also experimented with a model with cross-validation and balanced distribution, indicating RF as the best model among the 9 at 84% accuracy in predicting measles vaccination defaulting [19]. As mentioned in the preceding section, there is limited research regarding comparing Uganda and Kenya with reference to measles vaccination defaulting, use of interpretable ensemble deep learning methods to improve model performance with predictor diversity, quantifying significant measles vaccine default predictors with a clear explanation of feature influence on the model outcome, and remedial intervention strategies for policymakers in Uganda and Kenya contexts at a national level. Accordingly, this study aims to predict measles vaccine defaulters in Uganda and Kenya using advanced ensemble learning methods with interpretable influential predictors and context-specific remedial intervention strategies.

Ensemble methods achieve high accuracy [26]; yet, they require little preprocessing [27]. These methods merge base models’ individual predictions to output a single one using several techniques [28]. The techniques include voting classifiers, bagging and pasting, RFs, boosting, and stacking ensembles. Voting and stacking are nongenerative, while boosting and bagging are generative [28]. The voting classifier employs diverse algorithms trained in parallel and uses either hard or soft voting [27]. Another approach is using the same training algorithm for every prediction, but training them on different random subsets of the training set. Sampling may be done with replacement (bagging) or without (pasting). The RF is an ensemble of decision tree classifiers that uses bagging. Boosting, on the other hand, combines several weak learners into a stronger one trained sequentially, each trying to correct its predecessor [27,29,30].

Stacking is another ensemble method that trains a model to perform prediction aggregation from different base models as inputs for the final predictor (meta learner) [27]. This study experimented with TabNet stack and custom soft voting architectures to predict measles vaccination defaulters in Uganda and Kenya, interpreting influential predictors and suggesting country-tailored remedial intervention strategies for mitigating measles defaulting.

Study experiments were conducted using the data collected nationally from the Demographic and Health Survey (DHS) 2022 for Kenya and 2016 for Uganda. DHS data collection uses standardized questionnaires and modules. The study was a retrospective cohort study of infants (living) that were born between 2020‐2022 and 2013‐2016 for Kenya and Uganda, respectively. Although there is a 6-year difference in 2016 Uganda DHS (UDHS) and 2022 Kenya DHS (KDHS), currently the 10% of the 2022 UDHS dataset available [31] for public use may not provide representative estimates at the national level. Experiments under the same conditions were run separately on the country-specific datasets. The study considered immunization uptake of the measles first vaccination administered at 9 months, which marks a great milestone of completion of the basic childhood RI schedule for infants under one year, as indicated in Table S1 in Multimedia Appendix 1.

The 7th KDHS was conducted by the Kenya National Bureau of Statistics (KNBS) in 2022 [3], while the 6th UDHS was conducted by the Uganda Bureau of Statistics (UBOS) in collaboration with the Ministry of Health from June 15 to December 18, 2016 [32]. KDHS 2022 and UDHS 2016 were the most recent national representative datasets at the time of this study’s experiments. The DHS provides a comprehensive overview of population, maternal, and child health issues, and the data are accessible on request to all stakeholders for policy formulation, planning, and measuring progress [32]. The KDHS 2022 collected new indicators not collected in earlier KDHS surveys, like early childhood development, health expenditures, and information on chronic diseases [3], but it maintained the standard indicators for childhood immunization, enabling the current comparative study. Among the many survey objectives, DHS obtains data on many key aspects, like maternal and child health, including immunization coverage among children [32]. The sampling frame used for the 2016 UDHS was the frame of the Uganda National Population and Housing Census (NPHC), conducted in 2014. The census frame was a complete list of all census enumeration areas (EAs) or stratas created for the 2014 NPHC [32]. In Uganda, an EA is a geographic area that covers an average of 130 households [32]. The sample design for the 2022 KDHS provided estimates at the national level, for both urban and rural areas in 47 counties [3]. For purposes of comparative analysis in this study, a county in Kenya corresponds to a district in Uganda. Figure 2 illustrates the architecture of the proposed ensemble.

Evaluation establishes the level of excellence achieved by a proposed model [45]. This study employed metrics, including: accuracy, recall, precision, area under the receiver operating characteristic curve (AUROC), and confusion matrix.

Where TP is true positive, FP is false positive, TN is true negative, and FN is false negative.

The DHS program granted the use of UDHS and KDHS datasets in accordance with the Institutional Review Board (IRB) approved procedures for DHS public use datasets (see ethical approvals in Multimedia Appendix 1). Human participant consent is not applicable in this study since secondary datasets were used (see ethical approvals in Multimedia Appendix 1).

Models experimented with TabNet, RF, MLP, LR, GB, decision trees (DT), Support vector machine (SVM), and AB among others.

In Table 3, the SD for all metrics was very low, with the highest (0.0246) being in the acceptable range. This revealed a low variability and consistent performance across the five folds; therefore, models may generalize well to unseen data. GB was the best performing model with an F₁-score of 0.7034 for Kenya, closely followed by SVM (F₁-score=0.7013). For Uganda, RF outperformed others (F₁-score=0.6520), closely followed by GB (F₁-score=0.6495). Similarly, GB and RF had the highest accuracy, precision, and AUROC for the respective countries.

When comparing models, this study’s primary focus was to achieve a high TP (predict the highest number of defaulters), minimal FN, which represents the zero dose of the vaccination schedule, potential dropouts that act as reservoirs for VPD, and minimizing false alarms (see Table 4).

Table 5 illustrates that GB outperformed other models by predicting 79% (267/337) of all defaulters and 18% (169/944) false alarms for Kenya. For Uganda, NB had the highest predicted defaulters at 78% (236/301); however, it had a high 47% (493/1058) false alarms. On the other hand, AB at 71% (215/301) TP and 14% (150/1058) false alarms closely followed TabNet and GB. Figure 4 and Table 6 present the Uganda and Kenya hyperparameter tuning results.

An improvement in model performance is observed in 7/9 models across all metrics after hyperparameter tuning. For example, TabNet had an improved F₁-score of 2.5% (Kenya) and 1.41% (Uganda), RF at 1.89% (Kenya) and 1.57% (Uganda); DT at 11.72% (Kenya) and 9.77% (Uganda), among others, while only KNN and LR for Kenya performed best at default parameters. However, we noticed the reduced FN (defaulters; Tables 7 and 8); there was a high number of FP (false alarms) that necessitated further model refinement.

Table 9 revealed that the advanced ensemble (custom soft voting and TabNet stacking) improved performance compared to the best-tuned base models. TabNet+ GB+RF→Soft voting outperformed all stacked ensembles with an area under the curve (AUC) of 0.8847 and F₁-score of 0.7111 at an optimal threshold of 0.5125 for Kenya, while GB + RF+AB→TabNet was the best for Uganda with an AUC of 0.8867 and an F₁-score of 0.6756 at an optimal threshold of 0.5163. Individual models seemed to have a higher accuracy; however, the confusion matrix revealed that the models were failing to identify measles vaccine defaulters (minority) but overwhelmingly predicting the vaccinated (majority class) despite the application of SMOTE-NC in training.

Table 10 revealed that the proposed advanced ensembles GB + RF+AB→TabNet and TabNet + GB+RF→Soft voting with the highest F₁-score correctly classified 3% for Uganda (8/301) and 1% for Kenya (2/337) more of the total TP that were previously misclassified by the best optimized base models, leading to a subtle improvement yet significant reduction in Type II errors (FNs). TabNet + GB+RF→Soft voting in Kenya presented the highest TN and the lowest TP. For Uganda, GB + RF+AB→TabNet had moderate FPs (false alarms). This was a trade-off since FPs are less harmful compared to failure to identify a TP. In the context of this public health study, it is the defaulters, TP and FN (unimmunized) that pose the risk of VPD outbreaks.

Figures 5 and 6 and Table 11 indicate that the current age of the child, vitamin A1, respondent’s education, time to water source, district lived previously, and current district were the most influential global features in Kenya, respectively. In Uganda, vitamin A1, current age of the child, Polio inactive, Pentavalent 3, current district, and respondent’s education were the most influential factors to defaulting, respectively.

The statistical validation performance comparison based on McNemar test between Kenya and Uganda in Table 12 revealed that, in Kenya, a significant marginal performance was gained by TabNet + GB+RF ->Soft Voting over the tuned best base (AB; ΔF₁-score=0.0054; P=.007; χ²₁=7.224; Cohen g [effect size]=0.1716; CI for Δ F₁-score spans zero −0.0156 to 0.0260). The absolute magnitude of improvement remains tight yet nonrandom pattern of corrected errors confirms the advanced ensemble’s structural advantage in Kenya. Conversely, in Uganda, the TabNet hybrid demonstrated that the algorithmic complexity does not universally guarantee statistical superiority. While GB + RF+AB→TabNet exhibited a slightly higher nominal improvement of Δ F₁-score=0.0059, the 95% bootstrap CI again captures zero (−0.0113 to 0.0222). Crucially, the McNemar test registered no statistical significance (χ²₁=0.033; P=.86) with weak effect size (Cohen g) of 0.0333. It reveals that the differences in classification error between the best tuned base model (GB) and the TabNet hybrid may be stochastic. This also indicates that the additional computational overhead of the advanced ensemble architecture fails to provide a meaningful predictive advantage in this specific context.

Experimental results for individual standard models revealed that GB was the best with an F₁-score of 0.7034, closely followed by SVM at 0.7013, AB at 0.6955 for Kenya. For Uganda, RF outperformed others at 0.6520, closely followed by GB at 0.6495. Similarly, GB and RF had the highest accuracy, precision, and AUROC for the respective countries. These results resonate with [19,34,46], who indicated that the tree-based ensembles like RF were the best performing models. In addition, just like [47], tree-based models outperformed deep learning methods. NB was worst at 0.4600 (Uganda) and DT 0.5731 (Kenya).

Similarly, an improved performance among individual models was realized after hyperparameter tuning at a customized threshold. For example, TabNet had an improved (see Table 6) F₁-score of 0.025 (Kenya) and 0.0141 (Uganda); RF at 0.0189 (Kenya) and 0.0157 (Uganda); DT at 0.1172 (Kenya) and 0.0977 (Uganda), among others. On the contrary, KNN and LR for Kenya performed best at standard parameters. Despite the improvement, however, the confusion matrices (Tables 8 and 9) revealed a high number of FP (false alarms) as a trade-off for reduced FN (defaulters), which necessitated further model refinement.

The proposed TabNet + GB+RF→soft voting for Kenya and GB + RF+AB→TabNet for Uganda achieved the highest AUC of 0.8847, F₁-score of 0.7111 and AUC of 0.8867, F₁-score of 0.6756 compared to the best optimized individual models AB AUC of 0.8829, F₁-score of 0.7057 (Kenya) and GB AUC of 0.8829, F₁-score of 0.6697 (Uganda) respectively. Similarly, the confusion matrix revealed that GB + RF+AB→TabNet and TabNet + GB+RF→soft voting correctly classified more TP compared to the best tuned models, that is; 8 of 301 (3%) infants in Uganda and 2 of 337 (1%) in Kenya. Our results resonate with [47], whose study of predicting Pentavalent3 dropout in East Africa revealed that the all-stacked ensemble models outperformed individual learners benchmarking RF, generalized linear model (GLM), and Extreme Gradient Boosting (XGBoost) as meta learners [47].

In Kenya, the advanced ensemble model was statistically significant and supported by a moderate Cohen g effect of 0.1716. Conversely, the performance in Uganda was stochastic indicating that the ensemble architecture provided no statistically valid predictive advantage in that context.

Given the class imbalance in the datasets, the reliance on the default 0.5 threshold classification to determine the binary outcome of defaulter and immunized, assuming a perfectly symmetrical class distribution, would present a vulnerability. Similarly, it assumes an equivalent real-world cost for type I (FP) and type II (FN) errors, which is not the case in this study. A 0.5 threshold normally produces a high but deceptive accuracy biased towards the majority class while failing to identify the minority. To counteract this bias, we implemented the automated “find best threshold,” shifting the decision boundary away from an arbitrary midpoint and aligning it to the mathematical maximum F₁-score.

This pipeline adaptively raised the threshold above the standard 0.5 (TabNet + RF+GB→soft voting at 0.5125 and GB + RF+AB→TabNet at 0.5163), indicating models’ ability to separate the two classes well (see Figure 7). The models are neither completely drowning in the majority class nor suffering from massive probability suppression. It also indicates the models confidently push TP cases past the 0.5 mark without accidentally dragging a flood of FPs along. This trade-off was necessary since FNs are clinically more costly. FN results mean either a zero dose or a defaulter reduces herd immunity. Conversely, a FP is negligible since the child is already immunized, posing no VPD outbreak to the community.

The current study, like the one by Alemayehu et al [47], used SHAP to establish the global feature importance benchmarking the best advanced ensemble models. Experimental results using TabNet + GB+RF→Custom soft voting ensemble indicated current age of a child, most recent vitamin A1, education, time to water source, and district lived previously as the most influential global predictors in Kenya, with a mean absolute SHAP value (MASV) of 0.1795, 0.0747, 0.0427, 0.0333 and 0.0173, respectively. For Uganda, GB + RF+AB→TabNet revealed vitamin A1, age of child, polio inactive, Pentavalent3, and district were the most influential predictors at MASV of 0.0743, 0.0737, 0.0563, 0.0494, and 0.0317, respectively. Generally, the current age of a child and the most recent vitamin A1 administration are key universal predictors, closely followed by the district and the respondent’s education. The respondent’s education often correlates with the ability to understand the significance of adherence to the immunization schedule. On the other hand, as the child gets older, the chances of getting immunized increase, as a previously missed opportunity due to vaccine stockout or parents’ unavailability can be corrected at the following visit.

However, the influence of other features diverges between the two countries, reflecting unique socioeconomic and structural differences. For example, Polio inactive was a key predictor in Uganda and least in Kenya, and similarly, district lived previously was influential in Kenya and least in Uganda. This result reverberates with the study by Alemayehu et al [47], who stated that feature influence is context-dependent and advised that immunization drop-out policies and interventions should be tailored to subgroups or countries.

The proposed advanced ensembles present the opportunity to reduce the zero dose and immunization schedule defaulting since they could accurately predict 76% for Kenya (256/337) and for Uganda (226/301) of the potential defaulters while minimizing the false alarms. They also presented the highest F₁-score, balancing precision and recall. FPs (immunized predicted as unimmunized) may not cause an outbreak or threat, but can raise the budget of immunizing already vaccinated infants, causing a financial loss due to a false alarm. Therefore, models like the proposed advanced ensembles are ideal since they minimize both the FN and FP.

The proposed advanced ensemble learning methods improved performance compared to the best individual base models. In addition, they present an opportunity to reduce the zero dose and scheduled immunization defaulting since they could accurately predict 256 out of 337 (76%) infants in Kenya and 226 out of 301 (75%) infants in Uganda of all the potential defaulters that act as reservoirs for measles and pose a risk of measles outbreaks.

For interpretability, SHAP indicated that the age of a child and the most recent vitamin A1 administration are universal key predictors in both countries. However, other factors varied drastically by country. These findings emphasize the need for context-specific interventions rather than a one-size-fits-all approach to measles defaulting.

Based on the SHAP advanced ensemble explainability, the study recommends distinct and localized public health strategies in the 2 countries. In Kenya, the study recommends outreaches in areas with distant water sources, low literacy, and migrating families. Conversely, in Uganda, where immunization history was a predominant predictor, embedding proactive alerts to would-be defaulters and outreaches to high-default districts are recommended. Since the current age of a child and vitamin A1 were universally crucial drivers in both countries, timely reminders as infants reach their peak window of age vulnerability are ideal.

One of the proposed stack ensemble GB +RF+AB→TabNet was used to develop a pilot prototype application that proactively flags infants at risk of defaulting basic RI in Uganda (Figure 8). This functional, decisive tool can be used to provide targeted support, like sending reminders to parents or guardians whose infant was predicted to have a likelihood of defaulting. In addition, health workers would proactively schedule outreaches according to the concentration of many likely defaulters using the analytics (see district vs immunization status illustration in Figure 8). This can enable health workers to optimize immunization program delivery to high-default areas that can prevent sporadic VPD outbreaks and related mortality. This tool can also be integrated with EPIVAC (expanded program on immunization vaccination e-registry), a digital vaccination registry in Uganda built on District Health Information Software 2 (DHIS2) that captures individual longitudinal records nationally, like immunization schedule and maternal details, among others. Its operationalization can support early warning and response, just like malaria, climate change, machine learning, and modeling on DHIS2. Figure 8 illustrates the TabNet stack ensemble model prototype.

SHAP values represent the advanced ensemble model’s internal predictive logic that was based only on the used features, but the real-world measles defaulting may have other factors that may influence defaulting that this model may not have accounted for. Consequently, the high importance values may reflect training correlations rather than direct epidemiological drivers of defaulting.