DISCOVER is a three-armed RCT examining change in depression severity six months after online screening with tailored, nontailored, or no feedback (control). The study was advertised as a study on stress and psychological well-being [1]. Recruitment strategies involved print and online advertisements on social media platforms and in a nationwide online access survey panel in Germany [1]. Eligible participants were 18 years or older, proficient in German, and screened positive for at least moderate depression severity (Patient Health Questionnaire-9, PHQ-9 ≥10) [1]. Participants with missing data, or those with a diagnosis of or treatment for depression in the previous year were excluded.

Feedback comprised depression screening results, a recommendation to consult a mental health care professional or general practitioner, and further information regarding depression and related treatment based on national guidelines [1,9]. For the tailored feedback group, feedback was adapted according to participants’ symptom profiles, treatment preferences, and available guideline-recommended options [1,9].

Review and approval was provided by the Ethics Committee of the Hamburg Medical Chamber (PV7039) [9]. Participants provided online informed consent covering secondary data analyses [1]. Participants received a € 5 (US $5.85) voucher for compensation upon each completed follow-up. Data was deidentified prior to analysis.

The primary outcome was heterogeneity in depression severity change six months after online screening. Depression severity was measured with the PHQ-9 [10]. Predictors involved baseline depression (PHQ-9), anxiety (Generalized Anxiety Disorder Scale-7; GAD-7 [11]), and somatic symptom severity (Somatic Symptom Scale-8; SSS-8 [12]), health-related quality of life (EuroQoL-5 Dimension-5 Level visual analogue scale [EQ-5D-5 L VAS]) [13], illness beliefs (Brief Illness Perception Questionnaire; B-IPQ [14,15]), patient history, depression-related risk factors, and sociodemographic characteristics (Table S1 in Multimedia Appendix 1).

Causal Forests (CF) [5,6] have previously been used to investigate HTE [8] (Box S1 in Multimedia Appendix 1). CFs estimate conditional average treatment effects (CATE), which approximate individual-level treatment effects (ITE). ITE cannot be inferred directly since only one potential outcome is realized per participant. Thus, CATE are more granular than average treatment effects (ATE) but less granular than ITE.

Two CFs were trained based on either training (tau-forest) or test data (eval-forest), with a random split (1:1). CFs were trained with 2000 trees, a sample fraction of 0.5, a minimum node size of 5, and mtry = 30. ATE and CATE were computed by contrasting intervention (nontailored [1], tailored [2], or any [1/2] feedback) to no feedback (control) with depression severity change as the outcome and a propensity score for treatment allocation (P=.50) [4,5].

To assess the presence of HTE discretely, tau-forest predictions of CATE for the test data were grouped into quartiles. Then, ATE was estimated in each quartile using the eval-forest. To assess the presence of HTE continuously, we computed the area under the targeting operator characteristic curve (AUTOC) and tested for the presence of HTE with a significance test for AUTOC (H₁:AUTOC≠0) [4]. Significance of AUTOC was tested two-sided, with bootstrapped standard errors (n=200 bootstrap replicates).

For a comprehensive overview of model evaluation and sample code, see Box S2 in Multimedia Appendix 1, Sverdrup, Petukhova [4] and Klee [16]. Analyses were conducted using R (version 4.3.1; R Foundation for Statistical Computing) using the grf package [17].

After visual inspection of missingness patterns, 19 participants were removed, assuming missingness at random (Table S2 in Multimedia Appendix 1). In total, 946 participants were eligible for analysis. Participants were aged 18 to 79 years, mean 37.20 (SD 13.98) (Table 1).

We did not find evidence of non-zero ATE in either tau- or eval-forests, suggesting no benefit of providing any form of feedback compared with the control (no feedback) (Table 2).

There was no evidence of HTE when comparing nontailored (Figure 1) or any feedback with control (Figure S4 in Multimedia Appendix 1). However, there was a lower (ie, more favorable) ATE when comparing tailored feedback with control among participants with predicted CATE in the second most favorable quartile regarding depression severity change (Figure S1 in Multimedia Appendix 1).

AUTOC estimates (Figures S2 and S5 in Multimedia Appendix 1) did not suggest the presence of HTE in any comparison: AUTOC −0.48 95% CI −1.62 to 0.67, P=.41, nontailored; AUTOC 0.06, 95% CI −1.21 to 1.33, P=.93 tailored; AUTOC −0.20 (95% CI −1.30 to 0.89, P=.72 any feedback vs control). Allocating feedback based on predicted CATE did not substantially alter ATE (Figure 2). This is consistent with the near-zero AUTOC estimates, and indicates limited potential for altering the effects of feedback mode regarding depression severity change through targeted allocation.

The most important predictors of the tau-forest (comparing nontailored feedback with control) were items assessing depression–related treatment control belief (B-IPQ), trouble relaxing (GAD-7), anxiety severity (GAD-7) and trouble sleeping (SSS-8). For tailored feedback compared with control, items denoting age, somatic symptom (SSS-8) and anxiety (GAD-7) severity and depression-related treatment control belief (B-IPQ) were most important. For any feedback compared with control, illness beliefs (B-IPQ) were most important: treatment and personal control, concern, and emotional response.

Higher treatment control (nontailored feedback vs control) and personal control beliefs (any feedback vs control) were the only predictors significantly associated with a linear approximation of CATE (Table 3 and Tables S3 and S4 in Multimedia Appendix 1). Both predictors were associated with less favorable CATE estimates, suggesting less favorable depression severity change at follow-up, given feedback.

Covariate profiles of the most important predictors are depicted in Figure 3 and Figures S3 and S6 in Multimedia Appendix 1. Overall, a higher treatment control belief was more frequent in the highest (least favorable) CATE quartile. Findings are less clear for remaining top four most important predictors.

Examining model calibration, CATE functions estimated in tau-forests did not significantly contribute to predicting change in depression severity at follow-up above group-level mean prediction (Table S5 in Multimedia Appendix 1).

By applying CF to the DISCOVER online RCT, we did not find evidence for HTE with feedback (tailored, nontailored, or any) regarding change in depression severity six months after screening. As such, no apparent subgroup with an altered response to any type of feedback mode was detected.

First, the selection of participants with at least moderate depression severity increases the likelihood of regression to the mean, which may have impeded the investigation of HTE. Second, generalizability of findings is limited to individuals participating in an online study about psychological well-being, who may exhibit distinct severity trajectories. Third, follow-up time may have been too short to detect variation in severity change, given, for example, the latency of help seeking.

Our findings are in line with previous results showing no significant average benefit of any feedback mode for change in depression severity [1]. Beyond average effects, we investigated within-group harms and benefits in accordance with an evidence-based medicine approach [3]. We complement previous research suggesting no between-group detriments [18] with results suggesting that there are no latent subgroups that vary in their response to feedback. In contrast to previous research, our findings are valid irrespective of a priori defined categorical operationalizations of harmful or beneficial events. We show that included predictors do not alter response to feedback, providing an approximated assessment of individual-level harms and benefits [1,18].

Critically, sensitivity analyses suggested limited calibration of trained models. However, when testing HTE with CF, accurate mean prediction of the primary outcome is a first step necessary to detect deviations from it (ie, HTE). Previous research illustrates the notorious difficulty of predicting future depression courses, even with updated analytic tools [19-21]. Still, by employing a nonparametric method, we provide HTE estimates, that can account for a broad range of heterogeneity mechanisms potentially underlying depressive symptom trajectories and their variation following automated results feedback [8].

Applying CF, we could examine a broad range of predictors to detect HTE. In the absence of evidence for HTE, treatment prioritization based on trained models did not improve ATEs. We did not find evidence for harm or benefit of providing feedback after online depression screening regarding depression severity change after six months. Future studies may test if screening alone prompts behavioral activation and downstream depression severity reduction, considering the observed uniform changes across groups [22].