The overview and workflow of this study are presented in Figure 1. First, we identified and collected words expressing depression. We then scraped and manually labeled posts. Next, the posts in our data set were preprocessed and cleaned. Finally, state-of-the-art NLP algorithms were used for the detection of momentary depressive feelings in posts. This study was performed using Python (version 3.9; Python Software Foundation) and R (R Foundation for Statistical Computing) programming languages with different packages. The data collection, preparation, and ML algorithms used in this study are discussed in the following sections.

To investigate and evaluate the performance of competing algorithms, a 10-fold cross-validation (CV) approach is carried out. This involved randomly partitioning the input data into 2 sets: a CV set (2944/3680, 80% labeled posts) and a test set (736/3680, 20% labeled posts). The CV set was further divided into 10 subsets, allowing us to construct and train 10 distinct models. These models were subsequently evaluated using unseen data. The use of unseen data in CV is crucial for assessing the generalization capability of the models. This evaluation with unseen data helps mitigate the risk of overfitting and provides a more reliable estimation of the algorithm's performance in real-world scenarios [51]. The algorithm’s overall performance was computed by averaging the results obtained from the 10 runs. To assess the performance of competing algorithms to accurately identify posts expressing momentary depressive feelings, 6 evaluation metrics were calculated: area under the curve (AUC), accuracy, sensitivity, specificity, precision, and F₁-score [52].

To optimize algorithm performance, the tuning of hyperparameters emerges as a necessity. Yet, it is important to recognize that a universal approach for hyperparameter selection is not known to exist. In light of this, this study took a multifaceted approach by delving into distinct sets of hyperparameter values for each algorithm. This approach enabled us to carefully identify the configuration that attains high performance. A summary of contextual approaches and their respective hyperparameters used in this study can be found in Table 1.

In contrast to conventional research involving human participants, ethical guidelines pertaining to social media research propose that publicly accessible data (eg, posts publicly posted on X) can be used for research purposes without necessitating supplementary consent or ethics endorsement [53,54]. In this study, we did not interact and intervene with the users whose public posts were collected and analyzed user-generated posts. It is worth noting, however, that any potentially associated identifying personal information (eg, user IDs and URLs) has been carefully eliminated to uphold anonymity and safeguard the privacy of X users.

In this study, various contextual language approaches were used, each with a range of hyperparameters tuned to achieve optimal performance. Multiple iterations of each algorithm were carried out using different hyperparameter configurations. The hyperparameter combinations that yielded the highest performance for each model are summarized in Table 2.

In this study, we undertook both binary and multilabel classifications. Nevertheless, it is noteworthy that the outcomes of the multilabel classification (see Multimedia Appendix 1 for details) were not as encouraging as those achieved in the binary classification task. Our analysis revealed an interesting finding in the context of multilabel classification, namely, that posts expressing depressive feelings, regardless of their intensity or scale, pose a challenge for classification models. The complexity of these posts makes it challenging to achieve precise categorization since they encompass a range of emotional states that may not align neatly with predefined categories. This insight highlights the complexity of classifying nuanced sentiment, particularly in the context of depressive expressions.

The performance of different algorithms in detecting momentary depressive feelings with binary classification is presented in Table 3. Our results indicated that BERT and DistilBERT outperformed in momentary depressive feelings detection and achieved the highest values in almost all performance metrics with AUC values of 95.80% and 96.71%, respectively. Additionally, both algorithms demonstrated high accuracy (96.03% and 97.40%), sensitivity (96.22% and 97.57), specificity (95.83% and 97.22%), precision (95.96% and 97.30%), and F₁-score (96.09% and 97.44%). The performance of traditional ML algorithms was relatively poor with the highest scores achieved by CNN (AUC: 84.81% and accuracy: 84.79%) and BiLSTM (AUC: 79.91% and accuracy: 79.86%). These findings indicated that the transfer learning algorithms performed significantly superior by a substantial margin. For instance, DistilBERT achieved an AUC value nearly 12% points higher than the highest AUC achieved by CNN (84.81%). These findings confirm the feasibility of this algorithm in detecting momentary depressive feelings highlighting the effectiveness of transfer learning algorithms in NLP tasks.

It is important to note that the transfer learning algorithms, especially DistilBERT and BERT, achieved high values in other performance metrics such as sensitivity, specificity, precision, and F₁-score, in addition to overall accuracy. High sensitivity and specificity demonstrate that these algorithms were able to accurately identify posts with momentary depressive feelings while avoiding false positive and false negative predictions. The significant performance variation observed between BERT and its more lightweight counterpart, ALBERT, was an important finding of this study. ALBERT incorporates parameter-reduction techniques, which may impact its ability to capture intricate nuances within the data as effectively as BERT. Furthermore, we have closely examined potential disparities in pretraining strategies and fine-tuning procedures, seeking to identify any factors that might contribute to the observed divergence in performance. By elaborating on these architectural and procedural distinctions, we aim to provide a comprehensive understanding of the reasons underlying BERT’s superior performance over ALBERT. This analysis not only informs this study but also contributes to the broader discourse on the comparative strengths and limitations of these prominent language models.

Overall, these results support the use of transfer learning algorithms for momentary depressive feelings detection in posts. Figure 2 depicts the class-wise results of competing algorithms using confusion matrices.

This study aimed to detect momentary depressive feelings in X data using contextual language approaches. Our results indicated that (pretrained) transfer learning algorithms such as DistilBERT can effectively detect momentary depressive feelings with an accuracy of 97.4%. In this study, we used a comprehensive process of lexicon construction, data collection, and NLP-based ML algorithms to obtain accurate results. Our findings have practical implications in the mental health field by offering a potential framework for monitoring and detecting individuals’ mental health states in real time, which could facilitate timely interventions and support. For instance, this research can contribute to the development of automated systems that analyze social media posts, enabling mental health professionals to identify individuals expressing depressive feelings and subsequently provide support and resources.

The success of this study can be largely attributed to the effective use of a precise lexicon that contained a list of key terms relevant to depressive feelings based on prior research. These key terms were then manually evaluated to ensure their accuracy and relevance. The high intercoder reliability achieved by the researchers is a testament to the quality of the post labeling. This approach fostered the quality of the lexicon, leading to the accurate detection of momentary depressive feelings. Furthermore, the manual labeling of posts also contributed to the accuracy of the study because automatic labeling typically introduces noise into data and degrades algorithm performance [55].

In this study, we used a range of algorithms, including BERT, ALBERT, RoBERTa, DistilBERT, CNN, and BiLSTM. Notably, DistilBERT outperformed the other algorithms in detecting momentary depressive feelings. The use of multiple algorithms allowed for a comprehensive evaluation of the effectiveness of various algorithms in detecting momentary depressive feelings. These findings, consistent with previous studies, highlight the superiority of transfer learning algorithms in NLP tasks, particularly in the detection of momentary depressive feelings in social media data. This study used state-of-the-art algorithms that leverage large amounts of data and generalize to new tasks. Transfer learning algorithms are especially adept at processing large data sets and can identify patterns and features that are difficult to capture with traditional ML algorithms.

Within the domain of binary and multilabel classification, our analysis uncovers a compelling revelation. Specifically, our findings highlight the intricate nature of posts conveying depressive emotions, irrespective of their varying degrees or gradations. These posts present a formidable obstacle for classification models due to their nuanced character, encompassing a broad range of emotional states that defy simple categorization. This insight serves as a poignant reminder of the challenges inherent in accurately classifying such nuanced sentiments, especially when addressing the realm of depressive expressions.

The significant findings of this study have the potential to make a meaningful impact on mental health, particularly in momentary depressive feelings detection. Early detection of momentary depressive feelings can pave the way for timely interventions and ultimately improve mental health outcomes. The approach presented in this study could be integrated into social media monitoring tools to identify individuals who are at risk of developing depression or who may benefit from mental health interventions. This approach could lead to more efficient and effective mental health interventions, resulting in better outcomes for individuals with mental health conditions and reducing the burden imposed by mental health disorders. The findings of this study provide a beneficial stepping stone for the development of new and innovative approaches to mental health monitoring and intervention.

Although the findings of the study are promising, there are limitations. First, the study only focused on momentary depressive feelings, and the results may not be generalizable to other mental health conditions such as anxiety, stress, or other mood disorders. Second, the study relied solely on X data, which may not represent the broader population. Future research could investigate the use of other social media platforms or clinical data to assess the effectiveness of contextual language approaches for detecting mental health conditions in a more diverse population. Additionally, these approaches may be limited in their ability to capture the nuances and complexity of mental health conditions. Although this method is effective, it may not always detect posts that express subtle or indirect signs of mental health conditions. Future studies could explore the use of ML techniques to learn from the data to detect momentary depressive feelings rather than relying on predefined lexicons. This approach could lead to more accurate results and more effective detection of mental health conditions. Finally, the study only used English posts, which further limits the generalizability of the findings.

This study aimed to detect momentary depressive feelings using X data and contextual language approaches. In this study, we applied a methodology consisting of data collection, manual labeling, and post analysis with contextual language approaches. A lexicon containing 32 keywords relevant to depressive feelings was established, and then, using this lexicon, Twint was used to extract posts from January 2022 to December 2022. Six baseline algorithms were used for the detection of momentary depressive feelings, and the results were evaluated using AUC, accuracy, sensitivity, specificity, precision, and F₁-score.

Our results showed that DistilBERT, a transfer learning algorithm, had the highest performance in terms of the evaluation metrics described. The study found that transfer learning algorithms are promising tools in NLP tasks, for example, extracting knowledge and detecting patterns in posts, particularly in the detection of momentary depressive feelings.

Our findings demonstrated X data can be used for the detection of momentary depressive feelings. This is achieved through the development of an automated framework for continuously monitoring and detecting individuals’ real-time mental states. These findings have significant implications for timely mental health interventions. Early detection of momentary depressive feelings can prevent the escalation of these feelings to more severe depressive symptoms and reduce the burden imposed on people and society. This methodology can be easily applied to large X data sets, making it a useful tool for monitoring depressive symptoms on a large scale. Moreover, this methodology can be improved to be applied to other social media platforms and various mental health conditions. Overall, this study contributes to the growing body of research on using social media data for mental health research. Our approach provides a useful tool for researchers interested in studying momentary depressive feelings using social media data.