Comparing Machine Learning and IndoBERT for PPKM Twitter Sentiment Classification
DOI:
https://doi.org/10.24002/jbi.v17i2.15038Keywords:
Sentiment Classification, IndoBERT, Random Forest, XGBoost, PPKMAbstract
During the COVID-19 pandemic, Indonesia's PPKM policy generated broad discussion on Twitter/X, making sentiment analysis useful for understanding public responses. Indonesian tweets are informal, contain spelling variation and negation, and have imbalanced labels. This study compares Random Forest, XGBoost, and fine-tuned IndoBERT for three-class sentiment classification of PPKM-related tweets. For the classical models, preprocessing preserves important negation markers and represents text with TF-IDF. Because Neutral tweets dominate, Macro F1-score is used as the main metric alongside accuracy. IndoBERT achieves the best performance, with 91.27% accuracy and a Macro F1-score of 0.8429. Random Forest obtains Macro F1-score 0.6664, slightly above XGBoost at 0.6620. Execution time is reported descriptively because the recorded stages are not identical.
Kebijakan PPKM selama pandemi COVID-19 memicu diskusi luas di Twitter/X, sehingga analisis sentimen diperlukan untuk memahami respons masyarakat. Cuitan berbahasa Indonesia bersifat informal, memiliki variasi ejaan dan negasi, serta distribusi label tidak seimbang. Penelitian ini membandingkan Random Forest, XGBoost, dan IndoBERT yang di-fine-tuning untuk klasifikasi sentimen tiga kelas pada cuitan tentang PPKM. Pada model klasik, prapemrosesan mempertahankan kata negasi penting dan teks direpresentasikan dengan TF-IDF. Karena kelas Netral mendominasi, Macro F1-score digunakan sebagai ukuran utama selain akurasi. IndoBERT memberikan kinerja terbaik dengan akurasi 91,27% dan Macro F1-score 0,8429. Random Forest memperoleh Macro F1-score 0,6664, sedikit di atas XGBoost 0,6620. Waktu eksekusi dilaporkan secara deskriptif karena tahapan yang tercatat tidak identik.
References
[1] Y. Wang, J. Guo, C. Yuan, and B. Li, "Sentiment Analysis of Twitter Data," Applied Sciences, vol. 12, no. 22, Art. no. 11775, 2022, doi: 10.3390/app122211775.
[2] S. Hansun, A. Suryadibrata, R. Nurhasanah, and J. Fitra, "Tweets Sentiment on PPKM Policy as a COVID-19 Response in Indonesia," Indian Journal of Computer Science and Engineering, vol. 13, no. 1, pp. 51-58, 2022, doi: 10.21817/indjcse/2022/v13i1/221301302.
[3] W. Budiharto and M. Meiliana, "Prediction and Analysis of Indonesia Presidential Election from Twitter Using Sentiment Analysis," Journal of Big Data, vol. 5, Art. no. 51, 2018, doi: 10.1186/s40537-018-0164-1.
[4] A. Pak and P. Paroubek, "Twitter as a Corpus for Sentiment Analysis and Opinion Mining," in Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10), pp. 1320-1326, 2010.
[5] G. Venkateshwarlu, S. Akhila, V. Kavyasree, S. Vishnu, and V. S. Prasad, "Enhanced Text Classification Using Random Forest: Comparative Analysis and Insights on Performance and Efficiency," International Journal of Computer Engineering in Research Trends, vol. 11, no. 1s, pp. 1-8, 2024, doi: 10.22362/ijcert/2024/v11/i1/v11i1s01.
[6] L. Geni, E. Yulianti, and D. I. Sensuse, "Sentiment Analysis of Tweets Before the 2024 Elections in Indonesia Using IndoBERT Language Models," Jurnal Ilmiah Teknik Elektro Komputer dan Informatika (JITEKI), vol. 9, no. 3, pp. 746-757, 2023, doi: 10.26555/jiteki.v9i3.26490.
[7] Fransiscus and A. S. Girsang, "Sentiment Analysis of COVID-19 Public Activity Restriction (PPKM) Impact Using BERT Method," International Journal of Engineering Trends and Technology, vol. 70, no. 12, pp. 281-288, 2022, doi: 10.14445/22315381/IJETT-V70I12P226.
[8] A. N. Sutranggono and E. M. Imah, "Tweets Emotions Analysis of Community Activities Restriction as COVID-19 Policy in Indonesia Using Support Vector Machine," CommIT (Communication and Information Technology) Journal, vol. 17, no. 1, pp. 13-25, 2023, doi: 10.21512/commit.v17i1.8189.
[9] M. A. D. Naufalino, M. Al-Husaini, and Rianto, "Comparison of Support Vector Machine, Random Forest and XGBoost for Sentiment Analysis on Indodax," Journal of Computer Networks, Architecture and High Performance Computing, vol. 7, no. 2, pp. 562-572, 2025, doi: 10.47709/cnahpc.v7i2.5894.
[10] A. Samih, A. Ghadi, and A. Fennan, "Enhanced Sentiment Analysis Based on Improved Word Embeddings and XGBoost," International Journal of Electrical and Computer Engineering, vol. 13, no. 2, pp. 1827-1836, 2023, doi: 10.11591/ijece.v13i2.pp1827-1836.
[11] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 4171-4186, 2019, doi: 10.18653/v1/N19-1423.
[12] A. Widiarta, "Indonesian Twitter Sentiment Analysis Dataset-PPKM," Kaggle, 2023. [Online]. Available: https://www.kaggle.com/datasets/anggapurnama/twitter-dataset-ppkm. [Accessed: Aug. 24, 2026].
[13] T. Chen and C. Guestrin, "XGBoost: A Scalable Tree Boosting System," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785-794, 2016, doi: 10.1145/2939672.2939785.
[14] B. Wilie et al., "IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding," in Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 843-857, 2020, doi: 10.18653/v1/2020.aacl-main.85.
[15] M. H. Tsai and Y. Wang, "Analyzing Twitter Data to Evaluate People's Attitudes towards Public Health Policies and Events in the Era of COVID-19," International Journal of Environmental Research and Public Health, vol. 18, no. 12, Art. no. 6272, 2021, doi: 10.3390/ijerph18126272.
[16] S. F. Taskiran, B. Turkoglu, E. Kaya, and T. Asuroglu, "A Comprehensive Evaluation of Oversampling Techniques for Enhancing Text Classification Performance," Scientific Reports, vol. 15, Art. no. 21631, 2025, doi: 10.1038/s41598-025-05791-7.
[17] H. Imaduddin, F. Y. A'la, and Y. S. Nugroho, "Sentiment Analysis in Indonesian Healthcare Applications Using IndoBERT Approach," International Journal of Advanced Computer Science and Applications, vol. 14, no. 8, pp. 113-117, 2023, doi: 10.14569/IJACSA.2023.0140813.
[18] B. Setiawan, "A Review of Sentiment Analysis Applications in Indonesia Between 2023-2024," Journal of Information Engineering and Educational Technology, vol. 8, no. 2, pp. 71-83, 2024, doi: 10.26740/jieet.v8n2.p71-83.
[19] E. Kurniawati, A. I. Purnamasari, I. Ali, R. Kurniawan, and O. Nurdiawan, "Analisis Sentimen Ulasan Pengguna Aplikasi FLO di Google Play Store dengan Menggunakan Algoritma Naive Bayes," Jurnal Informatika dan Teknik Elektro Terapan, vol. 14, no. 1, pp. 791-799, Jan. 2026, doi: 10.23960/jitet.v14i1.8776.
[20] S. Kapoor and A. Narayanan, "Leakage and the Reproducibility Crisis in Machine-Learning-Based Science," Patterns, vol. 4, no. 9, Art. no. 100804, 2023, doi: 10.1016/j.patter.2023.100804.
[21] D. Pradipto and I. M. A. Agastya, "Comparative Analysis of Random Forest and Support Vector Machine Algorithms for Predicting Student Retention at Ibnu Sina University," SISTEMASI, vol. 15, no. 4, pp. 1491-1503, 2026, doi: 10.32520/stmsi.v15i4.6298.
[22] F. A. I. Suhendra, A. Darmayantie, A. S. Suhendra, and Pa Pa Min, "Comparative Analysis of Indonesian Pre-trained BERT Models for the Extractive Question Answering Task on an Indonesian-Translated SQuAD Dataset," MATRIK, vol. 25, no. 2, pp. 311-322, Mar. 2026, doi: 10.30812/matrik.v25i2.5847.
[23] L. Dube and T. Verster, "Enhancing Classification Performance in Imbalanced Datasets: A Comparative Analysis of Machine Learning Models," Data Science in Finance and Economics, vol. 3, no. 4, pp. 354-379, 2023, doi: 10.3934/dsfe.2023021.
[24] M. F. Kono, I. N. Fajri, and Y. Pristyanto, "Public Sentiment Analysis on Corruption Issues in Indonesia Using IndoBERT Fine-Tuning, Logistic Regression, and Linear SVM," Journal of Applied Informatics and Computing, vol. 9, no. 5, pp. 2616-2628, 2025, doi: 10.30871/jaic.v9i5.10537.
[25] D. G. Mandhasiya, H. Murfi, and A. Bustamam, "The Hybrid of BERT and Deep Learning Models for Indonesian Sentiment Analysis," Indonesian Journal of Electrical Engineering and Computer Science, vol. 33, no. 1, pp. 591-602, 2024, doi: 10.11591/ijeecs.v33.i1.pp591-602.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright of this journal is assigned to Jurnal Buana Informatika as the journal publisher by the knowledge of author, whilst the moral right of the publication belongs to author. Every printed and electronic publications are open access for educational purposes, research, and library. The editorial board is not responsible for copyright violation to the other than them aims mentioned before. The reproduction of any part of this journal (printed or online) will be allowed only with a written permission from Jurnal Buana Informatika.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
