Evaluasi Performa Kernel SVM dalam Analisis Sentimen Review Aplikasi ChatGPT Menggunakan Hyperparameter dan VADER Lexicon
DOI:
https://doi.org/10.24002/jbi.v15i1.7925Keywords:
SVM, kernel, hyperparameter, VADER lexicon, sentiment analysis, analisis sentimenAbstract
ChatGPT merupakan model bahasa kecerdasan buatan yang merespon pertanyaan dan pernyataan pengguna. ChatGPT memiliki manfaat dan kelemahan bagi pengguna. Hal ini menimbulkan komentar pada media sosial tentang manfaat dari ChatGPT. Penelitian ini membahas tentang analisis sentimen review aplikasi ChatGPT menggunakan SVM kernel linier, RBF, polinomial dan sigmoid. Pelabelan menggunakan VADER lexicon dan hyperparameter untuk menghasilkan parameter terbaik. Tujuan penelitian yaitu apakah aplikasi ChatGPT dapat memberikan manfaat dan membuktikan apakah kernel pada SVM dapat meningkatkan nilai akurasi. Diskenariokan persentase pembagian antara data uji dan data latih adalah 70:30, 80:30, dan 90:10. Setelah dilakukan preprocessing, kemudian dikelompokkan menjadi review positif dan negatif. Dilakukan hyperparameter terhadap parameter C dan Gamma sehingga menghasilkan nilai maksimal. Hasil eksperimen diperoleh akurasi tertinggi menggunakan SVM kernel RBF skenario 90:10 dengan nilai accuracy 92.72%, precision 92.44%, f1-score 96.10% dan AUC 88%.
References
M. Dowling and B. Lucey, “ChatGPT for (Finance) research: The Bananarama Conjecture,” Financ Res Lett, vol. 53, May 2023, doi: 10.1016/j.frl.2023.103662.
Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao, “Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT,” arxiv.org, 2022.
Y. Zhu, P. Zhang, E.-U. Haq, P. Hui, and G. Tyson, “Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Tasks,” Association for Computing Machinery, vol. 1, no. 1, 2023.
S. Biswas, “The function of chat GPT in social media: According to chat GPT. Introduction,” SSRN, pp. 1–6, 2023.
A. Ahmad and W. Gata, “Sentimen Analisis Masyarakat Indonesia di Twitter Terkait Metaverse dengan Algoritma Support Vector Machine,” Jurnal Teknologi Informasi dan Komunikasi, vol. 6, no. 4, p. 2022, 2022, doi: 10.35870/jti.
D. L. Rianti, Y. Umaidah, and A. Voutama, “Tren Marketplace Berdasarkan Klasifikasi Ulasan Pelanggan Menggunakan Perbandingan Kernel,” STRING (Satuan Tulisan Riset dan Inovasi Teknologi), vol. 6, no. 1, pp. 98–105, 2021.
P. H. Prastyo, I. Ardiyanto and R. Hidayat, "Indonesian Sentiment Analysis: An Experimental Study of Four Kernel Functions on SVM Algorithm with TF-IDF," 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), pp. 1-6, 2020, doi: 10.1109/ICDABI51230.2020.9325685.
R. Indraswari, A. Z. Arifin, and D. Herumurti, “RBF Kernel Optimization Method With Particle Swarm Optimization On SVM Using The Analysis Of Input Data’s Movement,” Jurnal Ilmu Komputer dan Informasi (Journal of Computer Science and Information), vol. 10, no. 1, pp. 36–42, 2017.
A. Ramadhanu, R. A. Mahessya, M. R. Zaky, and M. Isra, “Penerapan Teknologi Machine Learning Dengan Metode VADER,” JOISIE Journal Of Information System And Informatics Engineering, vol. 7, no. 1, pp. 165–173, 2023.
I. G. S. A. Putu, S. Deviya, Y. Sibaroni, and E. B. Setiawan, “Aspect-Based Sentiment Analysis on iPhone Users on Twitter Using the SVM Method and Optimization of Hyperparameter Tuning,” Jurnal Media Informatika Budidarma, vol. 7, no. 1, pp. 89–98, 2023.
M. D. Hendriyanto, A. A. Ridha, and U. Enri, “Analisis Sentimen Ulasan Aplikasi Mola Pada Google Play Store Menggunakan Algoritma Support Vector Machine,” Journal of Information Technology and Computer Science (INTECOMS), vol. 5, no. 1, pp. 1–7, 2022.
S. Ernawati, R. Wati, and N. Nuris, “Support Vector Classification with Hyperparameters for Analysis of Public Sentiment on Data Security in Indonesia,” Jurnal Riset Informatika, vol. 5, no. 1, pp. 85–92, 2022.
O. H. Rahman, G. Abdillah, and A. Komarudin, “Klasifikasi Ujaran Kebencian pada Media Sosial Twitter Menggunakan Support Vector Machine,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 5, no. 1, pp. 17–23, 2021.
M. Wankhade, A. Chandra, S. Rao, and C. Kulkarni, “A Survey on Sentiment Analysis Methods, Applications, and Challenges,” Artificial Intelligence Review, vol. 55, no. 7, pp. 5731–5780, 2022.
A. Haleem, M. Javaid, and R. P. Singh, “An era of ChatGPT as a significant futuristic support tool : A study on features, abilities, and challenges,” BenchCouncil Transactions on Benchmarks, Standards and Evaluations, vol. 2, no. 4, pp. 1–8, 2023.
Ö. Aydin and E. Karaarslan, “OpenAI ChatGPT Generated Literature Review : Digital Twin in Healthcare,” Emerging Computer Technologies, vol. 2, pp. 22–31, 2022.
J. Yin and Q. Li, "A Semismooth Newton Method for Support Vector Classification and Regression," Computational Optimization and Applications, vol. 73, pp. 477–508, 2019. DOI: 10.1007/s10589-019-00075-z.
M. Ahmad, S. Aftab, M. S. Bashir, and N. Hameed, “Sentiment Analysis using SVM : A Systematic Literature Review,” (IJACSA) International Journal of Advanced Computer Science and Applications, vol. 9, no. 2, pp. 182–188, 2018.
S. I. Nurhafida, F. Sembiring, J. Raya, and C. No, “Analisis Sentimen Aplikasi Novel Online Di Google Play Store Menggunakan Algoritma Support Vector Machine ( SVM ),” Jurnal Sains Komputer & Informatika (J-SAKTI), vol. 6, no. 1, pp. 317–327, 2022.
L. Parisi, “m-arcsinh: An Efficient and Reliable Function for SVM and MLP in scikit-learn,” 2020, doi: https://doi.org/10.48550/arXiv.2009.07530.
C. Çilgin, M. BAŞ, H. BİLGEHAN, and C. ÜNAL, “Twitter Sentiment Analysis During Covid-19 Outbreak with VADER,” Academic Journal of Information Technology, vol. 13, no. 49, pp. 73–89, 2022.
T. Yan, S.-L. Shen, A. Zhou, and X. Chen, “Prediction of geological characteristics from shield operational parameters by integrating grid search and K-fold cross validation into stacking classification algorithm,” Journal of Rock Mechanics and Geotechnical Engineering, vol. 14, no. 4, pp. 1292–1303, 2022.
R. G. S. K, A. K. Verma, and S. Radhika, “K-Nearest Neighbors and Grid Search CV Based Real Time Fault Monitoring System for Industries,” in 2019 5th International Conference for Convergence in Technology (I2CT), 2019, pp. 9–13.
K. Dataset, “ChatGPT Tweets,” Kaggle, 2022. [Online]. Available: https://www.kaggle.com/datasets/furqanrustam118/chatgpt-tweets. [Accessed: 01-Sep-2023].
M. Carrington et al., “Deep ROC Analysis and AUC as Balanced Average Accuracy , for Improved Classifier Selection , Audit and Explanation,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 45, no. 1, pp. 329–341, 2023.
W. Suharmawan, “Pemanfaatan Chat GPT Dalam Dunia Pendidikan,” Education Journal : Journal Education Research and Development, vol. 7, no. 2, pp. 158–166, 2023.
D. M. W. Powers and Ailab, “EVALUATION: FROM PRECISION, RECALL AND F-MEASURE TO ROC, INFORMEDNESS, MARKEDNESS & CORRELATION,” 2020, doi: https://doi.org/10.48550/arXiv.2010.16061.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright of this journal is assigned to Jurnal Buana Informatika as the journal publisher by the knowledge of author, whilst the moral right of the publication belongs to author. Every printed and electronic publications are open access for educational purposes, research, and library. The editorial board is not responsible for copyright violation to the other than them aims mentioned before. The reproduction of any part of this journal (printed or online) will be allowed only with a written permission from Jurnal Buana Informatika.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
