Penerapan Graph Neural Network dalam Pengenalan Alfabet BISINDO dengan Fokus pada Gerakan Dinamis
Keywords:
BISINDO, hand gesture recognition, Graph Neural Networks, MediaPipe, Dynamic Sign Language Recognition, pengenalan gerakan tangan, GNN, pengenalan bahasa isyarat dinamisAbstract
Sebagian besar studi pengenalan alfabet Bahasa Isyarat Indonesia (BISINDO) masih terbatas pada gesture statik, meskipun beberapa huruf seperti R dan J memiliki karakteristik gerakan dinamis yang tidak dapat direpresentasikan secara statis. Penelitian ini menggunakan MediaPipe untuk mendeteksi 21 keypoints tangan sebagai input fitur. Titik-titik ini dimodelkan dalam bentuk graf dan diproses menggunakan Graph Neural Networks (GNNs) guna mengenali alfabet secara simultan, termasuk huruf-huruf dinamis. Proses pelatihan menggunakan K-Fold Cross Validation untuk menguji konsistensi performa model. Model GNN menghasilkan akurasi sebesar 96% pada pengujian data alfabet BISINDO. Prototipe sistem dalam bentuk aplikasi web berhasil mengenali 26 huruf BISINDO secara dinamis dengan tingkat akurasi prediksi mencapai 91%, menunjukkan potensi implementasi nyata dari pendekatan GNN dalam mendukung aksesibilitas komunikasi inklusif.
References
A. Rahman, and S. Informatika, “Aplikasi Penerjemah Bahasa Isyarat Menggunakan Metode K-NN (K-Nearest Neighbour),” Jurnal Teknologi Pintar, vol. 2, no. 4, pp. 1–12, 2022. [Online]. Available: https://ejurnal.stmik-budidarma.ac.id/mib/article/view/7714
A. S. Nugraheni, A. P. Husain, and H. Unayah, “Optimalisasi Penggunaan Bahasa Isyarat dengan SIBI dan BISINDO pada Mahasiswa Difabel Tunarungu di Prodi PGMI UIN Sunan Kalijaga,” Jurnal Holistika, vol. 5, no. 1, pp. 28–33, 2023, doi: 10.24853/holistika.5.1.28-33. [Online]. Available: https://jurnal.umj.ac.id/index.php/holistika/article/view/9355
T. Tao and Y. Zhao, “Sign language recognition: A comprehensive review of traditional and deep learning approaches, datasets, and challenges,” IEEE Access, vol. 12, pp. 111216–111234, 2024, doi: 10.1109/ACCESS.2024.3387279
Y. E. Mulyanto, “Sign Language Recognition Based on Geometric Features Using Deep Learning,” Jurnal Nasional Pendidikan Teknik Informatika: JANAPATI, vol. 13, no. 2, pp. 338-348, Jul. 2024, doi: 10.23887/janapati.v13i2.82103.
F. Damatraseta, R. Novariany, and M. A. Ridhani, “Real-time BISINDO Hand Gesture Detection and Recognition with Deep Learning CNN,” Jurnal Informatika Kesatuan, vol. 1, no. 1, pp. 71–80, Jul. 2021, doi: 10.37641/jikes.v1i1.774.
Y. V. Via, W. S. J. Saputra, M. I. Fachrurrozi, E. Y. Puspaningrum, F. T. Anggraeny, and S. R. Nudin, “Object Localization and Detecting Alphabet in Sign Language BISINDO Using Convolution Neural Network,” Technium: Romanian Journal of Applied Sciences and Technology, vol. 5, no. 12, pp. 143–149, 2023. [Online]. Available: https://ideas.repec.org/s/tec/techni.html
H. Kolivand, S. Joudaki, M. S. Sunar, and D. Tully, “A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1),” Neural Computing and Applications, vol. 33, no. 10, pp. 4945–4963, May 2021, doi: 10.1007/s00521-020-05279-7.
B. Khemani, S. Qamar, and S. Anwar, “A review of graph neural networks: Concepts, architectures, and applications,” Journal of Big Data, vol. 11, no. 1, pp. 1–38, 2024, doi: 10.1186/s40537-024-00815-4
A. Sharma, A. Sharma, A. Tselykh, A. Bozhenyuk, and B. G. Kim, “Image and video analysis using graph neural network for Internet of Medical Things and computer vision applications,” CAAI Transactions on Intelligent Technology, 2024, doi: 10.1049/cit2.12306.
S. Dey, S. Samanta, and S. Banerjee, “Sign language recognition using convolutional neural networks: A review and open research challenges,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 14, no. 5, pp. 123–131, 2023, doi: 10.14569/IJACSA.2023.0140515
W. Chen, J. Liu, and Z. Luo, “A survey on hand pose estimation with wearable sensors and computer-vision-based methods,” Sensors, vol. 20, no. 3, pp. 1–25, Feb. 2020, doi: 10.3390/s20030856
H. Ansar, M. Ali, and S. Ahmed, “Hand gesture recognition based on auto-landmark localization,” Sustainability, vol. 13, no. 17, pp. 1–18, 2021, doi: 10.3390/su13179902
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright of this journal is assigned to Jurnal Buana Informatika as the journal publisher by the knowledge of author, whilst the moral right of the publication belongs to author. Every printed and electronic publications are open access for educational purposes, research, and library. The editorial board is not responsible for copyright violation to the other than them aims mentioned before. The reproduction of any part of this journal (printed or online) will be allowed only with a written permission from Jurnal Buana Informatika.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
