Evaluation of UAJY Learning Management System’s Usability using USE Questionnaire and Eye-tracking
DOI:
https://doi.org/10.24002/ijis.v4i2.5273Keywords:
usability, learning management system, eye-tracking, USE questionnaire, heatmapsAbstract
One of the influential learning resources utilized today is a web-based learning management system. The web-based e-learning’s usability is also influenced by the user interface. In measuring usability, eye-tracking technology can be employed to examine eye gaze of an individual when looks at a specific point at a certain time. The use of eye-tracking is beneficial in obtaining objective data. This study aims to evaluate the usability of UAJY Learning Management System website interface and examine the user's interaction with the website interface. Thirty-five participants were recruited in a usability testing experiment. Participants were asked to do three tasks related to the use of e-learning features while their eye movements were recorded. The USE Questionnaire and task-related question data were processed using statistical descriptive methods and eye movement data generated into heatmaps. The Usefulness, Ease of Use, Ease of Learn, and Satisfaction aspects of the UAJY LMS gains more than 80% feasibility percentage. Overall results of feasibility categorized the UAJY LMS as Very Feasible. There was no difference found in usability aspects between gender, faculty background, and eye condition groups. Heatmaps results show that navigational elements in the LMS are utilized properly and successfully help participants in performing tasks.References
B. Faghih et al., “User interface design for e-learning software,” arXiv Prepr. arXiv1401.6365, 2014.
J. @ E. Luaran, N. N. Samsuri, F. A. Nadzri, and K. B. M. Rom, “A Study on the Student’s Perspective on the Effectiveness of Using e-learning,” Procedia - Soc. Behav. Sci., 2014.
P. Ramakrisnan, A. Jaafar, F. H. A. Razak, and D. A. Ramba, “Evaluation of user Interface Design for Leaning Management System (LMS): Investigating student’s eye tracking pattern and experiences,” Procedia-Social Behav. Sci., vol. 67, pp. 527–537, 2012.
M. Radecký and P. Smutný, “Evaluating User Reaction to User Interface Element Using Eye-Tracking Technology,” pp. 475–480, 2014.
M. Nazrul, I. Harry, M. N. Islam, and H. Bouwman, “An assessment of a semiotic framework for evaluating user-intuitive Web interface signs,” Univers. Access Inf. Soc., pp. 563–582, 2015.
V. Cho, T. C. E. Cheng, and W. M. J. Lai, “The role of perceived user-interface design in continued usage intention of self-paced e-learning tools,” Comput. Educ., 2009.
C. K. Prantner, “The evaluation of the results of an eye tracking based usability tests of the so called Instructor’s Portal framework (http://tanitlap. ektf. hu/csernaiz),” in 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), 2015, pp. 459–465.
M. Gao, P. Kortum, and F. Oswald, “Psychometric Evaluation of the USE ( Usefulness , Satisfaction , and Ease of use ) Questionnaire for Reliability and Validity,” vol. 2, pp. 1414–1418, 2018.
S. K. Hendra and S. K. Yulyani Arifin, “Web-based Usability Measurement for Student Grading Information System,” Procedia Comput. Sci., vol. 135, pp. 238–247, 2018.
F. Firdaus, M. Irfan, N. Amaliah, Rasydianah, M. Wahid, and Nurhidayah;, “Evaluasi Usability eLearning UNSULBAR Selama Masa Belajar Dari Rumah (BDR),” BIOMA J. Biol. dan …, vol. 2, no. 2, pp. 18–25, 2020.
N. Asnawi, “Pengukuran Usability Aplikasi Google Classroom Sebagai E-learning Menggunakan USE Questionnaire (Studi Kasus: Prodi Sistem Informasi UNIPMA),” Res. Comput. Inf. Syst. Technol. Manag., vol. 1, no. 1, p. 17, 2018.
R. K. Sungkur, M. A. Antoaroo, and A. Beeharry, “Eye tracking system for enhanced learning experiences,” Educ. Inf. Technol., vol. 21, no. July 2015, pp. 1785–1806, 2016.
T. Ujbanyi, J. Katona, G. Sziladi, and A. Kovari, “Eye-tracking analysis of computer networks exam question besides different skilled groups,” 2016 7th IEEE Int. Conf. Cogn. Infocommunications, no. CogInfoCom, pp. 277–282, 2016.
P. V. Yulianandra, S. Wibirama, and P. I. Santosa, “Examining the effect of website complexity and task complexity in web-based learning management system,” in 2017 1st International Conference on Informatics and Computational Sciences (ICICoS), 2017, pp. 119–124.
I. S. O. Iso, “9241-11: 1998 Ergonomic requirements for office work with visual display terminals (VDTs)--Part 11: Guidance on usability,” Geneve, CH ISO, 1998.
J. Rubin and D. Chisnell, Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons, 2008.
A. M. Lund, “Measuring usability with the USE questionnaire,” Usability interface, vol. 8, no. 2, pp. 3–6, 2001.
N. Menzi-Çetin, E. Alemdağ, H. Tüzün, and M. Yıldız, “Evaluation of a university website’s usability for visually impaired students,” Univers. Access Inf. Soc., 2017.
S. Djamasbi, M. Siegel, T. Tullis, and R. Dai, “Efficiency, trust, and visual appeal: Usability testing through eye tracking,” in 2010 43rd Hawaii International Conference on System Sciences, 2010, pp. 1–10.
P. Faraday, “Visually critiquing web pages,” in Multimedia’99, 2000, pp. 155–166.
R. Mayer and R. E. Mayer, The Cambridge handbook of multimedia learning. Cambridge university press, 2005.
G. Rakoczi, “Analysis of eye movements in the context of E-learning: recommendations for eye-efficient user interfaces,” 2014.
I. Conference, O. N. Human, and B. In, “DESIGNING ONLINE TESTS FOR A VIRTUAL LEARNING ENVIRONMENT – EVALUATION OF VISUAL BEHAVIOUR BETWEEN TASKS,” no. October, pp. 1–8, 2014.
G. Rakoczi, “Cast your eyes on moodle: An eye tracking study investigating learning with Moodle,” in Moodle. si 2010 4th International Conference Proceedings, 2010, pp. 203–213.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Indonesian Journal of Information Systems
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Indonesian Journal of Information Systems as journal publisher holds copyright of papers published in this journal. Authors transfer the copyright of their journal by filling Copyright Transfer Form and send it to Indonesian Journal of Information Systems.
Indonesian Journal of Information Systems is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
