User Experience Analysis on Mobile Application Design Using User Experience Questionnaire

Abstrak. Saat ini, penggunaan konsep interaksi manusia dengan komputer merupakan hal yang sangat penting jika dikaitkan dengan desain aplikasi seluler. Penelitian ini bertujuan untuk menganalisis pengalaman pengguna aplikasi mobile Grab, khususnya pengguna layanan makanan Grab food dengan menggunakan kuesioner pengalaman pengguna. Peneliti melihat beberapa keluhan pengguna Grab food, seperti keluhan terkait lokasi, harga, dan pembayaran. Data tersebut kemudian dianalisis menggunakan Ms Excel. Temuan menunjukkan bahwa dari keenam skala User Experience Questionnaire (UEQ) semuanya baik kecuali skala ketergantungan. Hasil penelitian menunjukkan bahwa nilai variabel attractiveness adalah 1,64, perspicuity 1,69, dependability 1,29, efficiency 1,60, stimulation 1,57, dan kebaruan 1,21. Selain itu, nilai skala rata-rata yang dihitung menguntungkan, dan semua nilai lebih signifikan dari 0,8 dibandingkan dengan tolok ukur UEQ. Temuan


Sabukunze, Arakaza (User Experience Analysis on Mobile Application Design Using User Experience
Questionnaire) to order food, online transportation, and other types of services offered by Grab. In addition, the application accepts different kinds of payment, such as cash after service or e-money (OVO).

Literature study 2.1. Usability and Usability testing
System Usability Scale permits to review of a wide range of products and services, including hardware, software, mobile devices, and websites [16]. It provides a "fast and dirty," reliable tool for measuring usability. Various measures for measuring usability, such as the Computer System Usability Questionnaire [17], evolved for a group of a vast number of ended questionnaires to determine if the structural value found for the PSSUQ in a usability test will remain the study mailed. The SUS has befitted the most popular questionnaire for usability evaluation, usability research, and the survey [17]. Studies have revealed that SUS gives the most reliable validity and sensitivity to an extensive range of independent variables [17]. Usability is the study that links systems and the users [18]. Since most software program design seems to be inadequate to meet customer needs, various research has been directed on the usability, which leads to a better and more relevant understanding measures intended to cover all pertinent scenarios of one foundation or model [19]. Usability testing is a technique to evaluate the usability used to selfevaluate a product by experimenting on user agents in environmental agents [20]. To attain functional application, however, it is necessary to expand the standard of living for explicitly discussing usability issues. Usability testing uses measurable factors, namely efficiency, and effectiveness, to evaluate performers' actions [21].
The International Organization for Standardization (ISO) defines usability as "the expanse to which professionals can use an outcome to achieve specific goals with efficiency, effectiveness, and satisfaction in a particular area of application" [22]. The usability bears all details of usage such as learnability, regular usage, accessibility, and maintainability [23]. Learnability refers to the ability of new users to use the new system efficiently, effectively, and satisfied. Regular usability means that users can reach their objectives with effectiveness, efficiency, and satisfaction. Accessibility indicates that product is accessible and usable for users and a wide range of competencies. Maintainability means that the system is able to control duties to be achieved with effectiveness, efficiency, and satisfaction. In other words, usability testing is an experiment that intends to assess products for knowing usability issues, collecting eligibility information, qualitative and quantitative data on user satisfaction levels [24].
In the study [25], the authors evaluated the usability of an Amazon Kindle app. The Amazon Kindle is an electronic reader app that assists users in downloading, purchasing, getting access to online books, reading magazines, and many more. The findings have shown that the users who participated in this study already have experience with that application.
For video, a study conducted by [26] intended to address the gap in the YouTube application usability, particularly for principal areas and functions (login, upload, share, and many more) and restricting practices for teenager users. The result indicated that most participants watch YouTube for watching the video and do not consider other features in the application while doing that.
Usability is a subset of user experience telling about background information, issues, and effects encountered, helps problem identification while using the system, cooperation, replan, and restructuring results [27] [28]. However, to obtain the best findings of the usability issues, usability testing requires to be performed with five respondents [29]. Therefore, due to the pandemic, the usability testing phase was not able to be conducted. Thus, the authors focused only on the user experience test on Grab users.

User Experience
Mobile apps have become popular over the last two decades. A study in Brazil started with an experience when a bus could not stop at a bus stop even though passengers showed a signal to get on or off. To overcome this problem, the solution is to design a mobile application that can direct passenger bus routes. Application design to improve service quality and increase user satisfaction needs to apply user experience methods [30]. User experience is not referred to internally operate as a product or service. User experience refers to working outside, connecting with the outcome [31] [32]. A study conducted in Malaysia by Zahidi et al., within two cultural heritage websites showcased from cultural heritage collections, was selected to collect the results. These websites were chosen because of various character sharing; both are publicly accessible online platforms and motivate them to store cultural heritage content like user-generated content [33]. The User Experience is not only applied for mobile applications; it can also be applied for measuring the quality of web services using the user experience method [34], measuring the user experience of e-learning platforms using UEQ [35].
Usability principles are similar to the user experience questionnaire. The usability objectives are attributes of an interaction associated with users' functions to achieve their purposes. Based on the research conducted by [36], the author provided utility, ease to use, learnability, and many more. User experience objectives are the characteristics of an interaction connected to emotional learning about the general interaction with the item, such as enjoyment, innovation, and many more [37].

User Experience Questionnaire
This manuscript examined the interactive interface features of Grab's online transportation mobile application based on the user experience questionnaire. Much research has been carried out on the usability of online transportation, like the evaluation on Grab mobile application in Indonesia [38].
In the study conducted by Mussa, the researcher evaluated the user experience of an e-learning platform using the User experience Questionnaire. Among his participants, there were students and lecturers of the same online-learning website. The findings indicated that the platform needs some improvements in the user experience aspects [39].
Another group of researchers evaluated the user experience of the Halodoc mobile Health application using UEQ and usability testing. Based on the results, all items were good but not excellent [40]. Paramitha et al. carried out a study to evaluate a web-based academic progress information system using the User Experience Questionnaire. The findings have indicated that the study was positive because the score was more than 0.8 [41].

Research Methodology
This research was developed to investigate the analysis and interface design of Grab's online transportation mobile application. The survey was done using a questionnaire to analyze the online transportation mobile application based on the User Experience Questionnaire. In addition, a survey study has been applied in this research with UEQ (Indonesia version) to collect responses from Grab online transport users, in particular, Grab food service.

Research instrument used
This research adopted User Experience Questionnaire in the Indonesian language because many respondents were Indonesian citizens. This same version questionnaire was established then applied in the study carried out to calculate the UX of the education system [35]. UEQ includes a 7-point Likert scale and twenty-six elements showing the main feature. Therefore, it ensures that the questionnaire is clear and more precise.
The user experience questionnaire (UEQ) is dependent on the concept that the user experience can be computed by studying the usability and user experience objectives. This questionnaire is used to perform a user experience assessment perceived towards a product fast. There are six measurement scales in UEQ, namely [40], [42]:

Sabukunze, Arakaza (User Experience Analysis on Mobile Application Design Using User Experience
Questionnaire) • Attractiveness: The overall impression that users feel about the product. Do our users like the product? • Perspicuity: The ease that is felt when using the product. Is the product easy to become familiar with when used? • Efficiency: Interaction between users and products is made quickly and efficiently.
• Dependability: The user's feelings are in control over interactions. Can system behavior be predictable by users? • Stimulation: Pleasure and motivation obtained when using the product. Is the product is motivating and exciting when used by the user? • Novelty: Creativity and product innovation. Does the product have a display innovative and creative so that they can interest users? The environment that real. Based on the above explanation, the UEQ measures can be summarized as follows: UEQ has three principal characteristics: attractiveness, hedonic, and pragmatic quality, as displayed in Figure 1.

Questionnaire survey
The instrument was prepared and converted to Google Forms then distributed via WhatsApp. In other words, Google Forms was distributed to numerous Grab online transportation users that have already used Grab food service. The participants were given a questionnaire involving items from the items described in the research design used section. This stage examines the level of satisfaction and agreement with the user experience for Grab mobile application based on the 7-point Likert scale as in Figure 2.

Participants
The respondents of the questionnaire are those with the following characteristics: a) participants that had already used Grab online transportation mobile application especially Grab food service, b) located in Yogyakarta city in Indonesia, c) willingly to participate in this study.
This research attracted 90 (100%) participants freely and helped authors to assess the usability of the Grab mobile application. Among the 90 participants, 45 were females (50%), and 45 (50%) were males. Furthermore, all the 90 participants between the ages of 18 to 25 were 63, and those above 25 were 27 participants. Moreover, 62 were students, two (2) lecturers, and the rest worked in different sectors, and all were located in Yogyakarta.
Before analyzing data, the authors started by doing pre-treatment data because we realized that some participants responded without seriousness. The detected suspicious data were for 6 participants, which means we remain with 84 participants.

The Cronbach's Alpha
The findings of Alpha coefficient values of the scales of this study is presented in Table 2. The calculated Cronbach's Alpha of all UEQ scales usually indicated satisfaction of the values, which means that Alpha's value is more than 0.7. A combination of new values has a value lower than 0.7, except the Novelty scale, which is lower than 0.7. The Alpha coefficient of the novelty scale is smaller than 0.7, which might be obtained because the Grab food service does not make a great essential for Grab mobile application users. For that reason, it means that the reliability of the UEQ scales is high based on the statement of study [35].

Interpretation of the findings
This section analyzed the value of the UEQ measures' findings, and the sample given in the Excel tool used to analyze data. This juxtaposition is essential and the first sign of knowing whether the software or website assessed has a good UX to be called effective and fruitful. Hence, it is necessary to compare and relate to the benchmark value to identify the frailties and strengths of the assessed system. Table 3 represents the mean coefficients of UEQ measures of the benchmark data compared with any examined system's values. Meanwhile, Table 4 below shows the measure of scales in the overall responses of our respondents.  Table 4 indicates the outcome of means calculations for every scale by revealing either the findings are positive, negative, or average. Based on the literature [44], if the excel data analysis tool has upward arrows, it signifies that the findings are positive. The overall findings of the Grab mobile application are illustrated in Figures 3 and 4. The value of each UEQ item is displayed in Figure 3. The findings acquired by filling out the questionnaires are to give weight to each scale by rating from -3 (very bad) to +3 (excellent), as stated in the study [44]. Questionnaire) The overall outcome of the Grab mobile application is illustrated in Table 5, Figure 2 and 3. The value of each scale can be seen, where the Attractiveness scale is 1.64, Efficiency 1.60, Perspicuity 1.69, Stimulation 1.57, Dependability 1.29, and Novelty 1.21. Referring to the benchmark of the UEQ in the reference [43], the findings of the UEQ are in the relative normal range. Generally, the hedonic quality values (1.39) are lower than the pragmatic quality (1.53).
The findings from respondents designated that all UEQ measures are coherent like Alpha values were more significant than 0.7. Consequently, from the conclusions given in Table 3, the outcomes from participants reveal items except the Novelty are in a good category, with Cronbach's Alpha which is equal to or greater than 0.7 as stated in the study [33]. Table 5 and Figure 2 show that the dependability scale has only shown a mean value above average. The other scale is in a good category based on the UEQ benchmark, as shown in Table 4. Furthermore, Table 3 indicated that Novelty's alpha value is less than 0.7, which might be obtained because the Grab food service does not make a great essential for Grab mobile software users.

Conclusion
This study intended to analyze the user experience of the online transportation mobile application called Grab 5.158.0, especially for Grab food services considering the complaints in the Play Store. The survey of online transportation applications was conducted using a user experience questionnaire (UEQ) method, with users from Yogyakarta as the respondents. Based on the results displayed above, it can be summarized that the scale means of the Grab online transportation for Grab foodservice users is reasonable because several of them are in a good category compared to the values in the UEQ benchmark. The findings have shown a good indication of users' experience of the Grab mobile application because all values are above 0.8. Distinctly, there is a degree of difference between pragmatic quality scales (perspicuity, Efficiency, and dependability) and the other Hedonic Quality (Stimulation and Novelty). The recommendation regarding the result is improvement on the dependability variable.
Future studies hope to find more participants for data collection and use other usability testing methods and interviews.

Limitations
This research depends on the survey and has limitations. Due to the pandemic, the researchers only conducted the survey on the user experience but not on the usability testing methods.

Acknowledgement
The authors' gratitude goes to the University Atma Jaya Yogyakarta especially to the Department of Informatics Engineering and the International Office. The authors would also express gratitude to Alexis Arakaza in Burundi, a lecturer to ISGE for the assistance.