JSA 2Volume 4, No. 7, July 2023

p-ISSN 2722-7782 | e-ISSN 2722-5356

DOI: https://doi.org/10.46799/jsa.v4i7.671


USABILITY ANALYSIS OF C-ACCESS COMMUTERLINE APPLICATIONS USING THE SYSTEM USABILITY SCALE (SUS)

 

Safrudin Kurniawan, Ary Putra I, Ahmad Ependi���������

Management Transportation Railways Railway Polytechnic Indonesia M a diun

Email: [email protected]

 

Abstract : Trend enhancement Commuterline KRL passengers who are almost approach number before pandemic of course also need enhancement service in the field information . one _ service the information provided to commuterline KRL passengers is provision application C-Access which is development from application KRL - Access that can downloaded on the Play Store. As application new and still in development C-Access naturally need input and feedback from user For can give experience best and service best . follow up matter on so writer need For do research so you can get description from user related experience user during using C-Access. On research This writer use method study quantitative descriptive with approach use System Usability Scale (SUS). SUS method itself is tool the test results end later will produce global assessment of aspects usability . Objective from study This is For get evaluation from user to use application C-Access Commuterline and get interpretation from results evaluation the . The sampling technique used is Simple Random Sampling with population of users C-Access application . Research results on applications C-Access use method System Usability Scale (SUS) gain score of 66.125 scale 0 � 100. The score If interpreted use Acceptability Score is in the range Marginal , if interpreted use School Grading Scale is in grade D and if interpreted use Adjective Ratings is on a scale on OK However under good .

 

Keyword : Usabilty , System Usability Scale, SUS, C-Access, Applications .


 

 

INTRODUCTION

During the Covid Pandemic � 19 total commuterline KRL passengers experience very sharp decline . this _ caused Because exists restrictions activity community that followed with restrictions activity transportation of people to suppress and prevent transmission of the Covid virus � 19. Along with start ending The Covid � 19 pandemic then amount KRL Commuterline passengers also experienced improvement .

had time experience decline drastic in 2020 and 2021, total commuterline KRL passengers start experience increase in 2022. According to PT Kereta Commuter Indonesia (KCI) report , average number commuterline KRL users reached 538,537 people per day since beginning January until August 2022. In 2020 the amount passenger daily KRL Commuterline is of 422,382 people per day decrease drastic from amount passenger daily in 2019 ie _ of 921,297 people per day . Whereas in 2021 the average volume of Commuterline KRL users only 350,210 people per day . In this year 2023 until with month February the average number of passengers daily was 738,028 passengers per day ( Statistics, BPS, 2023) .

Table 1

Average _ Amount Railroad passengers

Year

Amount Passenger Average Daily

2019

921,297

2020

422,382

2021

350,210

2022

538,537

2023 sd February

738,028

Source : Central Bureau of Statistics

trend enhancement commuterline KRL passengers this is close approach number before pandemic of course also need enhancement service in the field information . one _ service the information provided to commuterline KRL passengers is provision C-Access application which is development from KRL application - Access that can downloaded on the Play Store.

C- Access is more applications _ user friendly because applicative function , more _ easy use and display more _ millennials ( Son, 2023) . C-Access is development from KRL Access, where C-Access was introduced feature new like purchase QR Code for a maximum of 4 people and top-up KMT balance with NFC ( Near Field Communication ) system . C-Access was created none other than because KAI Commuter is trying become ecosystem urban transportation that can collaborate with transportation other . Collaboration the is KAI Commuter journey planner which will be accomplished in 2026. Later on in C-Access No only can used For message ticket but also can used For order food ( Son, 2023) .

Figure 1

Appearance C-Access application

Figure 2

Purchase Features Tickets at C-Access

As application new and still in development of C-Access of course need input and feedback from user For can give experience best and service best . Don't until C-Access also received a low rating from user as happened in KRL Access . follow up matter on so writer need For do research so you can get description from user related experience user during using C-Access. Study the entitled " Analysis Application Usability C-Access Commuterline Use System Usability Scale ( SUS) �.

 

METHODOLOGY

On research This writer use method study quantitative descriptive with approach use System Usability Scale (SUS). SUS method itself is tool the test results end later will produce global assessment of usability aspects ( effectiveness , efficiency and satisfaction ) ( Broke, 2013) . SUS has used as an industry standard with used by thousands articles and publications .

Steps _ in study This can explained as following :

1.      Determination population and sample :

First step in study This is determine target population and retrieve sample from population the . Population in research This is user application C-Access Commuterline . After target population is determined , step furthermore is determine samples. The sampling technique used is Simple Random Sampling . For amount sample will use Formula Slovin . Formula This used For count amount sample in large population . _ The formula are :

n = N / (1 + N(e^2))

where : n = amount sample required _

N = size target population

e = margin of error or sampling error

Amount population C-Access users on an ongoing basis appropriate No known , however in accordance description in Playstore , C-Access has downloaded and used more than 100,000 times. For That so in study This used population of 100,000. If we enter in the formula on with a margin of error of 10% then will got result :

���������������������������� n = 100,000 / (1+100,000(0.1^2))

���������������������������� n = 100,000 / (1+100,000(0.01))

���������������������������� n = 99.90

So for study This with a 10% margin of error is required a minimum sample of 99.90 is rounded off to 100 people.

2.      Development instrument .

After sample selected , step furthermore is prepare instrument measurement , namely SUS ( System Usability Scale ). SUS consists from ten measuring statement _ convenience use something product or system .

First step use of SUS ie participant requested For rate 10 statement items with choose score response in 5 scale from the lowest i.e. Strongly No Agree to the highest is Strongly Agree (Figure 3)

Figure 3

Rating Scale in SUS

10 statements in SUS is as following :

1)      I think I am will often use system this .

2)      I found system This complicated and there things that don't necessary .

3)      I think system This easy used .

4)      I think I need help from a technical person For can use system this .

5)      I found various function in system This integrated with ok .

6)      I think There is too Lots inconsistency in system this .

7)      I imagine most people will Study use system This very fast . _

8)      I found system it's very tricky For used .

9)      I feel very confident self use system .

10)  I need Study Lots matter before I Can use system this .

Statement on the number odd (1,3,5,7,9) is meaningful statement _ positive whereas statement on the number even (2,4,6,8,10) means negative.

3.      collection .

After instrument developed , step furthermore is collecting data. Data collected with send SUS questionnaire to respondent . Questionnaire This will shipped online to users _ _ C-Access through social media platforms , namely Facebook and Whatsapp .

4.      processing .

After the data is collected will done data processing ie with give score on each answer respondent Where lowest score _ is 1 for Strongly No statement Agree and score 5 for Strongly Agree statement . Then answer from each respondent For each statement For statement numbered odd minus 1 and statement numbered even minus 5, p This done For balancing score between statement positive and negative statements . After That score each statement multiply by 2.5 and add up Then the average is calculated . Function multiplication with 2.5 here is For make it easy in presentation results end Because more easy for people to see score in scale 0 to with 100 rather than 10 to by 50. ( source ) Final results from the SUS method is score between 0 to with 100.

5.      Interpretation result .

After the data is analyzed , then can interpreted result . from here can determine is product or system own good usability _ or bad based on SUS score . There are several method For can interpret SUS score , in study This writer will use developed method _ by ( Bangor, 2008) that is Adjective Rating, Acceptability Score and School Grading Scale (Bangor, 2009) . Method This used Because Already through testing and getting very accurate and valid results ( Bangor, 2008) . In interpretation use Adjective Ratings SUS scores are interpreted with 7 adjectives namely Worst Imaginable, Awful, Poor, OK, Good, Excellent and Best Imaginable ( see table 3).

Table 2

SUS scores were compared with Adjective Ratings

While the interpretation of using the School Grading Scale is to use a scoring system that is generally used in schools namely A, B, C, D and F where A is worth between 90-100, B is worth between 80-89, C is worth between 70-79, D is 60-69 and F is under 60.

In addition (Bangor, 2008) also developed another interpretation (see figure), namely Acceptability Score which is divided into 3 categories, namely: Not Acceptable, Marginal and Acceptable , where Marginal is further divided into Low and High. Acceptable on the System Usability Scale (SUS) shows level reception user to application .

Figure 4

�Comparison of Acceptability Range, Grade Scale and Adjective rating with SUS Score

 

 

6.      Presentation result .

Final step is serve results research . Research results form score However can served in form table or chart For make it easy reader in understand results research .

7.      Conclusions, and suggestions for study next .

 

RESULTS AND DISCUSSION

Characteristics Respondents

Based on results deployment SUS questionnaire that has been done to 100 respondents , obtained results form characteristics respondent . those results can explained as following :

Table 3

Characteristics Respondents

Characteristics respondent

Amount respondent

Percentage

%

Type Sex

man _ _

79

79%

Woman

21

21%

Age

25 yrs or not enough

82

82 %

26 � 35 yrs

8

8 %

36 � 45 yrs

9

9 %

45 yrs or more

1

1 %

Work

Student / student

52

52 %

PNS/TNI/ Polri /BUMN

17

17 %

Private

13

13 %

Self-employed

8

8 %

Other

10

10 %

 

System Usability Scale Score Calculation

After obtained score from every respondent for each statement so then the data is processed in accordance SUS method ie with with way :

1)      Reduce score For statement numbered odd by 1 and subtract statement numbered even with 5, p This done For balancing score between statement pitched positive and statement pitched negative .

2)      Step two that is with sum up whole score from each respondent with 2.5, p This done to get results score end between 0 � 100 so later score end easy read and understood .

3)      After score of each respondent summed up Then totaled For all respondents and divided amount respondent For get average score . The final SUS score will be range between 0 and with 100. Need remembered that This No percentage .

������������� Following this is initial data from sheet spreadsheet (google form) that has been processed use method System Usability Scale (SUS ):

Table 5

SUS Score Processing Results

Respondents

Count Result Score (Data)

Amount

Mark

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

( Total x 2.5)

Respondent 1

3

3

3

3

3

3

3

3

3

3

30

75.00

Respondent 2

1

3

2

3

2

2

2

2

2

2

21

52.50

Respondent 3

3

3

4

2

3

2

3

4

3

1

28

70.00

Respondent 4

3

3

3

1

3

1

3

3

3

1

24

60.00

Respondent 5

4

0

4

0

4

0

4

0

4

0

20

50.00

Respondent 6

3

3

3

1

3

3

3

3

3

1

26

65.00

Respondent 7

4

4

4

4

4

4

4

4

4

4

40

100.00

Respondent 8

3

3

4

3

3

3

3

3

3

2

30

75.00

Respondent 9

4

2

3

2

4

3

4

2

3

2

29

72.50

Respondent 10

2

3

4

3

3

4

4

3

3

3

32

80.00

Respondent 11

3

3

3

3

3

3

3

3

4

3

31

77.50

Respondent 12

3

3

4

2

4

3

3

3

3

3

31

77.50

Respondent 13

2

3

2

1

3

2

3

3

2

1

22

55.00

Respondent 14

2

1

1

1

2

2

1

1

2

1

14

35.00

Respondent 15

3

3

3

3

3

3

2

2

3

2

27

67.50

Respondent 16

3

3

3

1

4

3

3

4

4

1

29

72.50

Respondent 17

3

3

3

4

3

3

4

3

3

2

31

77.50

Respondent 18

4

3

4

3

2

2

3

3

2

3

29

72.50

Respondent 19

4

3

4

4

4

3

4

4

4

3

37

92.50

Respondents 20

3

3

3

3

3

2

3

3

3

2

28

70.00

Respondent 21

2

3

3

3

3

3

2

1

2

1

23

57.50

Respondent 22

1

1

1

4

2

3

3

1

3

4

23

57.50

Respondent 23

3

3

3

3

2

3

3

3

3

1

27

67.50

Respondent 24

2

3

4

1

3

3

4

4

3

3

30

75.00

Respondents 25

3

2

2

1

2

2

1

2

3

0

18

45.00

Respondent 26

3

1

3

0

3

2

3

2

3

1

21

52.50

Respondent 27

3

3

4

3

4

3

3

3

4

4

34

85.00

Respondent 28

4

4

4

4

4

3

3

4

4

3

37

92.50

Respondent 29

3

1

3

1

3

1

3

1

3

1

20

50.00

Respondents 30

2

3

3

3

3

2

3

3

3

2

27

67.50

Respondent 31

3

1

3

1

3

1

3

1

3

1

20

50.00

Respondent 32

3

1

4

2

1

2

3

1

3

1

21

52.50

Respondent 33

3

2

2

3

3

2

3

3

3

2

26

65.00

Respondent 34

4

3

3

3

3

3

4

3

3

3

32

80.00

Respondents 35

2

4

4

4

4

4

4

4

4

2

36

90.00

Respondent 36

4

3

3

2

4

2

4

3

4

1

30

75.00

Respondent 37

4

0

4

4

4

4

4

4

4

4

36

90.00

Respondent 38

2

2

2

2

3

2

4

0

4

1

22

55.00

Respondent 39

2

3

2

1

3

2

3

2

2

1

21

52.50

Respondent 40

2

3

3

2

3

2

3

4

3

1

26

65.00

Respondent 41

4

0

4

0

4

0

4

0

4

0

20

50.00

Respondent 42

2

3

2

3

3

2

3

3

3

2

26

65.00

Respondent 43

3

3

3

1

3

2

4

3

3

1

26

65.00

Respondent 44

4

1

4

3

3

3

4

4

4

3

33

82.50

Respondent 45

4

4

4

2

4

2

3

4

4

1

32

80.00

Respondent 46

2

3

3

2

1

2

3

3

3

0

22

55.00

Respondent 47

4

4

4

4

4

4

4

4

4

4

40

100.00

Respondent 48

4

2

3

4

0

1

3

3

3

1

24

60.00

Respondent 49

4

0

4

0

4

1

4

0

4

0

21

52.50

Respondents 50

2

3

3

1

3

4

3

4

4

2

29

72.50

Respondent 51

2

2

3

0

4

0

4

1

4

1

21

52.50

Respondent 52

3

3

3

3

4

1

1

4

2

4

28

70.00

Respondent 53

3

1

3

1

3

1

3

1

3

1

20

50.00

Respondent 54

3

2

3

1

3

3

2

3

3

0

23

57.50

Respondents 55

4

3

4

2

3

3

3

3

3

2

30

75.00

Respondent 56

4

0

4

1

2

0

1

1

4

0

17

42.50

Respondent 57

3

2

4

1

2

2

4

3

3

0

24

60.00

Respondent 58

4

2

4

3

4

2

4

3

4

3

33

82.50

Respondent 59

3

4

3

2

3

3

3

4

3

1

29

72.50

Respondents 60

3

2

3

2

3

3

3

3

4

0

26

65.00

Respondent 61

3

3

3

1

3

3

3

3

2

1

25

62.50

Respondent 62

3

3

3

3

3

3

3

3

3

2

29

72.50

Respondent 63

2

3

4

3

3

3

4

3

3

2

30

75.00

Respondent 64

2

3

3

3

3

3

3

3

3

2

28

70.00

Respondent 65

4

4

4

3

3

3

4

3

3

3

34

85.00

Respondent 66

3

4

4

0

4

4

4

4

4

0

31

77.50

Respondent 67

4

2

3

3

4

2

4

3

4

0

29

72.50

Respondent 68

3

3

3

3

3

3

3

3

3

3

30

75.00

Respondent 69

3

1

3

1

3

1

3

1

4

2

22

55.00

Respondents 70

3

2

3

2

1

1

3

2

2

1

20

50.00

Respondent 71

4

2

3

2

4

2

3

2

4

1

27

67.50

Respondent 72

4

0

4

0

4

0

4

0

4

0

20

50.00

Respondent 73

3

3

3

0

3

3

3

1

2

3

24

60.00

Respondent 74

3

3

3

1

3

3

3

1

2

1

23

57.50

Respondents 75

3

3

4

3

3

3

3

4

3

1

30

75.00

Respondent 76

4

4

4

4

4

4

4

4

4

4

40

100.00

Respondent 77

3

2

3

2

3

1

3

2

3

2

24

60.00

Respondent 78

3

3

3

3

3

3

3

3

3

1

28

70.00

Respondent 79

1

3

2

1

2

1

2

1

3

2

18

45.00

Respondents 80

3

1

3

2

2

1

3

1

2

2

20

50.00

Respondent 81

2

1

1

0

2

0

2

1

2

0

11

27.50

Respondent 82

3

3

3

3

3

3

3

3

3

3

30

75.00

Respondent 83

3

1

3

1

3

0

4

4

1

1

21

52.50

Respondent 84

2

1

2

2

2

3

2

4

4

4

26

65.00

Respondent 85

4

1

3

4

3

2

4

0

4

2

27

67.50

Respondent 86

2

1

1

2

1

1

2

1

2

1

14

35.00

Respondent 87

3

1

3

1

3

3

3

3

3

2

25

62.50

Respondent 88

3

3

3

4

3

2

3

4

3

3

31

77.50

Respondent 89

3

4

3

3

3

2

4

3

3

3

31

77.50

Respondents 90

4

4

3

4

3

3

3

3

3

3

33

82.50

Respondent 91

3

3

3

3

3

3

3

3

3

3

30

75.00

Respondent 92

3

3

3

3

2

2

3

3

3

2

27

67.50

Respondent 93

3

3

3

2

3

2

3

2

4

2

27

67.50

Respondent 94

3

1

3

2

3

2

2

2

2

3

23

57.50

Respondents 95

2

3

3

3

3

2

3

3

2

1

25

62.50

Respondent 96

3

3

3

1

3

3

3

3

3

3

28

70.00

Respondent 97

1

4

2

2

2

0

3

1

3

1

19

47.50

Respondent 98

3

2

3

2

2

1

1

2

2

1

19

47.50

Respondents 99

3

3

3

4

4

3

4

3

4

3

34

85.00

Respondents 100

3

2

2

1

2

1

3

1

2

2

19

47.50

Amount

6612.50

 

Based on table above _ results count use method System Usability Scale (SUS) above , the total SUS score is 6612.50. And if counted average score _ with dividing 100 respondents result is 66.125.

Data Interpretation

Based on the average SUS score obtained from results SUS questionnaire data analysis , then can done interpretation results based on 3 methods variation , that is Adjective Rating, Acceptability Score and School Grading Scale . Following location score from results questionnaire System Usability Scale (SUS) against third method interpretation that ( figure 5)

Figure 5. Position of SUS Score against Interpretation

������������������������������������������������������������������������������������ ��������������������

Based on scale SUS measurements in fig on then , the interpretation of each variation method interpretation score System Usability Scale (SUS) for application C-Access is as following :

1.      For score on interpretation Acceptability Score , the SUS score is in the range Marginal High , p the means that level reception user to C-Access is Enough low and can accepted by the user , will be but need done repair repeat in application .

2.      On interpretation School Grading Scale , SUS score of 66.125 is in value D. Interpretation This based on the system assessment at school general , then For the value of D is in the range value 60 to with 69 on the scale measurement 0 � 100. By general For score graduation on the School Grading Scale is 70. So that can concluded that app SUS scores C-Access Not yet meet the threshold graduation .

3.      For interpretation use Adjective Ratings, SUS scores are in the OK and Good ranges . Rating OK on the Adjective Rating are on average score 50.9 with plus minus 1 standard error of the mean , meanwhile Rating Good at a score of 71.4 plus minus 1 standard error of the mean . So for the score is 66.125 between these ratings .

CONCLUSION

In accordance with analysis in Chapter 4 inside study This obtained results as following :

  1. Measurement results score C-Access Commuter Line application with use the System Usability Scale method gets a score of 66.125.
  2. The score obtained by the application C-Access after be measured with the SUS method , namely 66.125, can be interpreted as following :
    1. Use interpretation Acceptability Score , SUS score of 66.125 is in the range Marginal High , p the means that level reception user to C-Access is Enough low and can accepted by the user , will be but need done repair repeat in application .
    2. Use interpretation School Grading Scale , the SUS score of 66.125 is at a value of D where in a manner general passing grade mark is 70.
    3. Use interpretation Adjective Ratings, SUS score of 66.125 is in the range on OK however under good.

 

BIBLIOGRAPHY

ISO 9241-11, I. (1998). Ergonomic Requirements for Office Work with Visual Display Terminals,. Geneva: ISO.

Anggraini, WN (2020). Analysis on Student Academic Information Systems. Journal of Research , 185.

Arikunto, S. (2011). The research procedure : a practical approach . Jakarta: Rineka Cipta.

Bangor, AK (2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction , 574-594.

Bangor, AK (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies , 114-123.

Broke. (2013). SUS: A Retrospective. Journal of Usability Studies , 36.

Brooke, J. (1986). SUS: A "quick and dirty" usability scale. In BTPW Jordan, Usability Evaluation in Industry. London: Taylor and Francis.

Khuntari. (2022). Usability Analysis of Google Workspace for Education in Universities.

Kurniawan, EN (2022). Application of the usability scale system in measuring the usability of study program websites at STMIK royal. Science and social research.

Lee, JY, Kim, JY, You, SJ, Kim, YS, Koo, HY, Kim, JH, . . . Kim, H. (2019). Development and Usability of a Life-Logging Behavior Monitoring Application for Obese Patients. Journal of Obesity and Metabolic Syndrome , 194 - 202.

Nielsen, J. (2012, 01 03). nngroup.com. Retrieved 04 10, 2023 from nngroup.com: https://www.nngroup.com/articles/usability-101-introduction-to-usability/

Nielsen, J. (2023, 04 10). Usability 101: Introduction to Usability. Retrieved 04 10, 2023 from https://www.nngroup.com/articles/usabi: https://www.nngroup.com/articles/usabi

Pertiwi, V. (2019). Evaluation of Usability in KRL Access Applications Using. Journal of Information Technology Development and Computer Science , 2235-2242.

Primary, AF (2021). Usability Level Analysis . Edutic Science .

Son, RP (2023, 01 19). redigest.web.id . Retrieved 04 10, 2023 from redigest.web.id: https://redigest.web.id/2023/01/kai-commuter-doing-soft-launching-gratis-c-access/#.ZDQbZPtBzHa

Sauro, J. (2011). A practical guide to the System Usability Scale: Background, benchmarks, & best practices. Denver: Measuring Usability LLC.

Setiawan, MS (2021). Usability Level Analysis Using the SUS Method. National Seminar on Industrial Engineering and Management and Call for Papers .

Smith, KT (2011). Needs Analysis: Or, How Do You Capture, Represent, and Validate User Requirements in a Formal Manner/Notation before Design. CRC Press.

Statistics, BP (2023, 04 10). BPS. Retrieved 04 10, 2023 from BPS: https://www.bps.go.id/indicator/17/72/1/nomor-penumpang-kereta-api.html

Statistics, BP (2023). Number of KRL Passengers. Jakarta: BPS.

Tullis, TS (2004). A comparison of questionnaires for assessing website usability. Proceedings of UPA 2004 Conference. Minneapolis,Minnesota.

 

Copyright holders:

Safrudin Kurniawan, Ary Putra I, Ahmad Ependi (2023)

 

First publication right:

Journal of Syntax Admiration

 

This article is licensed under: