265
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
USABILITY OF EGOVERNANCE APPLICATION FOR
CITIZENS OF PAKISTAN
Abdul Samad Dahri
Business Administration and Social Sciences
Mohammad Ali Jinnah University, Karachi, (Pakistan).
E-mail: dahriabdulsamad@gmail.com ORCID: https://orcid.org/0000-0003-4517-3493
Shaq-ur-Rehman Massan
QEC and Co-ordination
Mohammad Ali Jinnah University, Karachi, (Pakistan).
E-mail: srmassan@hotmail.com ORCID: https://orcid.org/0000-0001-6548-6513
Ayaz Ali Maitlo
Department of Business Administration,
University of Sindh, Larkana Campus, (Pakistan).
E-mail: ayazalimaitlo@gmail.com ORCID: https://orcid.org/0000-0001-9831-8380
Recepción:
13/03/2020
Aceptación:
20/04/2020
Publicación:
30/04/2020
Citación sugerida Suggested citation
Dahri, A. S., Massan, S., y Maitlo, A. A. (2020). Usability of eGovernance application for citizens
of Pakistan. 3C Tecnología. Glosas de innovación aplicadas a la pyme. Edición Especial, Abril 2020, 265-277.
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
266
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
ABSTRACT
eGovernance is a vital component of any nations’ modern governmental structure and
suitable for mass-scale adaptation. Pakistan has also established an eGovernance council
for mitigation of the problems of the common man. One big step in this direction is the
establishment of Pakistan Citizens’ Portal (PCP) that facilitates eGovernance. Through
the PCP app, citizens can directly lodge complaints and report their issues for immediate
resolution.
This study looks at the important aspect of mobile usability through eld tests to determine
the viability of the PCP app in Pakistan. This study is vital towards determining the main
factors for large scale adaptation of the PCP app and underlines the weaknesses of the
present system.
Hence, the PCP app was evaluated in terms of eciency and eectiveness according to
ISO 92421-11. And the user satisfaction was measured through system usability satisfaction
(SUS) on ve-point Likert scale. The evaluation method included more than a dozen
citizens of Sindh and ndings showed that overall application enriched the user experience.
However, the few areas of PCP were identied as needing improvements. The main areas
that the PCP application needs to improve are the areas of registration and ‘ndability’
which would further improve user satisfaction and experience.
KEYWORDS
EGovernance, Usability evaluation, Commoner satisfaction, Pakistan Citizen Portal, ISO
designed usability standards.
267
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
1. INTRODUCTION
eGovernance is the most recent trend of governance worldwide. Proper implementation of
eGovernance requires SMART (Simple, Moral, Accountable, Responsive and Transparent)
work ethic (Mehra & Prabhu, 2013). The advent of information communication technology
(ICT) has infused a new sphere of governmental intervention. The issues of the citizens are
now being handled by eGovernance.
UNESCO denes eGovernance as the application of ICT towards participatory decision
making hence improving information and service delivery (Obi, 2007). Similarly, with the
advent of time the quality and type of services required by the common man in Pakistan is
changing. Presently, the government is facing a challenge to manage the vast requirements
of health, education travel, credit, security and documentation (Javaid & Arfeen, 2017).
eGovernance process is a fairly recent trend globally. It helps improve interaction within the
civil society by improving the processes of the government, interconnecting the citizens, and
building viable interactions within society (Brown, et al. 2013). Though, literature reveals
misconception regarding the dierence between traditional and eGovernance, where,
eGovernance appears to simply add more electronic devices to governance rather than
cumbersome paper based work. Therefore, to change perception and practices, government
needs to integrate technology with feasible policy change in the system (Arfeen, Khan, &
Amanullah, 2012).
According to united nations eGovernance survey, Pakistan was placed at 137 in 2003
out of 139 countries which declined to 146 in the year 2010 and further to 156 in 2012.
There could be many factors to this decline, but the problem of Pakistan is not the lack
of resources but bad governance (Rais, 2019). Even though, from layman of Asia to
future tiger of Asia, needs bold steps by the government of Pakistan (Ministry of Planning,
Development & Reform Government of Pakistan, 2020). Yet, lethargic bureaucracy and
lack of accountability left government helpless and leads ineciencies in the system towards
implementation of solutions to citizen problems and complaints.
Eective eGovernance implementation utilizes internet to provide of value added public
services to civil society as per OECD (Ubaldi, 2009). eGovernance therefore needs
268
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
connectivity of digital devices to deliver solutions and improve citizen lives (United Nations
Conference on Trade and Development, 2009). Mobile devices have become more user
friendly and useful and allow the users greater exibility to perform more tasks. Another
advantage is that mobile network signals cover 85% of the international population and
80% of these are smartphones users (Chaey, 2016). Similarly, in Pakistan 2G mobile
covers nearly 90% of population (PTA annual report 2014-2015). While, 57% of rural
and urban population have smartphones (Rizvi, 2018). This highlights penetration of
mobile technology and indicates untapped eGovernance intervention potential in Pakistan.
Therefore, eGovernance through mobile application holds the future for ecient and
quality solutions of mitigation of citizen complaints.
Capitalizing over mobile access, current government tapped true potential and launched
eGovernance Pakistan Citizens’ Portal (PCP) mobile application 2018. This initiative is
rst of its kind in the history of Pakistan. This app providing access of 70,000 government
oces to citizens at a single click and was recognized as 2nd best app in Global Summit
held in Dubai February 2019. Moreover, 1.173 million users including overseas, and local
citizens got registered. 553,125 complaints were lodged, and 509,153 complaints were
resolved making rate of 92% including the provinces of Khyber Pakhtunkhwa, Sindh,
Islamabad, Baluchistan and Azad Kashmir. While, Sindh provincial government was worst
at 40% in terms of complaint resolution and even worse at 84% for complaints pending
rate (News Desk, 2019).
On contrary, according to Furlow (2012), 95% of mobile applications are not eld tested.
Since PCP mobile application is a new phenomenon in Pakistan, its independent eld
testing is essential for upgrades by product developers. Similarly, the past literature on
mobile application usability comprises of surveys or literature reviews rather eld evaluation
(Dahri, Al-Athwari, & Hussain, 2019). This study serves the purpose of eld evaluation of
PCP mobile application usability evaluation tests. Indeed, mobile usability evaluation test
reveals the quality of how users adapt and utilize it and assess its actual eld performance.
According to Coursaris and Kim (2011), mobile usability evaluation test is vital for
smartphone users enabling the design of user friendly applications and determines success
of the product (Baharuddin, Singh, & Razali, 2013). In simple words, mobile usability
269
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
evaluation test facilitates users of the technology to achieve their specic tasks with ease
(Coursaris & Kim, 2011).
2. USER ASSESSMENT
The participants were requested to perform a few simple tasks enlisted below:
1) Registering themselves with the system (Figure 1),
2) Application home screen usability (Figure 2)
3) Searching for relevant department (Figure 3)
4) Lodging a complaint (Figure 4),
5) Feedback (Figure 5),
6) ISO 9241-11 Usability framework (Figure 6)
Figure 1. Registering themselves
with the system.
Figure 2. Application home
screen usability.
Figure 3. Searching for relevant
department.
270
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
Figure 5. Feedback
Figure 6. ISO 9241-11 Usability
framework.
3. MEASURES AND METHODS
Quantitative usability design is provided by ISO-9241-11 (see Figure 6) that outlines user
metrics with step by step implementation (UCD). The most common usability test is the
typical task performance to measure eectiveness of the application, i.e. to what extent the
user is able to achieve his desired goals. While eciency is the level of resources and the
amount of eort required by the user to attain accuracy and completeness of his desired
goal. And last but not the least satisfaction or discontent is measurable by experiencing his
acceptable usability level during the performance of a task.
Eectiveness as well as eciency during interaction with the application may be measured
by evaluating the completion of the given tasks and by counting the number of errors in
user’s attempt. Eciency is to be measured by the resources and the eorts undertaken to
achieve the desired task. Likewise, for general assessment of PCP, satisfaction is measured
with system usability scale (SUS) which was developed and designed by Brooke in 1996
consist of 10 items on a Likert scale of 0-4 during the experience of the user interaction
with the mobile application. We have utilized SUS in this study for validity, reliability and
sensitivity scores that range from a mere 0 to about 100. The SUS score of 50 or below is
271
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
considered to be poor, above 70 to be in the good range and while 85 or greater score is
considered to be excellent (Bangor, Kortum, & Miller, 2009).
The statistical methods used in this paper may be referenced from Shaikh, Massan and
Wagan (2015), Memon and Shaikh (2016), Shaikh, Memon and Hussain (2016), Arain,
Shaikh and Shaikh (2017), Khowaja et al. (2019), Shaikh, Massan and Wagan (2019),
Soomro et al. (2019) and Sultan, Shaikh and Chowdhry (2020).
4. RESULTS
The number of participants were 15 from Hyderabad city of Sindh province as complaint
resolution was the lowest from Sindh province of Pakistan. Most (70%) of the participants
were middle aged adults (30-40), 30% of the participants ranged between 20-29 years of
age, 70% of them had attained some education at college level and all used smartphones.
4.1. EFFECTIVENESS
Task 1 and 3 were recorded dicult having a total of 35% and 46% failure rates respectively.
Task 4 and 5 were completed in the absence of any errors. Task 2 accumulated largest
errors by participants and took more time. The kinds of error were:
1) Diculty to remember steps of the procedure.
2) Department options and typically.
3) Complaint resolution information was repeated.
4.2. EFFICIENCY
Table 1 reveals that task 1 (Getting registered with the system), 2 (Application home screen
usability), and 3 (Searching for relevant department) consumed more time due to expected
diculty and errors as well as had higher mean time. On the other hand, task 4 (Lodging a
complaint) and 5 (Feed Back) consumed less mean time as that of task 1, 2, and 3.
Table 1. Time taken for each task per task.
S.No Task 1 Task 2 Task 3 Task 4 Task 5
Mean
(SD)
4.21
(0.97)
3.95
(0.55)
4.01
(0.69)
2.58
(1.09)
1.89
(1.91)
272
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
4.3. SATISFACTION
The average SUS scores for the group was recorded at 85. This score indicates a sound
satisfaction level across these PCP users. However, is a quite wide variation in scores of
satisfaction that ranged from 62 to as high as 92 (at 30- point range). It was observed that
60 to 74 is a good rating and higher was excellent and under acceptable region where most
of the participants’ satisfaction scores were noticed.
Figure 7. Variation in scores of satisfaction.
The tasks were based on the given guidelines of ISO for real time citizen interaction with
application and validation. An error was reported whenever any user failed to complete
any given task. The eciency was estimated by measuring the average time utilized by a
user towards the completion of each task from beginning to the successful completion. The
evaluation of satisfaction score was done by measuring the SUS and the scores calculated
by the use of guidelines of Brooke. The descriptive statistical data was obtained by the use
of SPSS version 25.
5. INTERPRETATION
The interpretation of Task Performance, results, satisfaction and demographic trends
show that task 1 (getting registered) and 3 (searching for relevant department) had most
errors, was considered dicult and time consuming. These issues might arise due to lack of
application comprehensiveness as, even task 1 (getting registered) took time of users though
over several portals getting registered is routine task. However, overall user satisfaction was
273
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
excellent. The results also indicate areas of improvement. Such as, multiple registration
options, over informed grid on home screen and repetition of information regarding
complaint resolution.
The demography of participants revealed that male participants were slightly more
informed and performed better than female participants on SUS scores. Similarly, younger
participants took less time and made few errors in task 1 and 3.
6. RECOMMENDATIONS
The study results tender some recommendations that might advance the experience and
enhance the user satisfaction as follows:
1) User menu must be in list format.
2) Icon must illustrate departments included.
3) Registration process should follow less steps.
4) Application home screen should be user friendly.
5) Illiterate portion of citizens should also be considered.
6) Colors should be vivid and should match with actual logo of each department.
7. CONTRIBUTIONS TO LITERATURE
This is one of the foremost usability studies on the PCP application in the province of
Sindh in Pakistan. It has addressed the issues of eectiveness, eciency, and satisfaction
by using validated scores are recommended by (Lyles, Sarkar, & Osborn, 2014). This study
utilizes and validates ISO designed usability standards of the SUS instrument towards
the comparison of usability metrics and performance. This study bridges the present gap
within the literatures of usability and helps identifying eGovernance improvement criteria
of government of Pakistan. Following (Coursaris & Kim, 2011) this study also contributes
to understanding of demography and technology interaction in Pakistan’s society and its
inuence on usability performance. The scores may be utilized to design future interventions
274
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
for capturing the target population quickly and eectively. Hence, this study highlights main
issued in the PCP application that must be removed to make it a successful and viable tool.
8. CONCLUSION
The study ndings show that most of the users had good perceived usability satisfaction
and only one-third of the users were rated as possessing little or poor usability. The results
can be utilized by the developers objectively to design the necessary corrections in the PCP
application. The objectivity of the results indicates the mismatch between the developer
perception and the user experience which must be bridged.
ACKNOWLEDGEMENT
The authors wish to thank Dr. Haz Abdul Ghani Shaikh for his undaunted support.
REFERENCES
Arain, S., Shaikh, F., & Shaikh, M. M. (2017). Problem of Trac Congestion and
Correlation Analysis of Driving behaviors in Qasimabad, Hyderabad. Mehran
University Research Journal of Engineering and Technology, 36(1), 139–148. https://trid.trb.
org/view/1442479
Arfeen, M. I., Khan, N., & Amanullah, M. (2012). Public Sector Innovation through
e-Governance in Pakistan. Life Science, 9(3), 1226–1233. http://www.lifesciencesite.
com/lsj/life0903/174_9492life0903_1226_1233.pdf
Baharuddin, R., Singh, D., & Razali, R. (2013). Usability dimensions for mobile
applications-a review. Research Journal of Applied Sciences, Engineering and Technology, 5(6),
2225–2231. https://doi.org/10.19026/rjaset.5.4776
Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean:
Adding an adjective rating scale. Jour nal of Usability Studies, 4(3), 114–123. http://citeseerx.
ist.psu.edu/viewdoc/download?doi=10.1.1.177.1240&rep=rep1&type=pdf
275
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
Brown, W., Yen, P.-Y., Rojas, M., & Schnall, R. (2013). Assessment of the Health
IT Usability Evaluation Model (Health-ITUEM) for evaluating mobile health
(mHealth) technology. Journal of Biomedical Informatics, 46(6), 1080–1087. https://doi.
org/10.1016/J.JBI.2013.08.001
Chaey, D. (2016). Mobile marketing statistics compilation. https://doi.
org/10.15358/9783800653140-121
Coursaris, C. K., & Kim, D. J. (2011). A meta-analytical review of empirical mobile
usability studies. Journal of Usability Studies, 6(3), 117–171. http://uxpajournal.org/
wp-content/uploads/sites/8/pdf/JUS_Coursaris_May_2011.pdf
Dahri, A. S., Al-Athwari, A., & Hussain, A. (2019). Usability Evaluation of Mobile
Health Application from AI Perspective in Rural Areas of Pakistan. International
Journal of Interactive Mobile Technologies, 13(11), 213–225. https://doi.org/10.3991/
ijim.v13i11.11513
Furlow, B. (2012, November 2). mHealth apps may make chronic disease management easier.
Clinical Advisor. https://www.clinicaladvisor.com/home/features/mhealth-apps-
may-make-chronic-disease-management-easier/
Javaid, M., & Arfeen, M. (2017). Impact of eGovernment on Citizen Satisfaction: A Case of
Federal Government Agencies in Pakistan, 221–237. https://doi.org/10.1007/978-3-319-
65930-5_19
Khowaja, A., Mahar, M. H., Lashari, H. N., Wasi, S., & Massan, S.-R. (2019).
Personality Evaluation of Student Community using Sentiment Analysis. Nternational
Journal of Computer Science & Network Security (IJCSNS), 19(3), 167–180. https://www.
researchgate.net/publication/332395067_Personality_Evaluation_of_Student_
Community_using_Sentiment_Analysis
Lyles, C. R., Sarkar, U., & Osborn, C. Y. (2014). Getting a technology-based diabetes
intervention ready for prime time: a review of usability testing studies. Current Diabetes
Reports, 14(10), 534. https://doi.org/10.1007/s11892-014-0534-9
276
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
Mehra, K., & Prabhu, C. S. R. (2013). E-Governance: Concepts and Case Studies : ix. PHI
Learning Private Limited.
Memon, A. J., & Shaikh, M. M. (2016). Condence bounds for energy conservation
in electric motors: An economical solution using statistical techniques. Energy, 109,
592–601. https://doi.org/10.1016/J.ENERGY.2016.05.014
Ministry of Planning, Development & Reform Government of Pakistan. (2020).
Pakistan 2025. One Nation - One Vision. https://www.pc.gov.pk/uploads/vision2025/
Pakistan-Vision-2025.pdf
News Desk. (2019, October 3). PTI govt takes lead in citizens’ complaint resolution. The Express
Tribune. https://tribune.com.pk/story/2071030/1-pti-govt-takes-lead-citizens-
complaint-resolution/
Obi, T. (2007). E-governance: A Global Perspective on a New Paradigm (p. 27). IOS Press.
Rais, R. B. (2019). Citizens Portal: Improving governance through technology. Retrieved
November 21, 2019, from http://arabnews.pk/node/1587581
Rizvi, J. (2018, November 9). Pakistan’s urban-rural mobile ownership divide set to close soon, study
shows. The News. https://www.thenews.com.pk/print/391338-pakistan-s-urban-
rural-mobile-ownership-divide-set-to-close-soon-study-shows
Shaikh, M. M., Massan, S.-R., & Wagan, A. I. (2015). A new explicit approximation to
Colebrook’s friction factor in rough pipes under highly turbulent cases. International
Journal of Heat and Mass Transfer, 88, 538–543. https://doi.org/10.1016/j.
ijheatmasstransfer.2015.05.006
Shaikh, M. M., Massan, S.-R., & Wagan, A. I. (2019). A sixteen decimal places’
accurate Darcy friction factor database using non-linear Colebrook’s equation with
a million nodes: A way forward to the soft computing techniques. Data in Brief, 27,
104733. https://doi.org/10.1016/J.DIB.2019.104733
277
http://doi.org/10.17993/3ctecno.2020.specialissue5.265-277
3C Tecnología. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
Shaikh, M. M., Memon, A. J., & Hussain, M. (2016). Data on electrical energy
conservation using high eciency motors for the condence bounds using statistical
techniques. Data in Brief, 8, 529–535. https://doi.org/10.1016/J.DIB.2016.06.004
Soomro, M. A., Memon, S. A., Shaikh, M. M., & Channa, A. (2019). Indoor air
CO2 assessment of classrooms of educational institutes of hyderabad city and its
comparison with other countries. In AIP Conference Proceedings, 2119(1), 020014. AIP
Publishing LLC. https://doi.org/10.1063/1.5115373
Sultan, M., Shaikh, M. M., & Chowdhry, N. P. (2020). Comparative Analysis of Knee
Joint Replacement and Stem Cells Therapy Treatment for Knee Osteoarthritis
Using Statistical Techniques. Research in Medical and Engineering Sciences, 8(4), 887–897.
https://crimsonpublishers.com/rmes/pdf/RMES.000693.pdf
Ubaldi, B. (2009). OECD e-Government Studies: Rethinking e-Government Services; User-Centred
Approaches. www.oecd.org/gov/egov/services
United Nations Conference on Trade and Development. (2009). Information Economy
Report 2009, Trends and Outlook in Turbulent Times. United Nations. https://unctad.org/
en/Docs/ier2009_en.pdf
