Usability Evaluation of Clinician Web Back-Ends to Telemonitoring Systems: Two Case-Studies in Scotland

Cristina-Adriana ALEXANDRU
University of Edinburgh School of Informatics, Informatics Forum
room 5.24, 10 Crichton Street, Edinburgh, EH8 9AB

Brian McKINSTRY
Centre for Population Health Sciences, The University of Edinburgh Medical School
room 104, Teviot Place, Edinburgh, EH8 9AG

Abstract: One of the reasons why large scale deployment of telemedicine has not been successful is the difficulty healthcare workers have in managing the software. Good usability is essential to the success of a telemedicine solution. By ensuring that user needs are efficiently and effectively respected, usability encourages user acceptance and reduces the need for support. However, little is known about what healthcare workers require from telemedicine systems in terms of how patient acquired data is displayed and interrogated. This paper describes the results of a post-implementation investigation of the clinician web ‘back-ends’ of two telemedicine systems used for the monitoring of long-term conditions in Lothian, Scotland, focusing on the features healthcare workers would like to see in future systems. We conducted semi-structured interviews and questionnaires to ascertain the views of healthcare workers who had been using the systems. The results of the evaluation were used to design a new prototype generic telemonitoring website which we offered to participants to demonstrate possible improvements and to further seek their views. The prototype was very well received, participants considering that it was easier to use and more user friendly than the system that they had been using.

Keywords: Telemedicine, usability, long-term conditions, telemonitoring websites, qualitative research, case-study, thematic analysis.

>>Full text
CITE THIS PAPER AS:
Cristina-Adriana ALEXANDRU, Brian McKINSTRY, Usability Evaluation of Clinician Web Back-Ends to Telemonitoring Systems: Two Case-Studies in Scotland, Studies in Informatics and Control, ISSN 1220-1766, vol. 21 (2), pp. 181-190, 2012. https://doi.org/10.24846/v21i2y201208

1. Introduction

In a recent review of conference papers exploring factors which influence the implementation of telemedicine, Broens et al concluded that technical and behavioural (i.e. related to acceptance) barriers are the most important factors determining the success of a telemedicine system [1]. In particular, they point to software usability both as a technical barrier, as it is related to the usefulness to end users, and behaviour barrier, as it influences the overall acceptance of a system.

The most commonly quoted definition of usability is that provided by the ISO 9241-11 standard: “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [2]. A software system with good usability takes into consideration the users’ profile, work context and respects the users’ tasks. Good usability reduces the time and effort users need in their initial interaction and in the long-term use of the system, rendering interaction with the system easy and pleasurable. Usability has been shown to have positive influence on productivity [3]. By ensuring that the processes are intuitive to the users, it reduces the need for training and support and decreases error rates, as users learn to use the system just by trying it out. Despite the recognized importance of usability, developers do not appear to prioritise it in the design of telemedicine systems, concentrating on functionality [4].

Central to usability engineering is user-centred design. It involves end-users in the design of a system so that user requirements are better understood and respected by the system and, importantly, that users feel some ownership of the system, a process which has been shown to increase the system’s level of acceptance [5]. However, end-users are often not sufficiently engaged in the design of healthcare solutions because of the commercial focus on building systems within short time frames [6]. This results in top-down designs in which usability problems are only observed after the rollout, when they are more costly to amend and often lead to the rejection of the system by its users.

We aimed to understand which aspect of ‘back end design’ caused most problems for healthcare workers and to determine what changes might enhance the user experience. To do this we carried out a post-implementation usability evaluation of two telemonitoring websites used in Lothian, Scotland, that had taken part in the Telescot programme of telehealthcare trials [7]. This programme evaluated the use of telehealthcare in a range of long-term conditions. We then fed back the results of the evaluation to participants for their comments. Additionally we created, demonstrated and sought comments on a prototype of an improved generic website which incorporated suggestions from clinicians.

REFERENCES

BROENS, T. H. F., R. M. H. A. HUIS IN’T VELD, M. M. R. VOLLENBROEK-HUTTEN, H. J. HERMENS, A. T. VAN HALTEREN, L. J. M. NIEUWENHUIS, Determinants of Successful Telemedicine Implementations: A Literature Study, Journal of Telemedicine and Telecare, vol. 13, 2007, pp. 303-309.
ISO, ISO 9241-14 Ergonomic Requirements for Office Work with Visual Display Terminals (vdts)- Part 11: Guidance on Usability, 1998.
MAGUIRE, M., Methods to Support Human-centred Design, International Journal of Human-Computer Studies, vol. 55, 2001, pp. 587-634.
SINGH, J., C. LUTTEROTH, B. C. WÜNSCHE., A Taxonomy of Usability Requirements for Home Telehealth Systems, Software Engineering, University of Auckland, May 2010.
FAULKNER, X., Usability Engineering, Basingstoke: Macmillan Press Ltd, 2000.
PAGLIARI, C. Design and Evaluation in eHealth: Challenges and Implications for an Interdisciplinary Field, Journal on Medical Internet Research 2007, Volume 9, Issue 2, e15, p. 1.
Telescot. http://www.telescot.org, consulted on 10.04.2012.
IEM. http://www.iem.de/ consulted on 10.04.2012.
Intel. http://www.intel.com/corporate/healthcare/emea/eng/healthguide/index.htm consulted on 10.04.2012.
NIELSEN, J. Introduction to usability [Internet], Jacob Nielsen’s Alertbox [cited 29th March 2012] Available from: http://www.useit.com/alertbox/20030825.html
DIX, A., G. ABOWD, R. BEALE, J. FINLAY, Human-Computer Interaction, Prentiec Hall Europe, 1998.
SUDUC, A. M., M. BÎZOI, F. G. FILIP, User Awareness about Information Systems Usability, Studies in Informatics and Control, Volume 19, Issue 2, pp. 145-152, ISSN 1220-1766, 2010.
ROSE, A. F., J. L. SCHNIPPER, E. R. PARK, E. G. POON, Q. LI, B. MIDDLETON, Using Qualitative Studies to Improve the Usability of an EMR, Journal of Biomedical Informatics, vol. 38, issue 1, February 2005, pp. 51-60, ISSN 15320464.
WYATT, J. C., P. WRIGHT, Design Should Help Use of Patients’ Data, Lancet, 1998, p. 352.
NYGREN, E., J. C. WYATT, P. WRIGHT, Helping Clinicians to Find Data and Avoid Delays, Lancet, 1998, pp. 352, 1462-1466.