Automatic Construction of Graphical User Interfaces Semantic Models Using Robots for Mobile Application Testing

Feng XUE^1,2, Qingying LIU², Tao ZHANG²*, Shaoying LIU³, Jing CHENG⁴, Chunyan MA²
¹ School of Information, Xi’an University of Finance and Economics,
360 Changning Street, Changan District, Xi’an, 710100, China
120150792@qq.com
² School of Software, Northwestern Polytechnical University,
1 Dongxiang Road, Changan District, Xi’an, 710129, China
qyliu@mail.nwpu.edu.cn, tao_zhang@nwpu.edu.cn (*Corresponding author),
machunyan@nwpu.edu.cn
³ Graduate School of Advanced Science and Engineering, Hiroshima University, 1 Chome-3-2 Kagamiyama,
Higashi Hiroshima City, 739-8527, Hiroshima, Japan
sliu@hiroshimau.ac.jp
4 School of Computer Science and Engineering, Xi’an Technological University,
2 Xuefuzhong Road, Weiyang District, Xi’an, 710021, China
chengjing@xatu.edu.cn

Abstract: With the growing adoption of rich GUIs (Graphical User Interfaces) in mobile applications, researchers and practitioners have focused on GUI-based testing of the applications. Constructing a GUI model for AUT (Application Under Test) is a conventional strategy; nevertheless, automated modelling is typically grounded in source code logic, often lacking representation of functional semantics. However, manually constructing a GUI model enriched with semantics is inefficient. In this work, a GUI semantic model is proposed, namely FSM-ES (Finite State Machine with Extended Semantics), which not only reflects the syntactical structure of the GUI, but also the semantics of its elements. Moreover, visual technology is used to capture GUI information and use semantic ontology to guide robots in order to complete gesture actions such as clicking and sliding on the GUI, achieving the establishment of FSM-ES. The case study demonstrates that, while ensuring coverage of the core functions of the AUT, GUI semantic model proposed in this paper is 52% faster than manual modelling. Moreover, it facilitates the generation of test cases focused on functional semantics.

Keywords: Robotic testing, GUI semantic model, Automated modelling.

>>FULL TEXT: PDF

CITE THIS PAPER AS:
Feng XUE, Qingying LIU, Tao ZHANG, Shaoying LIU, Jing CHENG, Chunyan MA, Automatic Construction of Graphical User Interfaces Semantic Models Using Robots for Mobile Application Testing, Studies in Informatics and Control, ISSN 1220-1766, vol. 33(1), pp. 73-90, 2024. https://doi.org/10.24846/v33i1y202407