Saturday , May 18 2024

Automatic Construction of Graphical User Interfaces Semantic Models Using Robots for Mobile Application Testing

Feng XUE1,2, Qingying LIU2, Tao ZHANG2*, Shaoying LIU3, Jing CHENG4, Chunyan MA2
1 School of Information, Xi’an University of Finance and Economics,
360 Changning Street, Changan District, Xi’an, 710100, China
2 School of Software, Northwestern Polytechnical University,
1 Dongxiang Road, Changan District, Xi’an, 710129, China, (*Corresponding author),
3 Graduate School of Advanced Science and Engineering, Hiroshima University, 1 Chome-3-2 Kagamiyama,
Higashi Hiroshima City, 739-8527, Hiroshima, Japan
4 School of Computer Science and Engineering, Xi’an Technological University,
2 Xuefuzhong Road, Weiyang District, Xi’an, 710021, China

Abstract: With the growing adoption of rich GUIs (Graphical User Interfaces) in mobile applications, researchers and practitioners have focused on GUI-based testing of the applications. Constructing a GUI model for AUT (Application Under Test) is a conventional strategy; nevertheless, automated modelling is typically grounded in source code logic, often lacking representation of functional semantics. However, manually constructing a GUI model enriched with semantics is inefficient. In this work, a GUI semantic model is proposed, namely FSM-ES (Finite State Machine with Extended Semantics), which not only reflects the syntactical structure of the GUI, but also the semantics of its elements. Moreover, visual technology is used to capture GUI information and use semantic ontology to guide robots in order to complete gesture actions such as clicking and sliding on the GUI, achieving the establishment of FSM-ES. The case study demonstrates that, while ensuring coverage of the core functions of the AUT, GUI semantic model proposed in this paper is 52% faster than manual modelling. Moreover, it facilitates the generation of test cases focused on functional semantics.

Keywords: Robotic testing, GUI semantic model, Automated modelling.


Feng XUE, Qingying LIU, Tao ZHANG, Shaoying LIU, Jing CHENG, Chunyan MA, Automatic Construction of Graphical User Interfaces Semantic Models Using Robots for Mobile Application Testing, Studies in Informatics and Control, ISSN 1220-1766, vol. 33(1), pp. 73-90, 2024.