Informatics Tools, AI Models and Methods Used for
Automatic Analysis of Customer Satisfaction
George KOVÁCS1, Diana BOGDANOVA2,
Nafissa YUSSUPOVA2, Maxim BOYKO2
1 Computer and Automation Research Institute,
Kende u. 13-17, Budapest, 1111, Hungary
2 Ufa State Aviation Technical University,
K. Marx 12, Ufa, 450000, Russia
firstname.lastname@example.org; email@example.com; firstname.lastname@example.org
Abstract: Customer satisfaction is getting more and more importance world-wide. Informatics tools and methods are used to research customer satisfaction based on a detailed analysis of consumer reviews. The examined reviews are written in natural languages and some Artificial Intelligence (AI) techniques such as Text Mining, Aspect Sentiment Analysis, Data Mining and Machine Learning are used for the study. As input for running the investigations, we use different internet resources in which the accumulated customer reviews are available. These are for example yelp.com, tripadviser.com and tophotels.ru, etc. To see and show the efficacy of the proposed approach, we have carried out experiments on hotel client satisfaction. The results have proven the effectiveness of the proposed approach to decision support in product quality management and support applying them instead of traditional methods of qualitative and quantitative research of customer satisfaction.
Keywords: quality management; customer satisfaction research; decision support system; sentiment analysis.
CITE THIS PAPER AS:
George KOVÁCS, Diana BOGDANOVA, Nafissa YUSSUPOVA, Maxim BOYKO, Informatics Tools, AI Models and Methods Used for Automatic Analysis of Customer Satisfaction, Studies in Informatics and Control, ISSN 1220-1766, vol. 24 (3), pp. 261-270, 2015.
Quality assurance is currently realized by means of a process approach based on the model of a quality management system . It describes the interaction of the company and the customer during the process of product production and consumption. To correct the parameters of product quality in order to improve it for the customer, the models include feedback. For companies, one aspect of feedback during the process of quality management is information about the level of customer satisfaction, expressed in the form of customer reviews of the product quality. That is why customer satisfaction is the key information in quality management that influences decision-making. To collect data and to evaluate customer satisfaction, the International Quality Standard ISO 10004 recommends using the following methods: personal and phone interviews, discussion groups, mail surveys (postal questionnaires), online research and survey (questionnaire survey) . However, these methods of collecting and analyzing customer opinions show a number of drawbacks. A general drawback of the recommended methods is the need for a large amount of manual work: preparing questions, creating a respondent database, mailing questionnaires and collecting results, conducting personal interviews, preparing a report based on the results.
All this increases the research costs. Due to their discreteness these methods do not allow for the continuous monitoring of customer satisfaction. For this reason, the data analysis is limited to one time period and does not give an insight into the trends and dynamics of customer satisfaction. This also has a negative influence on the speed of managerial decision making, which depends on the arrival rate of up-to-date information about customer opinions. Existing scales of customer satisfaction and their subjectivity perception raise questions. Values of customer satisfaction expressed in the form of abstract satisfaction indices make it difficult to understand, compare and interpret the results. Methods of analysis of data collected through the recommended ISO 10004 procedures permit only the detection of linear dependencies. To increase the effectiveness of product quality management, we suggest approaching the research of customer satisfaction through the use of Informatics, as AI technologies. Applying Text Mining tools for analyzing customers’ reviews posted on the Internet is not novel. There are many studies concerning models and methods for data collection, sentiment analysis and information extraction. Recent studies show acceptable accuracy of methods for sentiment classification.
Gräbner et al.  proposed a system that performs the sentiment classification of customer reviews on hotels. The precision values are 84% for positive and 92% for negative reviews. Lexicon-based method  allowed the correct classification of reviews with a probability of about 90%. These achievements make sentiment analysis applicable for an application on quality management and customer satisfaction research. Jo and Oh  and Lu et al.  considered the problems of automatically discovering products’ aspects and sentiments estimation for these aspects, which are evaluated in reviews. For solving these problems, they suggested methods based on Latent Dirichlet Allocation  and its modifications. The main drawback of most social monitoring systems and frameworks for automatic analysis of reviews is that they can provide entirely only a quantitative survey of customer reviews, i.e., they can provide measurement of the degree of customer satisfaction with a product and its aspects, sometimes with a model . Qualitative survey were usually only conducting the extraction of products’ aspects. However, estimation of the significance of each products’ aspects for the customer is missed.
The information about products’ aspects that influence customers’ satisfaction and relative importance of products’ aspects for the customers is missing, as well as an insight into customer expectations and perceptions.
The most related work to this problem is . It is dedicated to the topic of aspect ranking, which aims to automatically identify important aspects of product from online consumer reviews. Most proposals used a probabilistic model with a large number of parameters that lead to low robustness of the model. Total weighting values of aspects are calculated as the average of the weighting values by each review.
Finally, significance values of aspects are estimated independently of sentiments of opinions. In real life we can speak about bad “signal connection”, in a review, but we usually omit comments int he case of good “signal connection”, as it should be caused by the phone. In our paper, we estimate significance values of aspects in accordance with their positive and negative sentiments. In this paper, for qualitative survey is used a novel approach based on transformation results of sentiment analysis and aspect-based sentiment analysis, such as sentiment labels of reviews and mentions about product’s aspects in reviews, into boolean data. After that, boolean data is processed with a data mining tool – decision tree. Qualitative survey aims to identify how the sentiment of reviews depends on the sentiment of different products’ aspects. In other words, how overall customer satisfaction with product depends on the customer satisfaction with a product’s aspects. Decision tree performs this aim and identifies latent relations between the sentiment of reviews and sentiment of a product’s aspects. Also using the decision tree allows to estimate the significance of product’s aspects for the customers. The output of the qualitative survey contains significant values of the product’s aspects for customers, and identifies latent relations between satisfaction with the product and satisfaction with each product’s aspect. These were produced as rules extracted by the decision tree.
The availability of both quantitative and qualitative surveys allows realizing Intelligent Decision Support System for Quality Management in accordance with quality standard ISO 10004.
- ISO9000:2008. The quality management system. Fundamentals and vocabulary.
- ISO10004:2010. Quality management. Customer satisfaction. Guidelines for monitoring and measuring.
- GRÄBNER, D., M. ZANKER, G. FLIEDL, M. FUCHS, Classification of Customer Reviews based on Sentiment Analysis, Proceedings of the International Conference in Helsingborg, Springer Vienna, 2012, pp. 460-470.
- TABOADA, M., J. BROOKE, M. TOFILOSKI, K. D. VOLL, M. STEDE, Lexicon-Based Methods for Sentiment Analysis, Computational Linguistics, June 2011, vol. 37(2), pp. 267-307.
- JO, Y., A. OH, Aspect and Sentiment Unification Model for Online Review Analysis, Proceedings of the fourth ACM international conference on Web search and data mining (WSDM ‘11), ACM New York, Feb. 2011, pp. 815-824.
- LU, B., M. OTT, C. CARDIE, B. TSOU, Multi-aspect Analysis with Topic Models, Proceedings of the 2011 IEEE 11th International Conference on Data Mining Workshops (ICDMW ‘11), Dec. 2011, pp. 81-88.
- BLEI, D. M., A. Y. NG, M. I. JORDAN, Latent Dirichlet allocation, Journal of Machine Learning Research, Jan. 2003, vol. 3 (4–5), pp. 993-1022.
- YU, J., Z.-J. ZHA, M. WANG, T.-S. CHUA, Aspect Ranking: Identifying Important Product’s aspects from Online Consumer Reviews, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (HLT ‘11), June 2011, pp. 1496-1505.
- HORVÁTH, L., I. RUDAS, New Method for Intellectual Content Driven Generic Product Model Generation 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE, 2014. pp. 1660-1665.
- RUDAS, I., Cloud Technology-Based Education with Special Emphasis on Using Virtual Environment Proceedings of the 14th WSEAS Conf. 2014.01.292014.01.31. Cambridge. USA, p. 23.
- THOMSEN, J., E. ERNST, C. BRABRAND, M. SCHWARTZBACH, WebSelF: A Web Scraping Framework, Proceedings of the 12th international conference on Web Engineering (ICWE 2012), July 2012, pp. 347-361.
- YUSSUPOVA, N., D. BOGDANOVA, M. BOYKO, Applying of Sentiment Analysis for Texts in Russian Based on Machine Learning Approach, IMMM 2012, Venice, Italy, pp. 8-14.
- EBERT, S., N. T. VU, H. SCHÜTZE, CIS-positive: Combining Convolutional Neural Networks and SVMs for Sentiment Analysis in Twitter In: Proceedings of the 9th International Workshop on Semantic Evaluation. SemEval 2015.
- TURNEY, P. D., Y. NEUMAN, D. ASSAF, Y. COHEN, Literal and Metaphorical Sense Identification through Concrete and Abstract Context. In: EMNLP., 2011, pp. 680-690.
- DURRANI, N., H. SCHMID, A. FRASER, P. KOEHN, H. SCHÜTZE, The Operation Sequence Model – Combining N-Grambased and Phrase-based Statistical Machine Translation. Computational Linguistics, vol. 41(2), 2015, pp. 157-186.
- YUSSUPOVA, N., M. BOYKO, D. BOGDANOVA, A. HILBERT, A Decision Support Approach based on Sentiment Analysis Combined with Data Mining for Customer Satisfaction Research, The International Journal on Advances in Intelligent Systems is Published by IARIA vol 8, no 1&2, 2015.