Amel Sifaoui
U.R.Automatique, ENIT École Nationale d'Ingénieurs de Tunis (ENIT) BP 37, Le Belvédère 1002 Tunis, Tunisie | Afef Abdelkrim
U.R.Automatique, ENIT Ecole Supérieure de Technologie et d'Informatique (ESTI) |
Sonia Alouane
École Nationale d'Ingénieurs de Tunis (ENIT) BP 37, Le Belvédère 1002 Tunis, Tunisie | Mohamed Benrejeb
U.R.Automatique, ENIT École Nationale d'Ingénieurs de Tunis (ENIT) BP 37, Le Belvédère 1002 Tunis, Tunisie |
Abstract: The proposed method to construct a Radial Basis Function (RBF) neural network classifier is based on the use of a new algorithm for characterizing the hidden layer structure. This algorithm, called HNEM-k-means, groups the training data class by class in order to calculate the optimal number of clusters in each class, using new global and local evaluations of the partitions, obtained by the k-means algorithm. Two examples of data sets are considered to show the efficiency of the proposed approach and the obtained results are compared with previous existing classifiers.
Keywords: Radial Basis Function neural network, classification, k-means, validity indexes.
Amel Sifaoui has obtained the Master of Science in Computer Science in 1998 at Faculty of Sciences of Tunis (FST) and the PhD in Electrical Engineering at the National School of Engineers of Tunis (ENIT) in 2008; she is now a Laboratory Assistant at the Faculty of Sciences of Gafsa. Her research interests are in the artificial neuronal network domain.
Afef Abdelkrim has obtained a Diploma of Engineer in 2000 and the PhD in Electrical Engineering at the National School of Engineers of Tunis (ENIT) in 2005; she is now an Assistant Professor at the High School of Technology and Computer Science (ESTI). Her research interests are in handwriting process modelling and synthesis and in the artificial neuronal network domain.
Sonia Alouane has obtained a Graduate Diploma in Computer Science applied to management in 1994 at Institut Supérieur de Gestion Tunis and a post Graduate Diploma a specified for teaching Computer Science and technologie in National School of Computer Science (ENSI) called "Technologue" in 2000. Presently, she is teaching as "Assistante technologue" at the National School of Engineers of Tunis (ENIT), her research interests are in the artificial neuronal network domain.
Mohamed Benrejeb has obtained the Diploma of "Ingénieur IDN" (French "Grande Ecole") in 1973, the Master degree of Automatic Control in 1974, the PhD in Automatic Control of the University (USTL) of Lille in 1976 and the DSc of the same University in 1980. He is currently a full Professor at the Ecole Nationale d'Ingénieurs de Tunis (Tunisia) and an Invited Professor at the Ecole Centrale de Lille (France). His research interests are in the area of analysis and synthesis of complex systems based on classical and non conventional approaches and recently in discrete event system domain.
Full text
CITE THIS PAPER AS:
Amel Sifaoui, Afef Abdelkrim, Sonia Alouane, Mohamed Benrejeb, On New RBF Neural Network Construction Algorithm for Classification, Studies in Informatics and Control, ISSN 1220-1766, vol. 18 (2), pp. 103-110, 2009.
1. Introduction
Introduced into the neural network literature by Broomhead and Lowe [1], the radial basis function neural networks have been widely used for function approximation, pattern classification and recognition due to their structural simplicity and faster learning abilities [2, 3]. However, their design still remains a difficult task due to the absence of systematic method giving an optimal architecture. A too small network cannot well learn the problem, but a too large network will lead to over-fitting and poor generalization performance [4].
A very important step for the RBF network training is to decide a proper number of hidden neurons (number of basis functions), because it controls the complexity and the generalization ability of RBF networks, so many works have proposed RBF classifier conception algorithms [5-11].
The radial basis function network is three layers feedback network, typically used for supervised classification. Its training procedure is usually split into two successive steps. First, the centers of the hidden layer neurons are selected by clustering algorithms such as k-means [11, 12], vector quantization [13], decision trees [14], and then the widths are calculated [15]. Second, the weights connecting the hidden layer with the output layer are determined by Singular Value Decomposition (SVD) or by Least Mean Squared (LMS) algorithms [16].
In this paper, a new learning algorithm is proposed for construction of the radial basis function networks solving classification problems. It determines the proper number of hidden neurons and calculates the centers values of the radial basis functions. After the selection of the centers of the hidden neurons, the widths of the nodes are determined by the P-nearest neighbour heuristic, and the weights between the hidden layer and the output layer calculated by the pseudo-inverse matrix.
The proposed approach consists of combining new evaluation measurement with the k-means algorithm, which have led to a new algorithm, called HNEM-k-means, allowing the automatic determination of the clusters number in the data of each class. Two different real databases are used in order to evaluate the classifier performances.
6. Conclusion
In this paper, a new algorithm, to design RBF neural networks classifiers and to select the centers of the hidden layer neurons in particularly, is proposed.
Based on pruning technique, it aims to construct the hidden layer of an RBF neural network and starts with the maximum number of groups, which is decreased during the different iterations of the algorithm.
The basic idea of this approach is to gather the training data class by class and to decide of the optimal number of groups in each class by using the proposed validity indexes measure which are integrated in the k-means algorithm.
The obtained classifier results are satisfactory in comparison with other considered classifiers in the literature for two real databases.
References