Genetic optimization of type-1, interval and intuitionistic fuzzy recognition systems

In this paper a new method for fuzzy system optimization is presented. The proposed method performs the intuitionistic or type-2 fuzzy inference system design using a hierarchical genetic algorithm as an optimization method. This method is an improvement of a fuzzy system optimization approach presented in previous works where only the optimization of type-1 and interval type-2 fuzzy inference systems was performed considering a human recognition application. Human recognition is performed using three biometric measures namely iris, ear, and voice, where the main idea is to perform the combination of responses in modular neural networks using an optimized fuzzy inference system to improve the final results without and with noisy conditions. The results obtained show the effectiveness of the proposed method for designing optimal structures of fuzzy systems. The design of optimal structures of fuzzy systems include among other parameters; type of fuzzy logic (Type-1, interval type-2 and intuitionistic fuzzy logic), type of model (Mamdani model or Sugeno model), and consequents of the fuzzy if-then rules.


Introduction
Human recognition using biometric measures has demonstrated to be a good approach for authentication systems [11,12,13,28], because these systems have as main advantage that a biometric measure cannot be stolen [10,26,27,32] some of these biometric measures are ear, iris, fingerprint, voice and, hand geometry, among others. A widely known model to accomplish this task is the modular neural network (MNN). This kind of neural network model has a significant learning improvement comparatively to a single neural network [12,13,21]. In modular neural networks, a problem is divided into smaller sub problems and their partial solutions or responses are combined to produce a final solution [10,30,34].
Soft computing consists of different techniques, among them fuzzy logic (FL), and this area can be highlighted as an approach that allows to reason with uncertainty. Fuzzy inference systems (FIS) are based on the concepts of fuzzy set theory, fuzzy if-then rules, and fuzzy reasoning. In a type-1 fuzzy inference system (T1 FS), fuzzy sets allow the membership degree to be any value between 0 and 1, i.e. a crisp number. In the real world, this can help when in a situation it is difficult to decide if something belongs or not to a specific class [37][38][39]. In an interval type-2 fuzzy inference system (IT2 FS), the membership degree for each element of this set is a fuzzy set in [0, 1] interval [3,5,19]. In a general type-2 fuzzy inference system (GT2 FS), three-dimensional membership functions (3D MF) are used and can be represented in four different ways: points, wavy slices, horizontal slices and vertical slices. In intuitionistic fuzzy systems uncertainty is modeled using membership and non-membership functions. Therefore, intuitionistic and type-2 fuzzy inference system are described using more parameters than a type-1 and an interval type-2 fuzzy inference system [20,22]. It is worth mentioning that these fuzzy inference systems have been successfully applied in different areas, such as data classification, control problems, time series prediction, robotics, and human recognition [3].
It is important to mention that, a FIS needs to have an optimal structure to obtain better results, and this can be achieved using the knowledge of an expert or some optimization technique [4,16], examples of the structural information are parameters of the membership functions and fuzzy if-then rules. Among some of these techniques applied to fuzzy inference system optimization, we can find; genetic algorithms (GAs) [14], particle swarm optimization (PSO) [36], cuckoo optimization algorithm (COA) [24], the bat algorithm (BA) [8] and chemical reaction optimization (CRO) [1].
In this work, fuzzy inference systems are used as integration units to combine MNN responses and hierarchical genetic algorithms (HGAs) are used to perform fuzzy inference system optimization. This optimization technique is a special kind of genetic algorithm. The difference is found in its structure because, a HGA is more flexible than a conventional GA [2,15]. The main difference is found in the use of control genes that determinate the behavior of the other genes. These chromosomes may provide a good way to solve the problem and have demonstrated to achieve better results in complex problems than the conventional GA [9,31,35]. This technique was also chosen because it is one of the most used techniques in a wide range of applications because the same nature and familiarity of the algorithm allow its easy application. The application and adjustment of parameters of other algorithms previously mentioned force to have more specialized knowledge about the area in which it is inspired. This paper combines MNNs, FL and HGA, because their combination have demonstrated the effectiveness of a hybrid intelligence approach, where the individual limitations of the particular methods have been satisfied [18,23,25,33].
The optimization is mainly performed because if we do not know the correct fuzzy inference system parameters and its structure, then it is almost impossible to achieve the best performance in any application, or at least if we have an expertise in the area where we will make the application perhaps good results could be obtained. For this reason, in this paper a HGA is proposed to perform the optimization of the structure and parameters of the FIS.
Experimental results of the proposed method show that the optimal architecture of fuzzy inference systems can be obtained and as a consequence the recognition rates can be improved with respect to previous works. This paper is organized as follows. The description of the proposed method is presented in Section 2. In Section 3, the description of the databases and application are presented. The obtained results are explained in Section 4. The statistical comparisons of results are presented in Section 5. Finally, conclusions are offered in Section 6.

Intuitionistic fuzzy logic systems
According to Atanassov [1], an IFS on the universum X ≠ ∅ is an expression A given by: where the functions µA, νA : X → [0, 1] (2) satisfy the condition and describe, respectively, the degree of the membership µA(x) and the non-membership νA(x) of an element x to A. Let therefore, function πA determines the degree of uncertainty.
For the first case we have: For the second case we have: In the graphical representation in both cases above 1, This is just a brief description of the extensive work on this area of Prof. Atanassov.

Proposed method
The proposed method combines the responses of MNNs using FL as response integrator, where each MNN deals with a biometric measure and provides an input to the fuzzy integrator. The proposed method allows the use of "N" number of inputs, this parameter is established depending on the responses to be combined, i.e., the number of modular neural networks and, a final output is obtained, in this case represented with an output in the fuzzy integrator. In this work, three modular neural networks are used to compare with previous works, where each MNN performs the identification using a different biometric measure, and the fuzzy inference system is shown in Figure 1.

Optimization of fuzzy inference systems
The proposed HGA performs mainly the optimization of type-2 and intuitionistic fuzzy systems, but for reasons of comparison with previous works the type-1 and the interval type-2 fuzzy inference system optimization is also performed because some improvements are obtained with respect to those previous optimizations, such as: number of membership functions (one previous work [21] only uses 2 or 3 membership functions in each variable), the fuzzy rules (the previous work [21] randomly selected them), and in [29] optimizations do not have a reinitialization process. Having said that, the proposed method allows the optimization of the type of fuzzy logic (type-1, interval type-2 fuzzy logic or general type-2 fuzzy logic), type of system (Mamdani Model or Sugeno Model), type of membership functions (Trapezoidal or GBell), number of membership functions in each inputs and output variables, their parameters, the consequents of the fuzzy rules and the fuzzy if-then rules. The chromosome of the proposed hierarchical genetic algorithm for the fuzzy inference systems is shown in Figure 2, where control genes allow determining the type of fuzzy logic, type of system and number of MFs for each variable (inputs and outputs) and depending of their values, the rest of the genes are activated.

Optimization of membership functions
The number of membership functions in each variable (inputs and output), the type of these membership functions (MFs) and their parameters are all considered in the optimization process. A contribution of the proposed method is in this point, because usually when the optimization of fuzzy inference systems is performed, the number of MFs is always fixed, i.e. only the membership function parameters are optimized, and when the optimization of the type of membership functions is performed all the membership functions of all the variables are of the same type. In the proposed optimization, each MF of each variable (inputs and output) is optimized, i.e., the combination of MFs in the same variable and different number of membership functions in each variable can be possible depending on the type of fuzzy logic. In this paper, only 2 types of MFs are used (Trapezoidal and GBell). The question would be, why are we using only these kinds of membership functions in this work?. The answer is easy, because the hierarchical genetic algorithm has a wide range of search, allowing for example when trapezoidal membership functions are used, their points can be very close to each other and looking to be like a triangular membership function. The same case occurs with the GBell membership function when their points are modified; it can look to be like a Gaussian membership function.

Optimization of fuzzy rules
The proposed optimization performs the fuzzy if-then rules design in two parts; the consequents and the number of fuzzy rules (using the activation genes). To perform the consequents optimization, depending of the maximum number of membership functions (MNMFs) used in each variable (this number is freely established before the evolution) all the possible rules are generated. To illustrate an example, we will assume that the fuzzy inference system has 3 inputs and 1 output, and the MNMFs are 3, i.e., each input can have up to 3 membership functions in each variable. The total number of possible fuzzy if-then rules is given by the equation: where N corresponds to the number of inputs in the fuzzy inference system and MNMFs corresponds to the maximum number of membership functions. In this work, N is 3, MNMFs is 5 and TNFR is 125. These fuzzy rules are shown in Figure 3, where 125 fuzzy rules are represented, and the genes for the consequents and activations of these fuzzy rules are used to complete the fuzzy rules used for the optimized fuzzy inference system. In the case of the consequents, the chromosome will have 125 genes for each possible number of membership functions, i.e. in the example described above, there are 125 genes, where the values can be from 1 to 2 (for when the output of the fuzzy inference system has only 2 membership functions), 125 genes where the value can be from 1 to 3 (for when the output of the fuzzy inference system has 3 membership functions) and so on. For the activations, there are 125 genes (one for each fuzzy rule).
When an individual of the HGA is going to be evaluated, depending of the number of membership functions indicated by the genes for the inputs. We are going to assume that based on the chromosome value of an individual, this one form the FIS shown in Figure 4, this FIS is used to perform human recognition and its parameters were obtained by the proposed method, where the number of membership functions for each input is respectively 2, 5 and 3. The possible fuzzy rules for this combination are considered with the consequents and activations values. These fuzzy if-then rules can be observed in Figure 5.
The consequents are taken depending on the number of membership functions indicated by the gene for the output. As in Figure 4 can be observed, the fuzzy inference system has 3 membership functions in the output, for this reason the consequents are considered from the genes with values from 1 to 3 as Figure 6 shows.  The number of fuzzy if-then rules is found by the genes that indicate the activation or deactivation of the fuzzy rules (as it was mentioned, each fuzzy rule has a particular gen). If the value of the gene is 0 the fuzzy rule is not used, and if the value is 1 it is used, this is for all the possible fuzzy rules depending on the combination indicated by the genes for the inputs. In Figure 7, the process when only the fuzzy rules with an activation gene with value 1 are chosen can be observed. Finally, the resulting fuzzy if-then rules are formed and added to the fuzzy integrator and the fuzzy inference system can be used and evaluated. In Fig. 8, the resulting fuzzy if-then rules are shown. It is important to remember that the antecedents of the fuzzy rules are automatically generated by the proposed method. The consequents and the use of each fuzzy rule (activation) are determined and optimized by the genes in the hierarchical genetic algorithm. We can summarize this process in Figure 9 and with 4 main steps: 1. The MNMFs is 5, i.e. all possible fuzzy rules are 125. 2. The fuzzy inference system has 2, 5 and 3 MFs respectively in the inputs, and the possible rules for these combinations are 30. 3. As the output has 3 MFs, the consequents with values among 1 to 3 are used, 4. Only the fuzzy rules with activated genes are added to the fuzzy integrator ( Figure 9). In the proposed method, the values for the minimum and maximum number of membership functions are from 2 to 5, but this range can be increased or decreased, the name of each membership functions (linguistic labels) depending of the number of membership functions used in the fuzzy inference system are shown in Table 1, and these names can be also easily modified. For the proposed method, the number of alpha planes in the optimization a range from 50 to 200 is established, because in [20] good results were found in this range. The genetic parameters [15] used to test the proposed hierarchical genetic algorithm are shown in Table 2, and these parameters are the same used in [21] and [29].

Elitism and reinitialization process
The elitism used for the fuzzy inference optimization is the conventional method, where the individual with the best performance is saved to avoid being modified with the genetic operators. The reinitialization process is activated when in the evolution, the objective error does not change during 10 generations, and then a new population is generated. In this population, the best individual of the last generation is inserted and the rest of the population is randomly generated and the evolution continues.

Comparison with previous works
In [21], the maximum number of membership functions was from 2 to 3, only 2 types of fuzzy logic were optimized: type-1 and interval type-2 fuzzy logic, and the fuzzy rules were randomly created based on a percentage provided by a gene. In [29], the maximum number of membership functions was from 2 to 5, also only 2 types of fuzzy logic were optimized: type-1 and interval type-2 fuzzy logic, and the consequents of the fuzzy rules were optimized. This optimization does not perform intuitionistic fuzzy systems optimization and do not have a reinitialization process. To understand the difference between the previous works and the proposed method, Table 3 presents a summary of the most important aspects of the optimizations.

Human recognition application
In this section, the application and databases used to test the proposed method are described in more detail.

Modular neural networks
A modular neural network is a kind of an artificial neural network (ANN), where the computation performed by a MNN is decomposed into two or more modules. In this work, 3 modules are used in each MNN, where each module learns information of a certain amount of persons. In this work, the human recognition of 77 persons is performed and the division of persons per module is illustrated in Figure 10. As it was already mentioned, the final output of each MNN is its maximum activation, as 3 MNNs are used, 3 activations with values between 0 and 1 will be the inputs of the FIS, and using the fuzzy if-then rules a final output can be obtained.

Human recognition
The general architecture of the proposed method presented in [21] is shown in Figure 11, each modular neural network (MNN) is design with 3 modules. In [21], the optimization of the modular neural networks and fuzzy inference systems are performed. In that work 3 biometric measures are used: iris; ear and voice, one modular neural network (MNN) for each biometric measure where, the responses of each modular neural network are combined using an optimized fuzzy inference system. Figure 11. Architecture of the MNN for person recognition based on iris, ear and voice biometrics [21] The hierarchical genetic algorithm aims at minimizing the fitness function (error of recognition). The fitness function is given by: where Xi is 0 if the person is correctly identified and 1 if not, and T is total number of data points (combinations) used for testing. For comparison with the previous work, the recognition rate is shown and given by:

Databases
In this section, the databases used in [21] and [29] are presented. The human recognition is performed for 77 persons, and for this reason only the first 77 persons of each database are used. These databases were chosen, because they were also used by other authors [10][12] [13].

Iris Database
The database of human iris from the Institute of Automation of the Chinese Academy of Sciences (CASIA) was used [7]. Each person has 14 images (7 for each eye). The image dimensions are 320 × 280, JPEG format. In Figure 12 some examples of the human iris images are shown. As image preprocessing, the method developed by [17] is used to find the coordinates and radius of the iris and pupil, iris is cut and a new image with a dimension of 21 × 21 is produced, finally the images are converted from vector to matrix form. Figure 12. Examples of the human iris images from CASIA database

Ear Database
The database of the University of Science and Technology of Beijing is used [6]. The image dimensions are 300 × 400 pixels, BMP format, and each person has 4 images. Figure 13 shows examples of the human ear images. For this database, a cut of the ear manually, a new image is created and resized to 132-91 and finally, the images are converted from vector to matrix form.

Voice Database
This database was collected from students of Tijuana Institute of Technology. Each person has 10 voice samples. Each sample is in the Microsoft's audio file format WAV. The word that they said in Spanish was "ACCESAR". Mel Frequency Cepstral Coefficients were used to preprocess the voice.

Experimental results
In this section, a comparison of the results achieved using a previous method [21] and the proposed method is shown, and a summary results and comparisons of results achieved comparing with [29] are shown in section 4.3. This work is focus on the fuzzy inference systems, where the combinations of activations of the biometric measures are performed using an improved fuzzy inference system, i.e. the results shown in this section are obtained of the combination of 462 sets (T = 462).

Previous results
In [21], 7 cases were established for combining different trainings of iris, ear and voice, using non-optimized and optimized trainings (in that work also a modular neural network optimization was proposed). These 7 cases were used without noise and with noise (Gaussian) in the images for the testing phase. In Tables 4 and 5, the training results and their combinations are presented.  Table 5. Trainings used for forming the seven cases (with noise) [21] The results achieved in [21] using its HGA are shown in Tables 6, where cases without noise and noise are shown, for noise cases Gaussian noise (statistical noise having a probability density function) was added to the images and voice samples, where the best, average and worst results are presented (of 20 evolutions). The proposed method in this paper has now the challenge of improving these results.

Results of the proposed method
The results achieved, using the proposed method, are shown below. In this case, 20 evolutions for each case without noise and noise are performed. Each evolution allows to generate a new best FIS with different: type of fuzzy logic, number of MFs in each variable, parameters of these MFs and fuzzy if-then rules.

Results of cases without noise
The best, average and worst results of the previous work [21] and the proposed method are shown in Table 7. In those cases where a 100% of recognition rate had not been obtained in [21], the recognition rate is now improved using the proposed method. In Figure 14, the averages of each method are shown, where the results obtained using the proposed method are better in almost all the cases (except in those cases where a 100 of recognition rate had been already obtained in [21]).

Case
The best evolution of case # 3 is the evolution #11, and the convergence can be observed in Figure 15. In this case, a 100% of recognition rate is obtained in only 33 generations. The best fuzzy inference system for this case is using type-1 fuzzy logic, Sugeno type, using 12 fuzzy rules out of 20 possible fuzzy rules. The inputs of the fuzzy integrator are shown in Figure 16, and the fuzzy rules generated by the proposed hierarchical genetic algorithm are shown in Figure 17. The best evolution of case #6 is evolution #4, and the convergence behavior can be observed in Figure 18. In this case, a 100% of recognition rate is obtained in only 95 generations. The best fuzzy inference system for this case is using interval type-2 fuzzy logic, of Sugeno type, using only 12 fuzzy rules out of 30 possible fuzzy rules. The inputs of the fuzzy integrator are shown in Figure 19, and the fuzzy rules generated by the proposed hierarchical genetic algorithm are shown in Figure 20.

Results of cases with noise
The best, average and worst results of the previous work [21] and the proposed method are shown in Table 8. In all cases, the proposed method in this paper allows to improve the previously obtained results, especially if we observe the worst values, they are also improved with the proposed method. In Figure 21, the averages of each method are shown, where the results obtained, using the proposed method, are better in all the cases. The best evolution of case # 1 is evolution #14, and the convergence behavior can be observed in Figure 22. In this case, a recognition rate of 92.86% is obtained. Figure 22. Convergence of case #1 (evolution #14)

Conclusion
In this paper, a method that combines modular neural networks (MNNs) responses using fuzzy logic as response integrators was proposed. A hierarchical genetic algorithm is proposed to optimize fuzzy inference systems, where the main contribution of the proposed method is to allow the optimization of the type of fuzzy logic (Type-1, Interval Type-2 and Intuitionistic fuzzy logic) and allow combination of different type of membership functions in the same variable and mainly the number of membership functions in each variable (inputs and output) with a range from 2 to 5 (but this range can be increased or decreased) and the optimization of fuzzy rules (number of rules and consequents), and a reinitialization process which had not been proposed in other previous works. The optimization of the type of system (Mamdani Model or Sugeno Model) is also proposed. Using a statistical comparison, we can prove that the results are significantly improved when the proposed method is used, i.e. that the increase of the range of the number of membership functions, optimization of fuzzy rules (number and consequents) and the use of intuitionistic and type-2 fuzzy logic allow improving the results.