USING GENETIC ALGORITHM WITH ADAPTIVE MUTATION MECHANISM FOR NEURAL NETWORKS DESIGN AND TRAINING Yuri R. Tsoy, Vladimir G. Spitsyn, Department of Computer Engineering Tomsk Polytechnic University http://qai.narod.ru neuroevolution@mail.ru Report contents 1. Introduction 2. Description of the algorithm 3. Adaptive mutation mechanism 4. Results of experiments 5. Implementation 6. Conclusion 1.Genetic algorithms and neural networks Genetic algorithms (GAs) use evolutionary concept (heredity, mutability and natural selection) to solve optimization tasks. The idea of Artificial neural networks (ANNs) is inspired by functionality of human brain. ANNs are often used to solve classification and approximation tasks 1. Use of neural networks 2. Description of the algorithm 3. Adaptive mechanism of mutation 3. Adaptive mechanism of mutation 4. Results of experiments 4. Future plans “Parameterless” variant of NEvA with adaptive mutation mechanism and adaptive population sizing. Some preliminary results: 5. Implementation The introduced algorithm is implemented to comply with the architecture of the software environment “GA Workshop” 5. “GA Workshop” At the time “GA Workshop” is under construction (there is no GUI), although it is already possible to use it for researches. The following researches have been done: Experiments with NEvA. Study of quasi-species model by M. Eugen. Study of majoring model by V.G. Red’ko. Investigation of simple population sizing techniques. Numerical optimization with use genetic algorithm. Experiments with compensatory genetic algorithm. Features: Features: 3 different variants of genetic algorithm, including “standard” GA, compensatory GA and NEvA; 5 different crossover operators for the binary encoded GA and 4 different crossover operators for NEvA; 4 selection strategies; 13 benchmark problems; 3 different population sizing strategies; Data available for analysis: Data available for analysis: Data that describes each generation of GA (fitness distribution for each generation averaged over multiple launches). Data that describes GA behavior (dynamics of averaged mean, the best and the worst fitness value for each generation, dynamics of averaged deviation of fitness, time per launch in milliseconds). Data that describes obtained solutions (number of object function calculations until solution is found, time until the first solution is found in milliseconds. All solutions and some data describing additional properties of the solutions are output into separate file for further analysis and use) Conclusion Results of experiments showed that NEvA performance is comparable and in some cases surpass results of other algorithms for the reviewed problems (XOR problem and full information pole balancing) Adaptive mutation rate caused increase of performance in comparison with NEvA with fixed Pm (up to 40% for 2-pole balancing task), although resulting networks were slightly worse for 1-pole balancing problems. Thank you for your attention! |