Pareto Neuro-Evolution: Constructing Ensemble of Neural Networks Using Multi-objective Optimization

Abstract

In this paper, we present a comparison between two multiobjective formulations 
to the formation of neuro-ensembles. The first formulation splits the training set 
into two nonoverlapping stratified subsets and form an objective to minimize the 
training error on each subset, while the second formulation adds random noise to 
the training set to form a second objective. A variation of the memetic Pareto artificial 
neural network (MPANN) algorithm is used. MPANN is based on differential evolution 
for continuous optimization. The ensemble is formed from all networks on the Pareto 
frontier. It is found that the first formulation outperformed the second. The first 
formulation is also found to be competitive to other methods in the literature.