Novel Algorithms for Multi-Objective Search and their Application in Multi-Objective Evolutionary Neural Network Training


Abstract

In a wide variety of application areas there is a need to trade-off competing objectives to achieve a resolution to a specific problem. Where the interdependencies of these objectives are unknown (as is often the case), this involves the searching and storing of a set of potential problem solutions which, without objective preference knowledge, cannot be said to be any better or worse than other members of the set. Many studies over the last 18 years have investigated ways to improve the efficiency of the search process for these solutions; principally through the tools of evolutionary computation. This increased efficiency has been manifest in the discovery of the best set of solutions in a fewer number of function (problem) evaluations, the lower the computational cost and the better the search process is judged to perform. A number of studies have focused on the additional computation complexity of some of the more advanced methods, however one major cause of realised run-time has been largely ignored. The current approach in the literature is to store potential solutions in a linear list. This means that a large proportion of an optimiser's time can be actually spent comparing stored solutions with new solutions, as opposed to function evaluation of solutions, or the search process itself. The first part of this work confronts this problem by developing new data structures for the representation of multi-dimensional points, which can be used in multi-objective search processes. These new data structures are shown to be significantly faster than linear lists and operational proofs are also derived that evinces this. The second part of the thesis is concerned with the benefits of these new data structures to multi-objective search beyond their application within a general framework - to their specific use in facilitating novel optimisation techniques. This is shown with the development of empirical validation of a multi-objective particle swarm optimisation model. The final part of the work is concerned with the development of a multi-objective evolutionary neural network framework. Until this point the technique of choice in this field has been the linear weighting method, whose shortcomings have been amply demonstrated in the general multi-objective optimisation field. This section therefore transfers the recent advances in multi-objective optimisation to a neural network training framework, and develops novel generalisation techniques to deal witht the unique properties of multi-objective error minimisation that are not apparent in the uni-objective case. Empirical validation is provided in terms of test problems from the literature and an extensive financial forecasting application.