Ordinal classification or ordinal regression is a classification problem in which the labels have an ordered arrangement between them. Due to this order, alternative performance evaluation metrics are need to be used in order to consider the magnitude of errors. This paper presents a study of the use of a multi-objective optimization approach in the context of ordinal classification. We contribute a study of ordinal classification performance metrics, and propose a new performance metric, the maximum mean absolute error (MMAE). MMAE considers per-class distribution of patterns and the magnitude of the errors, both issues being crucial for ordinal regression problems. In addition, we empirically show that some of the performance metrics are competitive objectives, which justify the use of multi-objective optimization strategies. In our case, a multi-objective evolutionary algorithm optimizes an artificial neural network ordinal model with different pairs of metric combinations, and we conclude that the pair of the mean absolute error (MAE) and the proposed MMAE is the most favourable. A study of the relationship between the metrics of this proposal is performed, and the graphical representation in the two-dimensional space where the search of the evolutionary algorithm takes place is analysed. The results obtained show a good classification performance, opening new lines of research in the evaluation and model selection of ordinal classifiers.