Metaheuristic Approaches to neural networks training

Abstract

In recent years, machine learning, and in particular neural networks (NN) have received much attention due to their numerous real-world applications. NN training is an essential step in building a model that can make reliable predictions based on given data. The process of NN training aims to find the optimal values for its internal parameters so that the network performs well on test data according to a given metric. The most common way to train an NN is to successfully use an optimizer based on gradient descent (GD). At each epoch, the optimizer updates the parameters based on the given data. In this paper, we are interested in using metaheuristics to guide the entire training process. The main idea is to identify the promising regions of the search space and invoke a GD -based optimizer in these regions as a local search procedure. For this purpose, we applied metaheuristics such as Variable Neighborhood Search and the Memetic algorithm to the NN training process and measured their performance on publicly available classification datasets, using classification accuracy as an evaluation metric.