Optimizing multilayer perceptron neural network hyperparameters

Abstract

Predictions across multiple disciplines rely on the efficacy of fundamental artificial neural networks, such as multilayer perceptron (MLP). Optimization is a key step in improving predictive performance of these models. In this study, a global nonlinear neural model was developed to predict the impact of using the digital technology in society, the economy and public administration on economic development. By conducting 17 experiments on the basic MLP neural network, authors investigated the effects of modifying the network architecture, learning rate and type of activation function. Standard measures of model errors and coefficient of determination were employed as criteria to prioritize configurations using the PROMETHEE II multi-criteria approach. The results reveal that models featuring two hidden layers, reduced learning speed and adequate activation functions achieve optimal performance with MSE16=0.012, RMSE16=0.110, MAPE16=12.186 and R²16=0.719. Conversely, too complex models complicate the learning process and lead to imprecise predictions as in the case of MSE5=0.019, RMSE5=0.138, MAPE5=17.225 and R²5=0.559. The results indicate the importance of adjusting the neural network hyperparameters to the nature of the research problem. Additionally, the study reveals the important role of MCDM in choosing the most adequate configuration when considering diverse criteria with different targets.