Authors: | Kalušev, Vladimir Popović, Branislav Janev, Marko Brkljač, Branko Ralević, Nebojša |
Affiliations: | Mechanics Mathematical Institute of the Serbian Academy of Sciences and Arts |
Title: | Measure of Similarity between GMMs Based on Autoencoder-Generated Gaussian Component Representations | Journal: | Axioms | Volume: | 12 | Issue: | 6 | First page: | 535 | Issue Date: | 2023 | Rank: | ~M21 | ISSN: | 2075-1680 | DOI: | 10.3390/axioms12060535 | Abstract: | A novel similarity measure between Gaussian mixture models (GMMs), based on similarities between the low-dimensional representations of individual GMM components and obtained using deep autoencoder architectures, is proposed in this paper. Two different approaches built upon these architectures are explored and utilized to obtain low-dimensional representations of Gaussian components in GMMs. The first approach relies on a classical autoencoder, utilizing the Euclidean norm cost function. Vectorized upper-diagonal symmetric positive definite (SPD) matrices corresponding to Gaussian components in particular GMMs are used as inputs to the autoencoder. Low-dimensional Euclidean vectors obtained from the autoencoder’s middle layer are then used to calculate distances among the original GMMs. The second approach relies on a deep convolutional neural network (CNN) autoencoder, using SPD representatives to generate embeddings corresponding to multivariate GMM components given as inputs. As the autoencoder training cost function, the Frobenious norm between the input and output layers of such network is used and combined with regularizer terms in the form of various pieces of information, as well as the Riemannian manifold-based distances between SPD representatives corresponding to the computed autoencoder feature maps. This is performed assuming that the underlying probability density functions (PDFs) of feature-map observations are multivariate Gaussians. By employing the proposed method, a significantly better trade-off between the recognition accuracy and the computational complexity is achieved when compared with other measures calculating distances among the SPD representatives of the original Gaussian components. The proposed method is much more efficient in machine learning tasks employing GMMs and operating on large datasets that require a large overall number of Gaussian components. |
Keywords: | autoencoders | classification | Gaussian mixture models | KL divergence | machine learning | Publisher: | MDPI | Project: | This research was supported by the Serbian Ministry of Education, Science and Technological Development through project no. 45103-68 / 2020-14 / 200156: “Innovative Scientific and Artistic Research from the Faculty of Technical Sciences Activity Domain" Internal project of the Faculty of Technical Sciences in 2023: “Research aimed at improving the teaching process and development of scientific and professional areas of the Department of Power, Electronic and Telecommunication engineering“ |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
MJanev.pdf | 612.34 kB | Adobe PDF | View/Open |
This item is licensed under a Creative Commons License