A model that combines Gaussian mixture models with Neural Networks
The project consists in studying a new algorithm, along the lines of Parzen Neural Networks, brings together the statistical approach and machine learning approaches for training an ANN from an unlabeled data sample of patterns randomly drawn from an underlying probability density function (PDF).
The algorithm leverages both the generalization capabilities of ANNs and the generality of the maximum-likelihood estimates of the parameters of Gaussian mixture models. Therefore, the proposed machine is termed Gaussian-mixture Neural Network (GNN). The best selling points of the GNN lie in its simplicity and effectiveness.
(This project is the continuation of the work carried out for the A.I. exam., Siena, 2024)
This project was submitted to ANNPR 2024.
data
folder:
Contains the save files of the training and test data. This files are automatically generated and saved when run the algorithm.
script
folder:
It contains all the code for running the project.
model
folder contains scripts for each model used in the paper (for example in moddel/nn_model.py
there is all the code for building the Neural Network used in the GNN and PNN models)
utils/data_manager.py
: the data manager for the creation of the PDF samples for the training and test setoptuna
folder: contains optuna scripts for searching the best hyperparameters for each model
main_MLP.py
: the main function for the training and evaluetion of the GNN and PNN modelsmain_STATISTIC.py
: the main function for the training and evaluetion of Statistic Models (Kn-NN, GMM, Parzen Window)
test_experiment.py
: script for change number of components and number of neurons in GNN models.miscellaneous
folder: some random code created at the beginning of the projectoptuna_database
folder: contains all the database file created with optuna with all the trials for all the experiments used to find the best hyperparametersproject
folder: contains old result created at the beginning of the project for A.I. Exams (with @AparnaPindali)In order to run all the script you need to run the following:
Clone the project:
git clone https://github.com/Duccioo/Gaussian-Mixture-Neural-Network.git
install the dependencies:
pip install -r requirements.txt
install Pytorch on your machine: https://pytorch.org/get-started/locally/
On the repo folder:
python script/main_MLP.py
You can also specify some additional arguments:
--pdf <str s>
: select the PDF type (default: exponential PDF with rate = 0.6). Other possibility is multivariate
to select a multivariate PDF, otherwise you can specify with your prefer parameters.--model <str s>
: select the model type, possibilities = [‘GNN’, ‘PNN’ ] (default: GNN)
--jobs <int n>
: specify the number of threads used to create the target samples for the MLP (default: 2)--samples <int n>
: number of samples used for training (default: 100)--show
: toggle used to show the predicted PDF (default: not show the graph)--gpu
: toggle to use, if possible, GPU acceleration python script/main_MLP.py --model=PNN --samples=150 --pdf=multivariate --show
This command run the training on the PNN (Parzen Neural Network) on the Multivariate PDF with 150 samples used for the training, and at the end of training show the predicted PDF on the screen