Overview

gmcluster is a python-based software package that automatically estimates the parameters of a Gaussian mixture model from sample data. This process is essentially similar to conventional clustering except that it allows cluster parameters to be accurately estimated even when the clusters overlap substantially. The resulting mixture model is useful for a variety of applications including texture and multispectral image segmentation.

The estimate_gm_params( ) function applies the expectation-maximization (EM) algorithm together with an agglomerative clustering strategy to estimate the number of clusters that best fit the data. The estimation is based on the Rissenen order identification criteria known as minimum description length (MDL). This is equivalent to maximum-likelihood (ML) estimation when the number of clusters is fixed, but in addition, it allows the number of clusters to be accurately estimated. The function includes an option to decorrelate coordinates to better condition the problem.

The package also includes three additional functions which make estimate_gm_params( ) more useful. The function split_classes( ) separates the parameters for each of the classes/clusters in the estimated multi-cluster mixture model and returns them as multiple sets of single-cluster parameters. The function compute_class_likelihood( ) can be used to calculate the log likelihood of a sample given the model parameters generated by estimate_gm_params( ) and split_classes( ). This can then be used to perform tasks such as maximum likelihood classification. The function generate_gm_samples( ) can be used to simulate samples for a Gaussian mixture model of any order.

How does the estimate_gm_params( ) function work?

The algorithm starts with initializing a set of cluster parameters for a user-selected number of clusters. The cluster means are generated by selecting the appropriate number of samples from the training data, and the cluster covariances are all set equal to the covariance of the complete data set. After this initialization, the algorithm performs multiple iterations of EM clustering until converged cluster parameters are estimated for the given number of clusters. For each iteration of EM clustering, the algorithm performs an E-step and an M-step. Then the algorithm enters a loop in which clusters are combined (or eliminated when empty) to reduce the order. For every reduced order, we again perform EM clustering to get the cluster parameters. Based on Rissanen criterion, the algorithm finally makes the decision on optimal order and associated cluster parameters (unless the optimal order is fixed by the user). The following figure illustrates the basic operations performed in the estimate_gm_params( ) function.

estimate_gm_params( ) function operation flowchart