Gaussian Mixture Model: How It Works and How to Train One
How a Gaussian Mixture Decides
K-Means describes each group with 1 thing, a center point, and assigns every row to the closest center. That quietly assumes every group is a round ball of about the same size. A Gaussian mixture describes each group with 3 things: where its center is, how spread out it is, and which directions it is stretched in. So one group can be a small tight clump and the one next to it a long wide smear, and the model knows the difference.
Here is why that matters. Say you are grouping customers by orders per year and average order size. You have a small, tight group of wholesale buyers and a big loose group of casual shoppers, and a customer sits between the 2 centers, a bit closer to the wholesale one. K-Means says wholesale, because it is closer. The mixture model looks at the spread too, the wholesale group is so tight that this customer is far outside its normal range, while the casual group is so wide that this customer is well within it. So it says casual, and that is the right call. Basically K-Means asks "which center is nearest?" and a mixture asks "which group would be least surprised to have this row?".
Training is a back and forth. The model starts with a rough guess at the k clouds, works out for every row how likely it is to belong to each one, then reshapes each cloud to fit the rows it mostly owns, and repeats until the clouds stop moving. Inside, every row has a share in every group, maybe 70% one and 30% another, which is the "soft" part. What you get back from a prediction is the single most likely group as a number starting from 0.
What a Gaussian Mixture Is Best At
- Groups of different sizes. A few heavy users next to a large crowd of light ones, which K-Means tends to split badly.
- Stretched groups. When 2 columns move together inside a group, like visits and spend, the cloud tilts and stretches to follow.
- Overlap. Where groups blend at the edges, it decides by likelihood instead of raw distance.
- Customer and behavior segments. Numeric profiles with a handful of columns are the natural fit.
- Measurements that really are bell shaped. Readings, sizes, timings, anything that clusters around typical values.
Where It Breaks
- It needs more rows than K-Means. Every group has to learn a center plus a full shape, and with 6 columns that is 27 numbers per group. With too few rows a cloud can collapse onto a couple of points and the results turn strange.
- Many columns. The shape description grows with the square of the column count, so past 10 or so columns it needs a lot of data and gets slow.
- Text. A text input becomes thousands of word columns, which is far too many for this model. Use
kmeansordbscanfor text and keep this one for numeric input. - Groups that are not cloud shaped. Rings, crescents, and chains do not fit a bell, DBSCAN handles those.
- It always returns k groups, and every row gets one. Like K-Means it has no way to say "this fits nowhere", and the numbering can change between training runs.
Parameters
{"k": 4}
k is the number of groups, and it is the only parameter. Choose it the same way as for K-Means, by training with a few values and reading what lands in each group, with 1 extra thing to watch for. Because a cloud can be any size, an extra group will sometimes be used to soak up a few scattered odd rows instead of finding a real segment. If raising k from 4 to 5 produces a fifth group with only a handful of rows that have nothing in common, the real answer was 4.
How to Train a Gaussian Mixture Model Step by Step
These steps use ML Prediction Engine, the free open source tool this topic uses for its examples, where the class is named gaussianMixture. The machine learning models guide covers installing it.
1. Create a pipeline with the gaussianMixture class
Create a pipeline, add one model step, set the category to clusterer, the class to gaussianMixture, and the input type to numeric. Leave the parameters empty or set {"k": 4}.
2. Choose a handful of numeric columns
Pick 2 to 8 columns that describe the behavior you want to group, for customers that might be orders per year, average order size, days since last order, and share of orders bought on discount. The engine does not scale columns for this class, and mostly it does not need to, since the model learns a separate spread for every column. It does start from a distance based first guess though, so bring columns into roughly comparable ranges when one is thousands of times larger than another.
3. Add plenty of unlabeled rows
Rows are inputs only. Plan for at least 50 per group you expect, 200 or more for 4 groups, and more as you add columns.
curl -X POST http://localhost:8080/api.php/dataset/add \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "modelIndex": 0, "input": [14, 85, 21, 0.1]}'
That customer placed 14 orders, averaging 85 dollars, last ordered 21 days ago, with 10% bought on discount.
4. Train the model
Click Train or call api.php/train with the pipelineID and modelIndex. It takes a little longer than K-Means because it is fitting shapes as well as centers, still about a second for a few thousand rows.
5. Read the groups and adjust k
Run a sample of rows through predict and sort them by group. For each group work out the typical values, "about 40 orders a year at 300 dollars, rarely on discount", and write that as 1 sentence. If 2 groups get the same sentence, lower k. If a group needs "and also" in its sentence because it is clearly 2 kinds of customer, raise k. If a group has just a few unrelated rows in it, lower k.
6. Name the groups and predict
curl -X POST http://localhost:8080/api.php/predict \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "input": [3, 40, 160, 0.8]}'
The answer is a group number, {"output": 1, ...}, which your code maps to a name like "lapsed bargain shopper". As with any clusterer the numbers are only valid for the current trained model, so after a retrain send a few known customers through and rebuild the map.
Gaussian Mixture or K-Means?
Start with K-Means when the data is text, when you have many columns, or when you have fewer than about 50 rows per group, it is simpler and needs less. Move to the mixture model when the input is a handful of numeric columns and you can see K-Means getting the borders wrong, a big group being split in 2 while a small one gets swallowed is the usual sign. And when you do not know how many groups there are, or need outsiders flagged, that is DBSCAN.
A Gaussian mixture gives each group its own center, size, and shape, so it handles overlapping and uneven groups that K-Means cuts wrong. Use it on a handful of numeric columns with at least 50 rows per group, pick k by describing each group in a sentence, and stay with kmeans for text.