K-Means Clustering: How It Works and How to Train One
How K-Means Decides
Clustering is the family for when you have no answers to learn from. Nobody labeled the rows, you just have a pile of inputs and want to know what kinds of things are in it. K-Means answers that with a simple loop, and it helps to picture the rows as dots on a map.
First it drops k center points onto the map. Then every dot is assigned to whichever center is closest. Then each center moves to the middle of the dots that were assigned to it. Now that the centers have moved, some dots are closer to a different center than before, so they get reassigned, and the centers move again. After a handful of rounds nothing changes anymore, and those final groups are the clusters. The name is literal, k groups, each defined by the mean of its members.
A prediction for a new input is just "which center is this closest to?", and the answer is that group's number, starting from 0, so with k at 4 you get back 0, 1, 2, or 3. The number carries no meaning, group 2 is not "more" than group 1, and the model has no idea what the groups are about. So basically K-Means means "here are 4 piles of similar things, you tell me what to call them".
What K-Means Is Best At
- Splitting into a set number of groups. Customer segments, feedback themes, product tiers, any time the count is a business choice.
- Short text by topic. Messages, survey answers, and search queries group by the words they share.
- Getting labels you do not have. Cluster first, look at each group, name them, and those names become labels for a classifier like Naive Bayes.
- Speed. It handles tens of thousands of rows in seconds, and a prediction is k distance checks.
- Routing new inputs. Once trained, each new row gets a group instantly, which is something DBSCAN does in a much heavier way.
Where It Breaks
- It always returns k groups. Ask for 4 on data that has 2 real groups and it cuts them in half, ask for 4 on data with no structure at all and it still hands you 4. It never says "there is nothing here".
- Every row must belong somewhere. An input that is nothing like any group still gets assigned to the nearest one. When you need "this fits nowhere", that is DBSCAN.
- Round, similar sized groups only. Because it is all about distance to a center, it assumes each group is a ball. Long stretched groups, or 1 huge group next to a tiny one, get carved up wrong. Gaussian Mixture handles those shapes.
- Every column counts the same. The engine scales numeric columns for this class so dollars do not drown out ratings, which also means a column you did not really care about shapes the groups as much as the ones you did.
- Results change between runs. The starting centers are random, so retraining can produce slightly different groups, and the numbers get shuffled, what was group 2 may now be group 0.
Choosing k
{"k": 4}
k is the only parameter and there is no formula that gives the right value, because the right value depends on what you will do with the groups. If the goal is 3 pricing tiers, k is 3. When you really do not know, the practical method is to try a few and read the results. Train with k at 3, 4, 5, and 6 and look at 10 or 20 rows from each group every time. Too few groups and each one is a mix of clearly different things. Too many and 2 groups look the same and you can not say what separates them. The right k is the largest one where you can still give every group a different name.
How to Train a K-Means Clusterer Step by Step
These steps use ML Prediction Engine, the free open source tool this topic uses for its examples, where the class is named kmeans. The machine learning models guide covers installing it.
1. Create a pipeline with the kmeans class
Create a pipeline, add one model step, set the category to clusterer and the class to kmeans. Choose text for messages or numeric for columns. Leave the parameters empty or set {"k": 4}.
2. Choose the columns to group by
For text there is nothing to do here. For numeric input the engine scales each column for the kmeans class before measuring distance, so yearly spend of 12,000 and 18 orders can go in as raw values. The decision that is yours is which columns to send, because the groups will be built from all of them equally. If you want segments by buying behavior, send spend, order count, and days since the last order, and leave out things like account ID or signup hour, which would split the groups along lines that mean nothing.
3. Add rows with no labels
A clusterer row is only an input. For a first load, send everything in one call:
curl -X POST http://localhost:8080/api.php/dataset/set \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "modelIndex": 0, "filecontent": "{\"input\": \"the app is slow on my phone\"}\n{\"input\": \"please add dark mode\"}\n{\"input\": \"checkout keeps timing out\"}"}'
Aim for at least a few dozen rows per group you expect, so 200 or more for 4 groups.
4. Train the model
Click Train or call api.php/train with the pipelineID and modelIndex. The reply confirms how many rows it grouped.
5. Look at the groups and adjust k
Run a sample of your rows through predict and sort them by the group number that comes back. Read each pile. If you can say "these are speed complaints, these are feature requests, these are payment problems", it is working. If one pile is clearly 2 things, raise k by 1, and if 2 piles are the same thing, lower it. Retrain and read again.
6. Name the groups and predict
curl -X POST http://localhost:8080/api.php/predict \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "input": "pages take forever to load"}'
The answer is a group number, {"output": 2, ...}. Keep a small map in your code from number to name, 2 means "speed". The important part is that this map is only valid for the current trained model, after any retrain the numbers can be reassigned, so run a few known inputs through and rebuild the map each time.
When to Move Off K-Means
If the renumbering after retrains is a problem, the usual fix is to treat clustering as a one time discovery step: cluster, name the groups, save those names as labels on the rows, and train a classifier on them, which gives stable named outputs from then on. If inputs that fit nowhere keep getting forced into groups, use DBSCAN, which answers -1 for noise and finds the number of groups itself. If your groups overlap or are different sizes and shapes, Gaussian Mixture is the closer fit.
K-Means puts every row into 1 of k groups by distance to a moving center. Use it when you know about how many groups you want, send only the columns you want the groups built from, pick k by reading what lands in each group, and remember the group numbers mean nothing and can change after a retrain.