Learn AI Engineering Rent GPUs By The Hour Docker VPS Hosting Automate 3000+ Apps No Code AI Agents Proxies For Your Agents
Learn AI Engineering Rent GPUs By The Hour
Websites To LLM Data Proxies For Scraping AI Support Chatbot AI Data Analyst AI Agent Workspace Hire AI Builders

Random Forest Classifier: How It Works and How to Train One

Updated November 2026
A random forest trains 100 decision trees, each on a different random sample of your rows and columns, and answers with the label most of them vote for. It is the best default classifier for table data with mixed columns, it needs no scaling and almost no tuning, and it is usually the first model to try when you have a few hundred rows and no strong reason to pick anything else.

How a Random Forest Decides

A single decision tree has a known problem, it is jumpy, it latches onto whatever quirks are in the exact rows it saw and a slightly different set of rows gives a very different tree. So the forest leans into that. It builds 100 trees and makes sure each one sees something different, and then lets the differences cancel out.

It does that with 2 kinds of randomness. First, each tree trains on a random sample of the rows, drawn with repeats, so every tree is missing about a third of the data and has doubles of other rows. Second, at every question a tree asks, it is only allowed to choose from a random handful of the columns, not all of them. That second part matters more than it sounds, without it every tree would open with the same strongest column and they would all end up nearly identical, which defeats the point.

A prediction runs the input down all 100 trees and counts the votes, 71 say cancel and 29 say stay, so the answer is cancel. Each tree alone is overfit and a bit wrong, but they are wrong in different directions, and what they agree on is the real pattern. So basically a forest means "ask 100 people who each saw part of the picture, and go with the majority".

What a Random Forest Is Best At

  • Table data with mixed columns. Counts, amounts, flags, ages, all in one row with no preparation, which is what most business data looks like.
  • Columns that matter in combination. It inherits that from trees, "new account and large order and overnight shipping" is just 3 questions down a branch.
  • Data with junk columns in it. A useless column rarely wins a split, so it mostly gets ignored, where the same column would wreck K Nearest Neighbors.
  • Some wrong labels. A mislabeled row only lands in about two thirds of the trees and gets outvoted by the rows around it.
  • Working well on the defaults. Most classes need a round of tuning to get good, this one is usually within a point or 2 of its best on the first try.

Where It Breaks

  • You can not read it. 100 deep trees is not an explanation, so if someone needs the reason for a specific answer, a forest does not have a short one.
  • Text. With thousands of word columns and each split only seeing a few of them, it is slower and usually less accurate than Logistic Regression on messages.
  • Very small data. Under 50 or so rows there is not enough for 100 different samples to be meaningfully different.
  • Rare labels. If 2% of rows are fraud, most of the random samples contain very few fraud rows and the vote leans to the common label. Balance the dataset or treat it as an anomaly problem with Isolation Forest.
  • Size and speed. The trained model is 100 full trees, so the file is much bigger than a linear model and a prediction walks all of them. Still milliseconds, but it is the heaviest of the simple classes.

Parameters

{"estimators": 100, "treeMaxHeight": 0}
  • estimators is the number of trees. More trees never hurt accuracy, they only make the vote steadier, and they cost training time and file size in a straight line. 100 is where most of the gain has already happened, going to 300 might give a fraction of a point, and dropping to 30 is fine if you want faster training on a small box.
  • treeMaxHeight is the depth limit for each tree, and 0 means no limit, every tree grows until its piles are pure. That would be a mistake for 1 tree, but here it is the normal setting because the voting takes care of the memorizing. Set a limit like 10 or 15 only if the dataset is large and the model file or training time is getting out of hand, or if you have very noisy labels.

How to Train a Random Forest Classifier Step by Step

These steps use ML Prediction Engine, the free open source tool this topic uses for its examples, where the class is named randomForest. The machine learning models guide covers installing it.

1. Create a pipeline with the randomForest class

Create a pipeline, add one model step, set the category to classifier, the class to randomForest, and the input type to numeric. Leave the parameters empty or set {"estimators": 100, "treeMaxHeight": 0}.

2. Gather every column that might matter

With most models you pick columns carefully, here you can be generous. For a churn model that could be logins in the last 30 days, open tickets, months as a customer, plan price, seats used, days since last payment, and a 0 or 1 for annual billing. Send raw values with no scaling. The one rule is that a column has to be something you will actually know at prediction time, a column that only gets filled in after the outcome happens will make testing look great and production useless.

3. Add labeled rows

A few hundred rows is where a forest starts beating simpler classes. Pull them from history, each customer as they looked at a point in time, with what happened next as the label.

curl -X POST http://localhost:8080/api.php/dataset/add \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_KEY" \
  -d '{"pipelineID": 1, "modelIndex": 0, "input": [2, 4, 7, 49, 3, 41, 0], "label": "cancel"}'

For a first load of hundreds of rows, dataset/set takes the whole set as jsonL in one call.

4. Train the model

Click Train or call api.php/train. Building 100 trees on a few thousand rows takes a second or 2, and the reply confirms the row count.

5. Test on held back rows

Keep 50 or more labeled rows out of the dataset and run them through predict. A forest scores close to perfect on the rows it trained on, so that number means nothing, only the held back rows count. Then switch the same step to logisticRegression, retrain, and run the same rows. If the forest is not clearly ahead, the simple model is the better choice, it is smaller, faster, and easier to explain.

6. Predict from your app

curl -X POST http://localhost:8080/api.php/predict \
  -H "Content-Type: application/json" \
  -H "X-API-Key: YOUR_KEY" \
  -d '{"pipelineID": 1, "input": [0, 6, 3, 49, 1, 58, 0]}'

The answer is {"output": "cancel", ...}. Because the trees are built from random samples, retraining on the same rows can flip a prediction on a very close case, that is normal and it tells you the case really was close.

When to Move Off Random Forest

For classification on table data there is often nowhere better to go, so the more common move is sideways. If you need to explain decisions, train a shallow Decision Tree on the same rows for the explanation and keep the forest for the answer. If the data is clean and you want to squeeze out the last points, AdaBoost builds its trees to correct each other instead of independently. And if what you want is a number, the boosted version of this idea is Gradient Boost, which is the usual accuracy leader for numeric prediction.

Key Takeaway

A random forest takes the majority vote of 100 trees that each saw a different random slice of the data. Make it the default for classifying table data once you have a few hundred rows, send raw columns and leave the parameters alone, and judge it only on rows it never trained on.