Deep Neural Network Classifier: How It Works and How to Train One
How a Neural Network Decides
Take logistic regression, which multiplies each input by a weight, adds them up, and turns the total into an answer. A neural network is that same move done many times in layers. The default here has a first layer of 128 units, each one its own weighted sum of all the inputs, followed by a simple rule that zeroes out negative totals. Those 128 results become the inputs to a second layer of 64 units that do the same thing, and those 64 feed a final layer with 1 output per label.
What that buys you is combinations. One unit in the first layer might learn to switch on when "refund" and "twice" appear together, another when the message is short and has a question mark. The second layer combines those into bigger ideas, and the model works all of it out on its own, nobody tells it which combinations to look for. So basically logistic regression means "each word gets 1 opinion" and a neural network means "words can form groups, and the groups get opinions too".
Training is repeated correction. The network starts with random weights and gets nearly everything wrong, then for each batch of rows it measures how far off it was and nudges every weight a tiny bit in the direction that would have been less wrong. One full pass through the data is an epoch, and it does up to 300 of them. That is why training takes seconds to minutes instead of the instant you get from the simpler classes.
And to be clear about the word "deep", this is 2 hidden layers and maybe a million weights. It is the same family as the large language models everyone talks about, but those have billions of weights and a very different structure, this one is a small model that learns 1 classification job from your rows.
What a Neural Network Is Best At
- Text where phrasing matters. When the meaning comes from words together and not single keywords, and you have thousands of examples.
- Patterns that are not straight lines. Both low and high values meaning the same thing, effects that only appear in combination, curves and pockets all at once.
- Large datasets. Most simple classes level off after a few thousand rows, a network keeps finding more as the data grows.
- Many inputs that are all a little bit useful. Sensor readings, long feature lists, anything where no single column decides it.
- Fast answers after slow training. The training cost is paid once, a prediction is a few matrix multiplications and comes back in milliseconds.
Where It Breaks
- Small data. With 128 units looking at up to 10,000 word columns, the first layer alone has over a million weights, and 200 rows can not pin those down. It will memorize the rows and fail on new ones. Under about 1,000 rows, Naive Bayes or logistic regression will usually beat it.
- No explanation. There is no path to read and no weight per word that means anything by itself. If someone asks why, the honest answer is "the numbers came out that way".
- Table data. On ordinary business columns, Random Forest is usually as accurate or better with no tuning at all.
- It is not the same twice. The random starting weights mean 2 training runs on the same rows give slightly different models, and a close case can flip between them.
- Tuning. 4 parameters that affect each other, and a bad combination looks the same as a bad dataset from the outside.
Parameters
{"layers": [128, 64], "epochs": 300, "learningRate": 0.001, "alpha": 0.0001}
layersis the size of each hidden layer in order.[128, 64]is 2 layers,[32]is 1 small layer,[256, 128, 64]is 3. Bigger means more capacity to learn and more capacity to memorize, so size it to your data, a few hundred rows wants[32]or[64], and the default suits a few thousand.epochsis the maximum number of passes through the data. Training stops early on its own once it stops improving, so 300 is a ceiling, raise it only if the model still looks unfinished.learningRateis the size of each nudge. 0.001 is a safe default. Too high, like 0.1, and it overshoots and never settles, too low, like 0.00001, and 300 epochs is not enough to get anywhere.alphais a penalty on large weights, and it is your brake on memorizing. Raising it to 0.001 or 0.01 forces a simpler model, which is the first thing to try when training accuracy is high and held back accuracy is not.
How to Train a Deep Neural Network Classifier Step by Step
These steps use ML Prediction Engine, the free open source tool this topic uses for its examples, where the class is named dnn. The machine learning models guide covers installing it.
1. Get a baseline score first
Do not start with the network. Create the pipeline with the class set to logisticRegression for text or randomForest for numeric columns, train it, and run 50 or more held back inputs. Write the score down. Without that number you have no way to know if the network is good or just complicated.
2. Switch the step to the dnn class
On the same pipeline step, change the class to dnn and set the parameters:
{"layers": [128, 64], "epochs": 300, "learningRate": 0.001, "alpha": 0.0001}
The dataset stays with the step, so every row you loaded for the baseline is still there.
3. Make sure there are enough rows
Aim for at least 1,000 rows in total and a few hundred per label. If you are short, this is the wrong class for now, stay on the baseline and keep collecting. Rows go in the same way as for any classifier:
curl -X POST http://localhost:8080/api.php/dataset/add \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "modelIndex": 0, "input": "the update wiped my saved settings again", "label": "technical"}'
4. Train the model
Click Train or call api.php/train. The engine scales the inputs for this class automatically, which neural networks need, so numeric columns can go in raw. Expect seconds on a thousand rows and a minute or more on tens of thousands of text rows, this is the one class where you wait.
5. Test and size the layers
Run the same held back inputs as the baseline. If the network scores lower, it is almost always memorizing, so make it smaller with "layers": [32] and raise alpha to 0.001, then retrain and test again. If it scores about the same as the baseline, keep the baseline, it is simpler in every way. Only keep the network if it wins by a margin you care about.
6. Predict from your app
curl -X POST http://localhost:8080/api.php/predict \
-H "Content-Type: application/json" \
-H "X-API-Key: YOUR_KEY" \
-d '{"pipelineID": 1, "input": "it logged me out and now my password does nothing"}'
The answer is {"output": "technical", ...}. Because each retrain starts from new random weights, retrain on a schedule, daily or weekly, instead of after every new row.
When a Neural Network Is the Wrong Tool
More often than people expect. If you have under 1,000 rows, need to explain decisions, or are working with ordinary table columns, one of the simpler classes is the better answer and not a compromise. The network earns its place when you have a lot of rows, the simpler classes have clearly levelled off, and the misses are cases where the meaning depends on how things combine. For predicting a number with the same kind of model, see the MLP Regressor.
A neural network classifier stacks layers of weighted sums so it can learn combinations the simple classes can not. Get a baseline score from logisticRegression or randomForest first, use dnn only with 1,000 rows or more, and when it memorizes, shrink the layers and raise alpha.