What Is Overfitting and Why Machine Learning Models Fail on New Data

Overfitting happens when a machine learning model learns its training data too well, including the noise and quirks that do not generalize. The result is a model that scores impressively on the examples it has already seen and disappointingly on new ones. It is one of the most common reasons a model that looked great in a notebook fails once it meets real users.

Why it matters now: training and fine-tuning models has become far more accessible, from classic classifiers in a few lines of Python to adapting large language models on a company’s own documents. The easier it is to train, the easier it is to ship a model that memorized instead of learned. Knowing how to spot overfitting is a basic skill for anyone building with AI.

Overfitting vs. underfitting

Every model sits somewhere between two failure modes:

• Underfitting: the model is too simple to capture the real pattern. It performs poorly on both training data and new data. Think of drawing a straight line through data that clearly curves.

• Overfitting: the model is so flexible that it bends around every individual point, including outliers and random noise. Training performance is excellent, but performance on new data drops.

The goal is generalization: a model that captures the underlying pattern well enough to make good predictions on data it has never seen. This tension is often described as the bias-variance trade-off. Simple models have high bias and low variance; very complex models have low bias and high variance. IBM’s explainer on overfitting covers this trade-off with clear examples. If you need the fundamentals first, read our guide to what machine learning is and what it is for.

Why models overfit

Too little data

With a small dataset, a model can effectively memorize every example. There are not enough cases for it to tell which patterns are real and which are coincidences.

Too much model complexity

A deep decision tree, a high-degree polynomial, or a neural network with millions of parameters can fit almost anything. Without constraints, it will fit the noise too.

Training for too long

In iterative training, performance on validation data often improves for a while and then starts to get worse even as training loss keeps falling. That turning point is where the model starts memorizing.

Leaky or unrepresentative data

If information from the test set slips into training, or if training data does not reflect the situations the model will face, scores look better than reality. This is called data leakage, and it produces a false sense of confidence that only appears in production.

How to detect overfitting

The core idea is simple: never judge a model only on the data it learned from.

• Split your data. Keep separate training, validation, and test sets. Train on the first, tune on the second, and report final results on the third, which the model never touched.

• Compare the curves. Plot loss or accuracy for training and validation data over time. A widening gap, with training improving and validation stalling or worsening, is the classic signature of overfitting. Google’s Machine Learning Crash Course module on overfitting shows what these loss curves look like.

• Use cross-validation. Instead of a single split, k-fold cross-validation trains and evaluates the model several times on different slices of the data, which gives a more reliable estimate. The scikit-learn documentation on cross-validation explains the main strategies and how to implement them.

How to prevent overfitting

Get more and better data

More varied, representative examples are often the most effective cure. When real data is scarce, teams sometimes augment it or generate artificial examples, as explained in our article on synthetic data, though synthetic data has its own risks.

Simplify the model

Start with a simple baseline and add complexity only when it clearly helps on validation data. Fewer features, shallower trees, or smaller networks are less able to memorize noise.

Regularize

Regularization adds a penalty for complexity during training. L1 and L2 penalties discourage large weights, dropout randomly switches off parts of a neural network during training, and pruning limits how deep a decision tree can grow.

Stop early

Early stopping monitors validation performance and halts training when it stops improving, keeping the version of the model from before it started memorizing.

Use ensembles

Methods like random forests combine many models trained on different samples, which averages out the individual quirks each one learned.

Overfitting in the age of LLMs

Overfitting is not only a concern for classic models. When teams fine-tune a large language model on a small set of examples, the model can lose general abilities or start repeating phrases from the training set. Evaluation benchmarks can also be “overfit” when models are tuned, directly or indirectly, to score well on a test that leaked into training data. The lesson is the same: evaluate on fresh data that reflects real use.

Overfitting is also different from drift. An overfit model was never good at generalizing; a model affected by drift was fine until the world changed. Our article on data drift explains how to monitor that second problem.

Frequently asked questions

Is 100% training accuracy a good sign?

Usually not. Perfect training accuracy on a realistic problem often means the model memorized the data. What matters is performance on validation and test data.

Can a model overfit and underfit at the same time?

Not on the same pattern, but a model can underfit some parts of the problem while overfitting others, for example memorizing rare categories while missing a general trend.

Does more data always fix overfitting?

More representative data helps a lot, but if it repeats the same biases or leaks test information, it will not solve the problem. Quality and coverage matter as much as volume.

Learn to build models that generalize

Spotting and preventing overfitting is a skill you build by training, evaluating, and debugging real models. To practice it with guidance, explore these courses:

• AI Engineering Course

• Data Analytics Course