What Is Feature Scaling: Normalization vs Standardization in Machine Learning

Feature scaling is the step in a machine learning workflow where numeric input variables are transformed so they share a comparable range or distribution. The two most common techniques are normalization (often min-max scaling, which squeezes values into a range like 0 to 1) and standardization (z-score scaling, which centers values at 0 with a standard deviation of 1).

Why it matters now: real datasets mix features measured in wildly different units—age in years, income in dollars, session time in seconds, clicks per day. Many algorithms treat those raw numbers at face value, so a feature measured in the thousands can drown out one measured in single digits. Scaling is a small preprocessing step that often decides whether a model trains quickly and fairly or struggles for no obvious reason.

Why machine learning models need scaled features

Scaling matters most for algorithms that rely on distances or on gradient-based optimization:

• Distance-based models such as k-nearest neighbors, support vector machines, and k-means clustering compute distances between points. Without scaling, the feature with the largest range dominates the distance.

• Gradient-based models such as linear and logistic regression and neural networks converge faster when features are on similar scales, because the loss surface is less stretched. You can see why in our guide to how gradient descent works.

• Regularized models penalize coefficient size. If features are on different scales, the penalty hits them unevenly.

• Variance-based techniques like principal component analysis (PCA) will mistake large units for large importance unless you scale first.

The scikit-learn example on the importance of feature scaling shows how a k-nearest neighbors model and PCA behave very differently before and after standardization.

Normalization (min-max scaling)

Min-max scaling rescales each value with the formula (x − min) / (max − min), so the smallest value becomes 0 and the largest becomes 1. It keeps the shape of the original distribution and produces a bounded range.

When to use it:

• Features with known, fixed bounds, such as pixel intensities or percentages.

• Algorithms that expect inputs in a bounded range, such as some neural network setups.

Watch out: a single extreme value stretches the range and compresses everything else into a narrow band. If your data has outliers, handle them first or pick a different scaler.

Standardization (z-score scaling)

Standardization subtracts the mean and divides by the standard deviation: (x − mean) / std. The result has a mean of 0 and a standard deviation of 1, but no fixed minimum or maximum.

When to use it:

• As a sensible default for linear models, SVMs, logistic regression, and PCA.

• When features are roughly bell-shaped or when you do not know their natural bounds.

Standardization is less sensitive to a single extreme value than min-max scaling, although large outliers still affect the mean and standard deviation.

Other scalers worth knowing

Robust scaling

Uses the median and interquartile range instead of mean and standard deviation, which makes it a good choice for skewed data with outliers.

Max-abs scaling

Divides each feature by its maximum absolute value, keeping zeros as zeros. Useful for sparse data, such as text features.

Log and power transforms

Not scaling in the strict sense, but often applied before scaling to tame heavily skewed features like revenue or page views. The Google Machine Learning Crash Course on normalization covers linear scaling, z-score scaling, log scaling, and clipping side by side.

The scikit-learn preprocessing guide documents MinMaxScaler, StandardScaler, RobustScaler, and MaxAbsScaler, along with transformers for power and quantile transforms.

When you do not need feature scaling

Tree-based models—decision trees, random forests, and most gradient boosting libraries—split on one feature at a time using thresholds, so the scale of a feature does not change where splits happen. Scaling them is harmless but usually unnecessary. Categorical features that are one-hot encoded are already 0 or 1 and typically do not need scaling either.

The most common mistake: data leakage

Always fit the scaler on the training data only, then use those learned parameters (min and max, or mean and standard deviation) to transform validation and test data. If you scale the whole dataset before splitting, statistics from the test set leak into training and your evaluation looks better than it will in production. Placing the scaler inside a pipeline, together with imputation and the model, keeps this automatic and makes deployment match training exactly.

Quick checklist

1. Check which algorithms you are using: distance-based or gradient-based models need scaling; tree-based models usually do not.

2. Look at distributions and outliers before choosing a scaler.

3. Default to standardization; use min-max for bounded features and robust scaling for outlier-heavy ones.

4. Fit the scaler on training data only, inside a pipeline.

5. Save the fitted scaler with the model so production inputs are transformed the same way.

FAQ

Is normalization the same as standardization?

No. Normalization usually means min-max scaling to a fixed range such as 0 to 1. Standardization means converting values to z-scores with mean 0 and standard deviation 1. People sometimes use “normalization” loosely for both, so check which formula is meant.

Should I scale the target variable?

For classification, no. For regression, scaling the target can help some neural networks train, but you must invert the transformation to report predictions in real units.

Does feature scaling improve model accuracy?

For distance-based and gradient-based models it often improves both accuracy and training speed. For tree-based models it typically makes little difference.

Do LLMs and deep learning models need feature scaling?

Neural networks trained on numeric tabular data benefit from scaled inputs, and deep learning architectures also use internal normalization layers. Text models work on tokens and embeddings rather than raw numeric features, so classic feature scaling applies mainly to structured data pipelines.

Learn to build reliable data and AI pipelines

If you want hands-on practice preparing data and training models that hold up in production, Coderhouse offers live online courses:

• AI Engineering Course: for developers moving into AI engineering, covering LLM applications, retrieval, agents, evaluation, and deployment.

• Data Analytics Course: for analysts and data-driven professionals who want to master metrics, SQL, Python, visualization, and experimentation.