What Is Logistic Regression and How Classification Models Work

Logistic regression is a supervised machine learning model that predicts the probability that something belongs to a class, such as "will this customer churn: yes or no?" Despite the name, it is used for classification, not for predicting continuous numbers. It takes a weighted sum of input features, passes it through the sigmoid function to squeeze the result between 0 and 1, and then applies a threshold to turn that probability into a label.

It matters because it is one of the most widely used classification methods in practice. It is fast, it is easy to explain to stakeholders, and it is often the baseline every more complex model has to beat. This guide explains how logistic regression works, how classification models are evaluated, and when to choose it over something more complex.

What is classification in machine learning?

Classification is the task of assigning an input to one of a fixed set of categories. It is a type of supervised learning, which means the model learns from examples where the right answer is already known. If you need a refresher on that distinction, see our guide to supervised vs unsupervised learning.

• Binary classification: two possible outcomes, such as spam or not spam, fraud or legitimate, churn or stay.

• Multiclass classification: more than two categories, such as routing a support ticket to billing, technical, or sales.

• Multilabel classification: an item can belong to several categories at once, such as tagging an article with multiple topics.

Regression, by contrast, predicts a number, such as next month's revenue. That is the main reason the name "logistic regression" confuses beginners: the model internally fits a regression-style equation, but its job is to classify.

How logistic regression works

Step 1: a weighted sum of features

Like linear regression, the model multiplies each input feature by a learned weight and adds a bias term. For a churn model, features might be months as a customer, number of support tickets, and whether the customer uses a key feature. The result is a single score that can be any number, positive or negative.

Step 2: the sigmoid turns the score into a probability

The score is passed through the sigmoid, also called the logistic function, which maps any number to a value between 0 and 1. Large positive scores land close to 1, large negative scores land close to 0, and a score of 0 lands at exactly 0.5. That output is read as the probability of the positive class. The logistic regression entry explains the underlying math, including how the model is linear in the log-odds.

Step 3: a threshold turns probability into a decision

By default, many tools label anything above 0.5 as the positive class. That default is not sacred. If missing a fraudulent transaction is far more expensive than reviewing a legitimate one, you might lower the threshold to catch more fraud and accept more false alarms.

Step 4: training finds the best weights

During training, the model adjusts its weights to make the predicted probabilities match the real labels as closely as possible, typically by minimizing a loss called log loss or cross-entropy. Regularization is usually added to keep weights from growing too large and to reduce overfitting. Google's Machine Learning Crash Course module on logistic regression walks through this loss and the sigmoid with interactive examples.

How to read the coefficients

One reason teams like logistic regression is that its weights are interpretable. A positive weight means the feature pushes the prediction toward the positive class; a negative weight pushes it away. When features are on comparable scales, larger absolute weights generally mean stronger influence. Exponentiating a coefficient gives an odds ratio, which lets you say something like "each additional support ticket multiplies the odds of churn by a certain factor, holding other features constant."

Two cautions apply. Coefficients describe association inside the model, not causation. And strongly correlated features can make individual weights unstable, even when overall predictions are fine.

How classification models are evaluated

Accuracy alone can be misleading, especially when one class is rare. If only 2% of transactions are fraud, a model that always predicts "legitimate" is 98% accurate and completely useless. Better metrics come from the confusion matrix:

• Precision: of everything the model flagged as positive, how much really was positive.

• Recall: of all real positives, how many the model caught.

• F1 score: a single number that balances precision and recall.

• ROC AUC: how well the model ranks positives above negatives across all thresholds.

Our explainer on the confusion matrix, precision, and recall covers these trade-offs with examples.

A simple workflow in Python

In practice, most people train logistic regression with scikit-learn, whose linear models documentation covers the options in detail. A typical workflow looks like this:

1. Split the data into training and test sets so you evaluate on examples the model has not seen.

2. Clean the data and encode categorical features, for example with one-hot encoding.

3. Scale numeric features so regularization treats them fairly.

4. Fit the model on the training set.

5. Predict probabilities on the test set and choose a threshold based on business costs.

6. Report precision, recall, and AUC, not just accuracy.

When to use logistic regression and when not to

Good fit

• You need a fast, explainable baseline before trying complex models.

• Stakeholders or regulators need to understand why a prediction was made.

• The relationship between features and the outcome is roughly linear in the log-odds.

• You care about well-calibrated probabilities, for example to rank leads by likelihood to convert.

Weaker fit

• The patterns are highly non-linear or depend on complex feature interactions; tree-based models often do better.

• Inputs are raw images, audio, or free text, where deep learning or pretrained models are usually stronger.

FAQ

Why is it called regression if it classifies?

Because it fits a regression-style equation to the log-odds of the outcome. The classification only happens when you apply a threshold to the predicted probability.

Can logistic regression handle more than two classes?

Yes. Multinomial logistic regression, often called softmax regression, extends it to several classes, and most libraries support it directly.

Is logistic regression still relevant with modern AI?

Very much. It remains a standard baseline, a common choice in regulated settings, and a useful way to learn the core ideas behind loss functions and probability outputs that large models also rely on.

Learn to build and evaluate models with live practice

If you want hands-on practice with data, models, and evaluation, guided by practitioners, Coderhouse offers live online courses with one class per week:

• Data Analytics Course: 11 weeks for career switchers, analysts, and data-driven professionals, covering metrics, SQL, Python, visualization, experimentation, and AI-assisted analysis.

• AI Engineering Course: 8 weeks for experienced developers moving into AI engineering, covering LLM applications, retrieval, agents, evaluation, and deployment.