Linear vs. Logistic Regression: How Do You Choose the Right Model?

Linear and logistic regression are often introduced as two algorithms on a beginner’s machine-learning checklist. That framing obscures the more useful distinction: they answer different kinds of questions. Linear regression estimates a continuous quantity; logistic regression estimates the probability of a categorical outcome.

Choosing between them is therefore not mainly a matter of accuracy scores or personal preference. It begins with a simpler question: what exactly are you trying to predict?

Two different questions

Suppose you want to predict a house’s sale price. The outcome can take many values — £240,000, £317,500, £1.2m — so linear regression is a natural starting point. It models the expected value of a continuous response as a linear function of predictors.

Now suppose you want to predict whether a customer will cancel a subscription. The outcome has two categories: cancelled or retained. You could encode these as 0 and 1, but applying ordinary linear regression directly creates a problem: its predictions are not constrained to lie between 0 and 1. A model might output 1.7 or -0.3, neither of which is a valid probability.

Logistic regression solves this by modelling probability through the logistic function:

$p = \frac{1}{1 + e^{-(\beta_0 + \beta_1x_1 + \cdots + \beta_kx_k)}}$

Here, p is the predicted probability that the outcome equals 1. The linear part,

$\beta_0 + \beta_1x_1 + \cdots + \beta_kx_k$

is transformed so that predictions always fall between 0 and 1.

The core distinction

Question	Linear regression	Logistic regression
What does it predict?	A continuous numerical value	A probability for a categorical outcome
Typical outcome	Price, temperature, duration, demand	Yes/no, churn/no churn, disease/no disease
Output range	Any real number	Between 0 and 1
Standard estimation method	Ordinary least squares	Maximum likelihood
Coefficient meaning	Change in the expected outcome per unit change in a predictor	Change in log-odds per unit change in a predictor
Common misuse	Using it for binary classification without constraining predictions	Treating predicted probabilities as direct causal effects

Linear regression is usually fitted by ordinary least squares, which chooses coefficients that minimise the sum of squared differences between observed and predicted values. Logistic regression is instead commonly estimated by maximum likelihood: it finds the coefficients under which the observed class labels would be most probable.

Why “logistic regression” is misleading

The name is genuinely unhelpful. Logistic regression is called a regression method because it models a probability — a continuous quantity between 0 and 1. In practice, however, it is most often used for classification: assign class 1 if the predicted probability exceeds a threshold, usually 0.5.

That threshold is not part of the model itself. It is a decision rule added afterwards.

If a medical model predicts a 0.08 probability of disease, using 0.5 as the threshold classifies the patient as “no disease”. That may be reasonable when false positives are costly, but dangerous if missing a case is far worse. The appropriate threshold depends on the costs of errors, the base rate of the outcome, and the intended use — not on a universal default.

What the coefficients mean

This is where the two models diverge sharply.

In linear regression, a coefficient has a direct interpretation:

$\text{price} = \beta_0 + \beta_1 \cdot \text{area}$

If \beta_1 = 2500, then, holding other variables fixed, each additional square metre is associated with an expected price increase of £2,500.

In logistic regression, coefficients operate on the log-odds scale:

$\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1x_1 + \cdots + \beta_kx_k$

Exponentiating a coefficient gives an odds ratio:

$\text{odds ratio} = e^{\beta}$

For a one-unit increase in a predictor, the odds of the outcome are multiplied by e^{\beta}, holding other predictors constant.

A coefficient of 0.693 corresponds to an odds ratio of about 2, meaning the odds roughly double. A coefficient of -0.693 corresponds to an odds ratio of about 0.5, meaning the odds roughly halve.

That is not the same as saying the probability doubles. Odds and probability are related but not identical:

$\text{odds} = \frac{p}{1-p}$

A probability rising from 0.10 to 0.20 doubles in probability terms, but its odds rise from 0.111 to 0.25 — an increase of about 125%. This distinction matters enormously when interpreting medical, credit-risk, or churn models.

Linear regression is not automatically wrong for classification

It is tempting to say: “binary outcome? Use logistic regression.” That is usually the right default, but the reason deserves care.

Linear regression can sometimes produce sensible classifications when predictor values are bounded and the fitted line stays within a reasonable range. But it has no mechanism to respect the probability scale. As predictor values become extreme, its output can leave the interval [0, 1], and its residuals can show patterns that undermine reliable inference.

Logistic regression is preferable not because it is universally more accurate, but because its output has the right structure for a probability. It also provides a coherent way to interpret effects through odds ratios and to evaluate calibration — whether a predicted 70% probability actually corresponds to roughly 70% of comparable cases experiencing the outcome.

When linear regression is the right choice

Choose linear regression when the target is genuinely continuous and you care about its expected value.

Common examples include:

Predicting delivery time from distance, traffic, and package weight.
Estimating energy consumption from weather and building characteristics.
Forecasting sales revenue from advertising spend and seasonality.
Modelling the relationship between study hours and examination score.

Linear regression can also include transformed predictors. A model may be linear in its coefficients while containing terms such as x^2, \log(x), or interactions:

$y = \beta_0 + \beta_1x + \beta_2x^2$

This is still linear regression because the unknown quantities being estimated — \beta_0, \beta_1, and \beta_2 — enter linearly. “Linear” refers to the parameters, not necessarily to straight lines in the plotted data.

When logistic regression is the right choice

Choose logistic regression when the response is categorical and you want probabilities or class predictions.

Typical cases include:

Will a loan applicant default?
Will an email be spam?
Will a patient be readmitted within 30 days?
Will a user click an advertisement?
Will a machine component fail before its next inspection?

Logistic regression naturally extends beyond two classes. Multinomial logistic regression handles unordered categories with more than two outcomes, while ordinal logistic regression is designed for ordered categories such as “low”, “medium”, and “high”. The underlying idea remains the same: model category probabilities using a link function rather than fitting a straight line directly to category labels.

A decision rule that avoids most mistakes

A practical sequence looks like this:

Identify the outcome’s true structure. Is it continuous, binary, unordered categorical, ordinal, or a count?
Match the model family to that structure. Continuous outcomes usually point to linear regression; binary outcomes usually point to logistic regression.
Ask what decision the prediction supports. A price estimate, a probability, and a hard classification require different evaluations.
Check assumptions and diagnostics. For linear regression, inspect residuals, non-linearity, influential points, and heteroscedasticity. For logistic regression, examine calibration, separation, multicollinearity, and class balance.
Validate on data the model did not see. Accuracy alone is inadequate, especially with imbalanced classes.

For binary classification, do not rely only on accuracy. A model that predicts “no fraud” for every transaction may achieve 99% accuracy in a dataset where fraud occurs 1% of the time — while identifying no fraud at all. Precision, recall, ROC curves, precision-recall curves, calibration plots, and cost-sensitive thresholds usually reveal more than a single accuracy figure.

The deeper point

Linear and logistic regression are not rivals competing to solve the same problem. They are members of a broader family of generalised linear models: both combine predictors linearly, but they use different link functions to connect that linear combination to the outcome’s scale.

Linear regression says: here is the expected value of a quantity.

Logistic regression says: here is the probability of a category.

Once you phrase the problem that way, the choice often becomes obvious. The harder — and more valuable — question is not “which model should I run?” but “what am I actually trying to estimate, and what would count as evidence that the estimate is trustworthy?”