Effect size Definition and Examples | The Behavioral Scientist

What is an effect size?

An effect size describes how large a difference or association is. It helps answer the practical question: how much changed?

How it works

Common measures include Cohen’s d, correlations, odds ratios, and R-squared. Cohen’s d expresses a difference between group means in standard-deviation units. The familiar d = 0.2, 0.5, and 0.8 labels for small, medium, and large effects are rough guides, not verdicts on usefulness.

Two overlapping normal curves have means half a standard deviation apart: Cohen’s d equals 0.5. — A mathematical illustration, not study data: both groups have a standard deviation of 1, and their means differ by 0.5. The overlap shows why an average difference does not separate everyone into two distinct groups.

Imagine an educational intervention with p < 0.001 and d = 0.05. The groups differ by just one-twentieth of a standard deviation under the chosen definition of d. The impressive-looking p-value does not make that difference large. As the American Statistical Association explains, a p-value measures neither effect size nor practical importance.

Interpreting effect-size symbols and direction

There is no single effect-size symbol or range. A standardized mean difference such as d uses standard-deviation units; a correlation r ranges from −1 to 1. A negative d indicates which group has the lower mean under the chosen subtraction order, not that the result is unimportant. In regression, R-squared describes the proportion of outcome variation accounted for by the fitted model.

Why it matters

A statistically significant result can offer very little practical benefit. Even a small effect can matter at scale, but cost, uncertainty, the outcome, and the alternatives determine whether it is worth pursuing. Report the estimate and its uncertainty, not just whether it passed a significance threshold.

Lakens’s primer explains how the measure and study design affect comparisons. For the classic treatment of effect-size benchmarks, see Cohen’s Statistical Power Analysis.

From an effect size to actual attendance

Here is a hypothetical reminder experiment. Randomly assign 10,000 eligible people to two equal groups before an event. In the usual-invitation group, 1,000 of 5,000 attend; in the reminder group, 1,100 of 5,000 attend. Measure attendance at that event in the same way for both groups, with no missing outcomes.

Absolute difference: 22% − 20% = 2 percentage points, or two additional attendees per 100 invitations.
Relative increase: (22% − 20%) ÷ 20% = 10%. The baseline is the usual-invitation attendance rate.
Risk ratio: 22% ÷ 20% = 1.10. Here “risk” simply means the probability of the counted event: attendance.

These are three descriptions of the same observed comparison. A 10% relative increase does not mean ten extra attendees per 100 invitations. Under an independent two-proportion normal approximation, the 95% confidence interval for the absolute difference is about 0.4 to 3.6 percentage points. That calculation addresses sampling uncertainty under the model; it does not establish that next year's attendees will respond identically. The confidence-interval entry works through the calculation.

When a standardized difference helps

For a continuous outcome, suppose a separate hypothetical teaching program produces mean scores of 72 and 70, with a pooled within-group standard deviation of 10 points. Cohen's d is (72 − 70) ÷ 10 = 0.20. This expresses the two-point difference in standard-deviation units. It is not a 20% improvement, nor does it mean 20% of students benefited. Different definitions of d use different standard deviations, so identify the formula and design when comparing studies.

A common scale can help compare related measures, but standardization does not make unlike outcomes equivalent. A change in a satisfaction rating and a change in actual attendance still answer different questions. To decide whether the attendance gain justifies the work, continue to practical significance.