Meta analysis

What is a meta-analysis?

A meta-analysis combines numerical results from separate studies to estimate an effect or relationship. It often uses a weighted average, giving more influence to more precise estimates. The result can be more informative than any one study, but it inherits important problems in the evidence being combined. Cochrane's methods guidance begins with the question of whether pooling is appropriate at all.

A worked example: combining two estimates

Suppose two independent, hypothetical experiments compare the same reminder with the same usual invitation in comparable settings. Both measure attendance at the next event. Study A estimates a two-percentage-point increase with a standard error of one percentage point. Study B estimates a four-percentage-point increase with a standard error of two percentage points.

For an inverse-variance analysis, the weights are 1 ÷ 1² = 1 for A and 1 ÷ 2² = 0.25 for B, using percentage-point units consistently. The combined estimate is:

(1 × 2 + 0.25 × 4) ÷ (1 + 0.25) = 2.4 percentage points.

A gets 80% of the total weight and B gets 20%. The result is closer to A because A is more precise, not because its finding is more exciting. Under this common-effect model, treating the standard errors as known, the pooled standard error is √(1 ÷ 1.25) ≈ 0.894 percentage points. A normal-approximation 95% interval is about 0.65 to 4.15 percentage points. These invented inputs demonstrate the calculation; they are not evidence that reminders have this effect.

One average can hide different effects

A common-effect analysis assumes a shared underlying effect. A random-effects analysis allows underlying effects to vary across studies and estimates their mean under a model for that variation. Heterogeneity can arise from different participants, delivery, comparisons or measurements. Sampling error also makes observed estimates differ, even when the underlying effect is the same.

Imagine that Study A measures next-day attendance among employees, while B measures attendance six months later among volunteers. A tidy pooled number may conceal the distinction the decision needs. Describe those differences before deciding whether to combine the studies. A random-effects model does not make incompatible outcomes interchangeable, and its confidence interval for a mean effect is not a prediction interval for a new setting.

Read the evidence behind the diamond

A forest plot displays individual estimates and intervals; a diamond commonly represents the pooled estimate and its interval. Inspect the studies, not only that diamond. Ten publications can report overlapping participants rather than ten independent experiments. Counting each as independent would overstate how much evidence exists.

A systematic review explains how the studies were found and assessed. A meta-analysis is the statistical synthesis it may contain. If disappointing findings are missing, a pooled estimate can be precise and still misleading. Check publication bias, outcome selection and sensitivity to defensible analytical choices. Finally, translate the effect into the actual outcome and cost before deciding what to do.