Test–retest reliability

What is test–retest reliability?

Test–retest reliability concerns the consistency of scores when the same people complete a measure on separate occasions under specified conditions. It is most informative when the characteristic is expected to remain reasonably stable over the interval being studied.

A change in score can reflect measurement error, actual change or a change in the circumstances of testing. Two administrations do not automatically separate those explanations.

Why the interval matters

Retesting too soon can allow memory of the questions or answers to affect the result. Waiting longer reduces that concern but gives more opportunity for real change. A sensible interval depends on the attribute and intended use; there is no universal number of days.

Researchers should record the interval, instructions, setting and relevant intervening events. A questionnaire about today’s mood is expected to change more quickly than a measure of a broad personality tendency. Low stability over time need not mean a mood measure is failing to capture mood.

Correlation is not the same as agreement

Consider a deliberately small hypothetical example. Three people score 10, 20 and 30 on the first occasion, then 15, 25 and 35 on the second. Their order is identical and their Pearson correlation is 1. Yet every score increased by five points.

If the purpose is to preserve relative ordering, that consistency is relevant. If the purpose is to detect a five-point change in an individual, the systematic shift is a serious concern. An appropriate analysis must examine the kind of agreement the intended use requires. Three people are used here to make the arithmetic visible, not as a sufficient validation sample.

Intraclass correlation coefficients are often used in reliability studies, but different forms answer different questions. Koo and Li’s guidance emphasizes selecting and reporting the form that fits the design. An unexplained “ICC” is not enough information.

Does a changed score mean the person changed?

Not necessarily. Suppose someone takes a planning questionnaire before a demanding deadline and again during a quiet week. Their responses may reflect genuine short-term conditions rather than a lasting change in their usual planning tendency. Conversely, assuming all score changes are error could hide meaningful development.

Inspect score uncertainty, repeated patterns and relevant external observations. A measure suitable for comparing group averages may be too imprecise to interpret a small individual change. The testing standards discuss reliability and precision in relation to the proposed use.

For the broader comparison with validity and other forms of consistency, see reliability. For the separate question of durable trait development, see can personality change?