Psychometrics

What is psychometrics?

Psychometrics is the study of psychological measurement: how observations become scores, what those scores mean, and how much confidence to place in them. It includes the theories and statistical methods used to develop and evaluate measures of abilities, personality, attitudes and other characteristics.

A questionnaire can produce a number in seconds. Establishing what that number measures is the harder task. As the Psychometric Society’s introduction explains, the field extends beyond administering tests to the models and methods that connect observations with psychological attributes.

From an answer to a score

Imagine a researcher developing a short questionnaire about planning. This is a hypothetical example, not a validated assessment. One item asks whether you plan tomorrow’s tasks before finishing work. You choose a response from 1, “almost never,” to 5, “almost always.”

The response is an observation. Planning tendency is the proposed construct: the characteristic the researcher wants to measure. The two are not identical. You might make plans because your employer requires them, misunderstand the question, or answer as the person you would like to be.

Suppose four appropriately scored items receive responses of 4, 3, 5 and 2. Their sum is 14 on a possible range of 4–20. The arithmetic is straightforward. Whether those items belong together, deserve equal weight or adequately represent planning requires evidence. A total of 14 is not “70% organized.”

What makes a score interpretable?

• Reliability and error: Would scores remain reasonably consistent under the conditions relevant to their use? A single administration cannot show stability over time. See reliability.
• Validity: Do the questions and the pattern of results support the claimed interpretation? Four nearly identical questions about making lists might miss other aspects of planning. See construct validity.
• Reference group: If a report gives a percentile, who supplied the comparison scores? A rank within a volunteer sample of students is not automatically a rank among all adults.
• Intended use: Describing a tendency, predicting a later outcome and deciding who gets a job are different claims. Evidence for one does not automatically support the others.

The Standards for Educational and Psychological Testing organize validity around proposed interpretations and uses. The relevant question is what the evidence permits you to conclude from this score.

Psychometric theory: different ways to study measurement

Classical test theory separates an observed score into a true-score component and error. “True score” has a technical meaning: the expected score over the specified hypothetical repetitions. It is not a person’s hidden, perfectly known essence.

Item response theory models how the probability of an item response relates to a measured attribute and properties of the item. It can help investigate which items provide information at different points on a scale. Factor analysis examines patterns of relationships among items or measures to evaluate their proposed structure. A model that fits the data is evidence to examine, not proof that the named attribute is the only possible explanation.

Using psychometrics in applied behavioral science

Before using a score to evaluate a project, define the outcome. If you want people to complete a task, completion records answer a different question from a questionnaire about confidence. Both can be useful, but improvement in one cannot stand in for improvement in the other.

For personal reflection, a well-supported measure can help you notice individual differences that matter when choosing activities. Compare the result with your interests, skills and repeated experience. The score helps frame a question; it does not choose your life for you.