Guide
How to Evaluate Personality Tests
A personality report says you are analytical, independent and suited to leadership. The first two descriptions sound familiar. But does recognizing yourself in them give you a reason to trust the career advice?
Those are different claims. A questionnaire might describe a recurring tendency reasonably well without showing which career would suit you. To judge the report, you need to connect what it measures with what you want to learn or decide.
That is the practical question behind personality test validity: what evidence supports this interpretation of this score? The steps below show how to answer it using a test’s documentation. You do not need to buy or take a test to begin.
1. Start with the question you want to answer
Be specific. “Understand myself” might mean exploring why you prefer quiet work. “Choose a career” might mean ruling out an occupation because of a score. The second use asks much more of the test: it must tell you something useful about how you would do in that occupation, not just describe a preference.
For the example running through this guide, imagine that you want to understand why some work routines suit you better than others. A questionnaire could give you a description to compare with your experience. You can then ask whether that description helps explain the pattern, before trusting any advice attached to it.
Still choosing a test? The comparison of 14 personality tests starts with the same question and shows what each test can tell you.
2. Find out which test you are taking
Find the test’s name, version, language, number of questions or statements (its items), scoring instructions and documentation date. These details tell you which questionnaire you are evaluating. “Big Five” names a broad model of personality, while different questionnaires measure those traits in different ways. “DISC” also appears on assessments from different publishers. A study helps only if it applies to the questionnaire and scores you would actually receive.
For example, the Eysenck Personality Questionnaire and the Eysenck Personality Inventory have similar names but are different instruments. Revisions can matter too. If a website cites research on a related questionnaire, check which findings apply to the version it offers; the similar name does not answer that question.
Next, identify the score behind the description. A domain is a broad grouping of personality tendencies, such as conscientiousness. A facet describes a narrower part of a domain. In our work-routine example, a report might make a broad statement about conscientiousness and then a more specific one about planning. The test could support the broad description while giving less dependable information about planning alone. Evidence for the whole domain does not automatically carry over to each smaller part.
3. Separate reliability from validity
Reliability concerns consistency and measurement error. One kind, internal consistency, asks how responses to the questions within a scale relate to one another. Another asks how closely scores correspond when people take the test again after a stated interval. These checks answer different questions. Cronbach’s alpha, a numerical estimate of internal consistency, cannot tell you how stable scores are over time.
Construct validity concerns what the score means. A construct is the characteristic researchers are trying to measure. They examine whether the questions cover it, whether the scores relate to other measures and observations as expected, and whether another explanation could account for the results. Consistency helps, but it does not settle those questions.
Imagine a questionnaire whose “planning” questions all ask whether you keep a tidy desk. Your answers might agree with one another because they keep asking about the same thing. Yet those questions do not ask whether you set priorities, anticipate difficulties or revise a plan when circumstances change. The questions could produce a consistent score while leaving important parts of planning unmeasured.
4. Find out who you are being compared with
A percentile describes where a score falls within a comparison group. It does not tell you what percentage of a personality trait you possess. To understand the comparison, find out who supplied the scores, when they were collected and why that group is relevant to the conclusion you want to draw.
Suppose the report compares you with volunteers who completed an online questionnaire. A large sample can tell you a great deal about those volunteers, but the people who chose to take the test may differ from those who did not. More volunteers do not automatically make the group representative of all adults. Your percentile could change with a different comparison group even if your answers stayed the same.
The precision of the score matters too. If a report labels a score of 59 “poorly suited” and a score of 60 “well suited,” it is treating one point as important. This is a hypothetical reporting rule, but it shows the question to ask: could ordinary measurement error account for a difference that small? A sharp change in the report’s wording needs evidence that the scores support that distinction.
5. Match the evidence to the promise
| Claim | Evidence to ask for |
|---|---|
| “Describes a broad personality tendency.” | Research showing that the questions cover the trait, the score is sufficiently consistent, and it relates to other measures or observations in the expected ways. |
| “Predicts a future outcome.” | Research on the promised outcome in the relevant population and time period, including a check on new cases that were not used to develop the prediction. |
| “Improves decisions.” | A comparison showing that people make better decisions when they use the result than when they use a suitable alternative, taking costs and errors into account. |
| “Allows group comparisons.” | Evidence that scores have comparable meanings across the groups being compared. Otherwise, a score difference could partly reflect a difference in how the test measures them. |
The Standards for Educational and Psychological Testing tie validity to the interpretations and uses a test is meant to support. In our example, evidence that a score describes conscientiousness would not be enough to show that following the report’s recommended routine improves your work. The recommendation adds a new claim, so it needs evidence of its own.
A description that feels accurate gives you something to consider. To evaluate the test’s claims, though, look for the study behind them: what did the researchers compare, who took part, and what did they find? A provider’s documentation can explain its scoring and report. Research can examine whether the resulting claims hold up. Who produced that research is relevant, but neither a commercial name nor an academic affiliation replaces reading what was actually tested.
6. Decide how to use the result
Return to the work-routine question. Suppose studies support using the test to describe broad differences in conscientiousness among adults, but say nothing about assigning particular routines. The result can still help you ask useful questions. When do you prepare ahead? When does a plan help you get started? When does detailed planning become another task to avoid?
Look for repeated examples in your life, ask people who know your work, and consider the demands of the task. If planning seems to help you begin, you might try writing down the first task for tomorrow. Then observe whether that change helps. The test suggested something to investigate. Trying the routine tells you what the trait score alone cannot.
If you cannot find the documentation, say exactly what is missing: “I could not verify the evidence behind this score or recommendation.” That is a reason to put less weight on it. It does not prove the test is useless; it means you do not yet have the support you would need to rely on that particular claim.
A short record you can reuse
- My purpose: The question or decision I want help with.
- Exact test and score: Version, language, scale and scoring method.
- What the evidence supports: The specific interpretation, sample and source.
- What I still need to know: How much the score might vary, who it compares me with, or whether it predicts the outcome I care about.
- My next step: What I can reasonably use the result for, whether another source would help more, or whether I need a test at all.
The opening report made two kinds of claim: a description of you and advice about your future. You now have a way to examine them separately. Psychometrics explains how researchers assess measurement, and predictive validity looks more closely at claims about future outcomes.
The evaluation criteria and ownership disclosure explain how this site approaches test evidence.
For help choosing a test, see the comparison of 14 personality tests. If you already have your results, the guide to reading personality test results explains what the numbers mean.