Structured Interviews: Questions, Scorecards, and Limits

You have just finished a hiring loop for a team lead. Three interviewers liked the candidate and one did not. In the debrief, you discover that each asked different questions, and nobody wrote down what she actually said. The decision goes to whoever argues most confidently.

That meeting raises two questions. What can an interview actually tell you about how someone will do the job? And if you interview, how should you score the answers so that four people are judging the same thing?

This guide takes a position. In Stop interviewing your candidates - seriously, I argued that the best measures of a candidate are samples of their work, discussed with them, and references from a random selection of past coworkers. An interview day can hide a strong candidate who is having an off day, and it can flatter a candidate who is at their once-in-three-years best. That is still the order of preference here: past work and references first.

If you also interview, structure it and score it. A structured interview asks every candidate the same job-related questions and rates each answer against the same predefined scale. Structure is a reason to replace loose, conversational interviews with a better one. It is not a reason to add another round.

“Structured interview” also names a research method for collecting comparable answers from participants, and a clinical tool for reaching a diagnosis. This page is about hiring. For interviews as a research method, see the research methods guide.

What makes an interview structured?

The US Office of Personnel Management (OPM) gives a short working definition in its structured interview guide. All candidates get the same questions in the same order. All candidates are evaluated on a common rating scale. The interviewers agree in advance on what an acceptable answer looks like. In an unstructured interview, candidates may get different questions, and no standard rating scale is required.

Michael Campion, David Palmer and James Campion reviewed nearly 200 articles and books and identified 15 components of structure, some shaping the questions and some shaping the evaluation. Their shorter summary names six as most influential. These include questions based on a job analysis, the same questions for everyone, anchored rating scales, and interviewer training.

Why does this matter? If two candidates get different questions, a difference in impressions may reflect the questions rather than the people. If interviewers rate “overall fit,” a pleasant conversation can raise every judgment at once: the halo effect, or the similar-to-me bias when a shared hobby becomes a judgment about competence. Anchored ratings do not make interviewers immune, but they force each score to rest on something the candidate said.

A structured interview example: hiring a customer support team lead

The rest of this guide follows one hypothetical hiring decision. A software company with 12 support agents needs a new team lead. The hiring manager runs the process with one current lead as a second interviewer. The company, candidates and answers are invented.

Step 1: Start with the job, not the questions

A job analysis identifies the tasks a role involves and the skills and behaviors needed to do them well. It comes first because a question that does not trace back to a real task cannot be scored in any defensible way.

The hiring manager and two current leads list the recurring tasks, then describe recent incidents where a lead handled each one well or badly. Campion and colleagues call this critical-incident approach the most common job-analysis method. The list:

1. Take over escalated customer cases after an agent has tried and failed.
2. Review agents’ tickets each week and give written feedback on quality.
3. Give corrective feedback in person when an agent’s work falls below the standard.
4. Plan shift coverage, including spikes after product releases.
5. Raise recurring customer problems with the product and billing teams.

The incidents matter as much as the list. One lead approved refunds far above their authority to end angry calls. Another let an agent’s slow replies drift for two months. Those incidents become the raw material for questions and for descriptions of weak and strong answers.

Step 2: Decide what the interview needs to cover

Now assign each requirement to its most direct evidence. Past work comes first: material the candidate produced in previous jobs, anonymized where needed and discussed with them so you learn what they did themselves. If they cannot share it, a short set task is the fallback.

The interview gets only what past work cannot show and references can show only secondhand. That means a detailed account of a past case that references can later confirm, and how the candidate reasons through a new situation from this job. Here that is three competencies, two questions each: about 45 minutes, plus unscored time for the candidate’s questions.

No study cited here tests whether a structured interview adds prediction beyond past work and coworker references. The right-hand column is a design judgment, not a tested finding.

The references follow the essay’s method: contact a random selection of former coworkers from the candidate’s last company or department, not names supplied by the candidate. Follow company policy and local law. OPM’s reference checking guidance recommends asking every reference the same job-based questions.

Step 3: Write structured interview questions tied to real tasks

OPM describes two formats. A past-behavior question asks what the candidate actually did. A situational question describes a realistic job situation and asks what they would do. Paul Sackett and colleagues suggest, as a matter of design, that situational questions suit candidates without direct experience and past-behavior questions suit candidates who have done similar work. This interview uses one of each per competency.

The six structured job interview questions for the role:

1. Escalations, past behavior. “Tell me about the last time a customer escalated a case to you after an agent had already tried to resolve it. What did you do, and what happened?”
2. Escalations, situational. “A long-time customer wants a $450 refund outside the 30-day policy. The agent has already said no. The customer says they will post about it publicly. You can approve exceptions up to $200. What do you do in the next hour?”
3. Corrective feedback, past behavior. “Tell me about a time a teammate’s work fell below the standard for several weeks. When did you raise it, what did you say, and what changed?”
4. Corrective feedback, situational. “One of your strongest agents has started closing tickets before customers confirm the problem is fixed. Their numbers look excellent. What do you do?”
5. Cross-team problems, past behavior. “Tell me about a recurring customer problem you got another team to fix. How did you make the case?”
6. Cross-team problems, situational. “Billing errors have doubled your refund requests this month, and the billing team says it is a low priority. What do you do this week?”

Write the permitted follow-ups in advance (for example, asking what the candidate did first). Campion and colleagues explain why: improvised follow-ups are the main way an interviewer can lead a candidate toward a good or bad answer.

The classic weakness question is missing on purpose. It traces to no task, so nothing in the job analysis tells you how to score it.

Step 4: Build the interview scorecard with anchored ratings

An interview scorecard lists each question, its competency, and the rating scale. A behaviorally anchored rating scale describes each score level with examples of job behavior, not with bare adjectives such as good or excellent. OPM’s guide builds its scales this way: people who know the job write example answers for each level and keep the ones they agree on.

This scorecard uses a 1 to 5 scale with written anchors at 1, 3 and 5; a 2 or 4 falls between anchors. The refund question’s anchors, written from the Step 1 incidents:

An evidence column records what the candidate said. A score without recorded evidence cannot be checked by anyone, including you a week later. Before the first candidate, walk both interviewers through the questions, permitted follow-ups and anchors; Campion and colleagues count training among the most influential components.

Step 5: Score independently, then discuss

Each interviewer scores every answer alone, during or immediately after the interview. Keep past-work and reference results from interviewers until they have recorded their scores, or the interview stops being a separate observation. If interviewers talk first, the most senior opinion tends to shape the rest. Independent scores also show how much interviewers agree, a check on reliability.

OPM has each panelist rate alone before any discussion, and its detailed panel procedure has them compare notes after rating all candidates. Campion and colleagues advise against discussing candidates between interviews. This team holds one calibration meeting after the last interview.

On the refund question, the hiring manager scores Candidate A a 4 and the current lead scores a 1. Their notes explain the gap:

• Hiring manager: “Offered $200 now, said she would ask her manager to approve the other $250 and call the customer back by end of day. Called the agent afterward.”
• Current lead: “Gave the refund to stop the public post.”

In discussion, the lead finds their notes stop partway through the answer; the manager’s record the approval request. The lead revises to a 3 and records why: the answer met the 3 anchor but did not check the account history or ask why the customer wanted the refund. The manager keeps a 4, because the answer also set a follow-up time and closed the loop with the agent.

The team’s rule, set before the first interview, averages the two scores for each question after discussion (3.5 here), then averages the two questions for each competency. Any change must cite something the candidate said. Interview scores are one input, alongside the past-work review and a written reference summary. A candidate who charmed the panel but showed weak past work does not get rescued, because the debrief compares recorded evidence.

What the research says about structured interviews

The research measures criterion-related validity: the correlation between selection scores and job performance, most often supervisor ratings. Many studies tested current employees rather than applicants, so the figures stand in for predictive validity. A correlation of .42 does not mean the interview picks the right person 42% of the time. It means higher scorers tend to perform better, with a great deal of overlap.

Frank Schmidt and John Hunter’s 1998 review became the standard summary; OPM’s assessment strategy page still reproduces it. In 2022, Paul Sackett, Charlene Zhang, Christopher Berry and Filip Lievens revisited those meta-analyses. They argued that earlier reviews had substantially overcorrected for range restriction: the narrower spread of scores among already-hired employees.

Most estimates fell, typically by .10 to .20. Two rose: empirically keyed biodata, slightly (.35 to .38), and vocational interests (.10 to .24), after a new meta-analysis measured how well interests fit the job. Structured interviews came out on top.

Both sets of figures are corrected for unreliable performance ratings; the 2022 figures are corrected for range restriction only where the authors judged the data supported it. Sackett’s team set reference checks aside for insufficient information. The work-sample figures describe standardized tests, not reviewing a candidate’s past work, which this research has not separately tested.

The authors noted that the strongest predictors (structured interviews, job knowledge tests, keyed biodata and work-sample tests) are all specific to the job. Measures that sample job behavior tended to beat measures of general traits. The research and the essay agree that a free-form conversation is weak evidence. They part ways on the interview itself: on these estimates, a well-built interview predicts as well as anything on the list.

The spread matters as much as the average

Structured interviews had a wider spread than the other job-specific predictors near the top, with a standard deviation of .19. In a follow-up article, the same authors put the 80% credibility interval at .18 to .66. They wrote that the figure is better read as “.42, plus or minus .24.”

The 2022 paper suggests a risk-averse employer might rank methods by that lower bound. Ranked that way, structured interviews (.18) fall to fifth, behind keyed biodata (.26), contextualized conscientiousness (.25), job knowledge tests (.23) and work-sample tests (.21). The gaps are small. The label covers interviews of very different quality: one built without a job analysis, with vague anchors, and scored in a group discussion may sit near the bottom of that range.

The estimates are still debated

In-Sue Oh, Huy Le and Philip Roth argued in 2023 that Sackett’s team underestimated range restriction, disputing what counts as a plausible level. Sackett and colleagues replied that they support correcting whenever the needed data exist, and not otherwise. Treat the exact numbers as method-dependent; carry the broad pattern, with job-specific methods near the top, into hiring decisions.

Loose interviews can also make decisions worse. In a study by Jason Dana and colleagues, students interviewed a peer and were also given the peer’s prior GPA. Their predictions of the peer’s grades were less accurate (r = .31) than prior GPA alone (r = .65). They felt just as confident when the peer secretly answered at random for the second half of the interview. That was a student setting, not hiring.

Why past work and references still come first

If structured interviews have the highest average, why does this guide put past work and references first? Not because the research ranks them higher. It does not. On the 2022 estimates, work-sample tests average .33 against .42 for structured interviews. Reference checks have no current estimate, and their last one (.26, in 1998) was about half the structured interview’s .51 in the same table.

The essay did not address structured interviews. This guide applies the essay’s reasoning about performance under evaluation to them. Past work shows what the candidate actually produced over months, not on one day, and the discussion shows what they did and why. References from coworkers the candidate did not pick describe how the candidate behaves over time, including toward people they have no reason to impress. A structured interview, however well scored, collects the candidate’s own account, on one day, in a setting they know is an evaluation.

The next point is also this guide’s reasoning, not a position from the essay.

The validity research mostly measures success as a supervisor’s rating of overall performance. Sackett’s team cautions that ratings differ in how much they capture task work, helping others, and harmful behavior at work. And the research has tested neither of the essay’s methods as the essay describes them: past work discussed with the candidate, and coworkers chosen at random. It cannot settle the order. What it does show is that if you interview, structure matters: .42 against .19 for a free-form conversation.

Group differences and the order you choose

Sackett’s team also paired each method with the average Black-White difference in applicant scores, mostly from earlier summaries. Structured interviews showed a small difference (d = .23); work-sample tests showed a much larger one (d = .67). These figures are for set work-sample tests. No comparable estimate exists for reviewing past work.

If a work-sample test screens people out before anything else, expect larger group differences in who advances than if the interview carried more weight. The authors do not call for dropping a method because of a group difference alone; work samples make sense when the job needs trained skill. They recommend combining several methods, which has long been advocated as a way to reduce these differences. A structured scorecard makes your process consistent and reviewable. It does not guarantee that the process is fair or accurate.

What interviews, past work, and references can establish

So what can’t a structured interview tell you? Four limits stay however well you build it.

• It collects accounts, not behavior. A high score on the escalation questions shows a strong account of handling escalations. Whether it measures the skill itself is a construct validity question. Past work and references are how you check it.
• It is still one occasion. A sleepless night or a great morning still affects the answers. Two interviewers and six questions reduce random error; they do not remove the day.
• It can reward fluent speakers. Anchors focus attention on content, but a clear, confident account may still read as a better one. The illusion of validity describes the trap: a coherent story feels predictive whether or not it is.
• Its validity in your company is unknown. If you hire enough people into one role, compare interview scores with later performance and check how closely your interviewers agree. Expect a raw correlation lower than the published figure, which is corrected for unreliable ratings. Small numbers of hires make the check noisy.

When this approach changes

Suppose the same company needs 60 entry-level agents for a seasonal peak, and most applicants have never worked in support. Most have no relevant past work and no former coworkers in a comparable role. Work-sample tests that assume trained skill do not fit, as Sackett and colleagues note, and they observe that traditional in-person structured interviews are generally not viable for high-volume screening.

The order of preference still holds in spirit: look at real work, scored against anchors. A short task needing no prior training, such as replying to a customer email, keeps that principle; check whether its scores relate to later performance before relying on it. If you interview, situational questions suit people without experience, though Sackett’s team notes that research has not shown whether interviews predict as well there. You still match each requirement to its most direct evidence; the evidence available is different.

A second changed condition runs the other way. Sometimes past work and references cover every requirement. Then a scored interview adds cost and another chance for impressions to crowd out evidence. A conversation where the candidate meets the team can still help them decide; do not score it or let it override the evidence.

Try it: a product analyst hire

A hiring manager for a product analyst role proposes four 45-minute interviews, each to “get a feel for the candidate.” The questions include a resume walk-through and the weakness question. What would you change?

Answer. Start with a job analysis. Suppose the analyst writes queries against product data, designs and reads experiments, and explains results to product managers who wanted a different answer.

The first two are best shown by past work: an earlier analysis, anonymized and discussed so you learn which parts were theirs. If they cannot share any, use a set task. A random selection of former coworkers can describe how the candidate handles disagreement over months. That leaves one competency for the interview: explaining an unwelcome result to a stakeholder.

Replace the four unscored conversations with one two-person panel. Use a past-behavior question (“Tell me about a time your analysis contradicted what a manager expected”) and a situational one, each with written anchors. Drop the resume walk-through and the weakness question, because neither traces to a task. Have each interviewer score alone.

The new plan uses less interview time than the original. Structure replaced volume; it did not add to it.

The accompanying scorecard worksheet follows the five steps, with a filled example that includes anchors for all six questions.

The question to ask before you book an interview

A structured interview gives you comparable, scored accounts of how candidates handled real situations and reason through new ones. It cannot show you the work itself or how someone behaves when nobody is evaluating them.

Before your next opening, write the job analysis and the evidence plan before you write a single question. If the plan leaves nothing for an interview to cover, you have your answer.

References

• Campion, M. A., Palmer, D. K., & Campion, J. E. (1997). A review of structure in the selection interview. Personnel Psychology, 50(3), 655–702.
• Campion, M. A., Palmer, D. K., & Campion, J. E. (1998). Structuring employment interviews to improve reliability, validity, and users’ reactions. Current Directions in Psychological Science, 7(3), 77–82.
• Dana, J., Dawes, R., & Peterson, N. (2013). Belief in the unstructured interview: The persistence of an illusion. Judgment and Decision Making, 8(5), 512–520.
• Oh, I.-S., Le, H., & Roth, P. L. (2023). Revisiting Sackett et al.’s (2022) rationale behind their recommendation against correcting for range restriction in concurrent validation studies. Journal of Applied Psychology, 108(8), 1300–1310.
• Sackett, P. R., Berry, C. M., Lievens, F., & Zhang, C. (2023). Correcting for range restriction in meta-analysis: A reply to Oh et al. (2023). Journal of Applied Psychology, 108(8), 1311–1315.
• Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, 107(11), 2040–2068.
• Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2023). Revisiting the design of selection systems in light of new findings regarding the validity of widely used predictors. Industrial and Organizational Psychology, 16(3), 283–300.
• Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262–274.
• U.S. Office of Personnel Management. (2008). Structured interviews: A practical guide.
• U.S. Office of Personnel Management. Designing an assessment strategy and Reference checking. Accessed October 4, 2026.