Unstructured Interviews Can Make Hiring Decisions Worse
In one experiment, interviewers given deliberately random answers formed confident impressions. Interviewing made the predictions worse than the file alone.
Most hiring runs on the unstructured interview: a conversation, no fixed questions, an impression formed afterwards. The evidence on it is worse than "it does not add much". It suggests the conversation can make the decision worse than the file alone.
The experiment
Researchers asked participants to predict students’ next-semester grade point average. Some had only biographical information including prior grades; others could also interview the student.
Prior grades alone predicted actual results at a correlation of 0.65. Predictions made after conducting an interview fell to 0.31.
The design included a condition in which the interviewee answered closed questions using a pseudo-random system. The answers carried no information about the person at all. Participants receiving random answers formed impressions as confidently as those receiving truthful ones, and rated the two equally informative.
In a second study, observers watching the interviews could distinguish random from genuine only 66% of the time. In a third, 57% of participants preferred conducting a random interview to conducting none.
Why it happens
The researchers name two mechanisms. Sensemaking: people construct a coherent narrative from whatever they are given, including noise. And dilution: adding non-diagnostic information degrades the predictive value of the diagnostic information already held.
That second mechanism is why the effect is not merely neutral. The interview does not sit alongside the file evidence, it displaces it.
What works instead
The current best meta-analytic estimates, published in 2022 after a major correction to the field’s statistics, put structured interviews at 0.42 and unstructured interviews at 0.19.
Structured means the same questions in the same order, scored against defined criteria. It is less pleasant, and it is roughly twice as predictive.
- Ask every candidate the same questions in the same order
- Score answers against criteria written before you meet anyone
- Keep the file evidence in front of you rather than replacing it with an impression
The caveats
The criterion in the experiment was academic performance, not job performance, and the participants were not professional interviewers. Employment interviews also do things besides predict, including selling the role to the candidate.
But the meta-analytic gap between structured and unstructured is measured on job performance across thousands of cases, and it points the same way. The wider shift in what predicts performance is covered in what actually predicts job performance. For candidates, the practical implications sit in the software engineer guide and in the skills Botswana says it needs.
Frequently asked
Are job interviews actually useful?
Structured ones are, at a predictive validity of about 0.42. Unstructured ones sit at about 0.19, and one experiment found that conducting an interview reduced prediction accuracy relative to using file data alone, because non-diagnostic impressions dilute good information.
What is a structured interview?
The same questions asked of every candidate in the same order, with answers scored against criteria defined before anyone is interviewed. It is roughly twice as predictive of job performance as a free-form conversation.
Why do interviewers trust their gut when the evidence says not to?
Because people construct a coherent story from whatever they are given. In one condition of the experiment, interviewees answered at random, and interviewers formed impressions just as confidently and rated those interviews just as informative as genuine ones.