The Candidate Nobody Hired

I.The First Known Thing — the people who were turned away

personnel selection psychology · criterion validity

An organisation wants to know whether its hiring test works. The question has an obvious form: administer the test, hire, wait, rate performance, correlate. The coefficient that comes back is called criterion validity, and it is the central quantity of a discipline that has been assembling it for a century — industrial and organisational psychology, which builds tests of cognitive ability, structured interviews, work samples and personality inventories, and then argues about how well each of them predicts what a person will do in the job. The literature is large, meta-analytic, and unusually self-critical, because early in its history it discovered that the obvious form of the question is broken.

the correlation is computed on survivors of the correlation's own instrument

It is broken because performance ratings exist only for people who were hired, and the test decided who was hired. Whatever spread of test scores existed in the applicant pool, the hired group has less of it: the low scorers are missing, removed by the very instrument now being graded. A correlation computed inside that narrowed band is smaller than the one that would be found in the full pool, and it is smaller by an amount that depends on how hard the selection was rather than on anything about the test. Karl Pearson set out the algebra in 1903 and Robert Thorndike codified its use for selection in 1949, distinguishing the cases that matter: restriction directly on the predictor being validated, restriction on the predictor with a third variable also entering, and — the one that turns out to be commonest and hardest — indirect restriction, where selection acted on something else that was itself correlated with the test. Lawley's multivariate extension generalises the corrections. Taylor and Russell had already shown, in 1939, how the practical yield of a test depends jointly on its validity, on the base rate of success and on the selection ratio, so that the same test is worth very different amounts at different degrees of choosiness.

what the corrections require, and what they cost

Two features of that body of work matter here, and they pull in opposite directions. The first is that the uncorrected coefficient is not a conservative estimate of the true one — it is an estimate of a different quantity altogether, the residual association among those the rule admitted, and it moves with the rule. Push the selection ratio down, admit only the top few per cent, and the observed validity tends toward zero while the test's usefulness is at its maximum. The discipline's standard language for this, validity generalisation in the tradition of Hunter and Schmidt, treats the observed coefficients in the literature as artefact-laden readings of a stabler underlying parameter and corrects them upward as a matter of routine. The second feature is that every such correction is an inference about people who were never observed, and rests on a model of how selection happened that is almost never documented at the time. In 2022 Sackett and colleagues went back through the meta-analytic estimates and argued that the corrections had been applied where their assumptions did not hold, systematically overcorrecting; their revision cut the headline operational validity of cognitive ability tests by roughly a third and reordered the standing of the predictors against each other. The dispute is live. What is not disputed is the structure underneath it: without either a documented selection function or a stream of people admitted for reasons unrelated to the test, the quantity everyone wants is not identified at all, and the number in hand is a reading of the organisation's own choosiness.

II.The Second Known Thing — the two-fold number

drug discovery · target selection and target validation

Walk to a hall that hires molecules. Before anything is screened, optimised or dosed, somebody has to choose a target — a protein, a pathway, a gene product — and pair it with a disease. That choice is the origin of nearly everything downstream, and its failure mode is the expensive one: a programme that does exactly what it was designed to do to a target that does not move the disease. For two decades the field's most persuasive answer has been human genetics. If natural variation in a gene shifts the risk or severity of a condition in people, then modulating its product is an intervention nature has already run, and the target-disease pair inherits a kind of evidence no cell line can supply.

the number that settled the argument

The argument was settled, as far as the industry is concerned, by a statistic. Nelson and colleagues, in 2015, matched target-indication pairs in the development pipeline against the genetic associations known for those indications and reported that pairs with human genetic support were about twice as likely to reach approval. King and colleagues revisited it in 2019 with a different pipeline census and different matching rules and put the ratio a little higher, around two and a half. Minikel and colleagues refined it again in 2024, with tighter similarity criteria between the genetic evidence and the indication, and with evidence classes separated — the relative success rises well above the headline figure for the strongest classes and falls toward nothing for the weakest. The number is now load-bearing. It is quoted in strategy documents, it justifies the construction of platforms like Open Targets that score and rank target-disease pairs, it underwrites biobank partnerships, and it is the single most cited quantitative reason for the shape of the modern early portfolio.

and the thing the number is computed on

The census from which the ratio is computed is the development pipeline: target-indication pairs that somebody decided to work on. There is no other census available, and this is not an oversight — a pair nobody pursued has no clinical outcome, favourable or otherwise, so it cannot enter a calculation about approval. The authors of these papers are careful people and the confounders they discuss are real ones: genetic tractability differs by target class, some indications are better powered for association than others, and the growth of genome-wide association studies means that support itself is easier to have in 2024 than it was in 2008. What is not in the discussion, anywhere that a search can find it, is the observation that from around 2015 the statistic began to be used, by the people assembling the very pipeline it is computed on, as a reason to admit a pair.

III.The Bridge

no target paper cites Thorndike

The halls share nothing. Industrial and organisational psychology and human genetics have no common citations, no shared journals and no review that spans them; the word validity means a property of a measuring instrument in one hall and a property of a biological hypothesis in the other, and the word selection means a hiring decision in one and, awkwardly, natural selection in the other. What they have in common is an arithmetic situation. Both fields possess a predictor. Both estimate its worth by correlating it with an outcome. And in both, the outcome exists only for the cases the predictor itself admitted, so the estimate is taken inside a band the predictor drew. One of the two halls discovered this in 1903, has argued about the correction for eighty years, and knows exactly which quantity the uncorrected number reports.

X: the instrument grades itself inside the band it drew

Call it admission restriction. Where a predictor is used to decide which cases are admitted, its measured association with the outcome can be computed only among the admitted, in whom the predictor's own variance has already been reduced by the admitting; so the coefficient reports the spread the rule permitted rather than the predictor's relation to the outcome, and it is not identified at all without either a documented selection function or a stream of cases admitted for reasons unrelated to the predictor. From which follows the part that is easy to miss and is the whole of the interest here: the measured validity of such a predictor falls as the predictor is trusted more, so a rule that is working looks worse each year that it is adopted more widely, and the field's estimate of it tracks its own uptake rather than its truth. Two further consequences are worth stating exactly, because they are not the general caution about selection bias. First, the direction is determinate, not merely unknown: under restriction on the predictor the observed coefficient is attenuated, so the genetics ratio computed on a pipeline that increasingly uses genetics is a lower bound that loosens over time, and every revision that tightens the matching without modelling the admission is estimating a moving quantity more precisely. Second, the remedy has a shape the psychometric hall has already worked out and the genetics hall has not looked for: the correction needs the selection ratio and a partially unselected admission channel, and the pharmaceutical pipeline has one sitting in it — the pairs that entered for reasons that had nothing to do with genetics at all.

IV.The Question That Becomes Possible

neither discipline would ask this alone

The genetics hall asks whether the ratio is right, and answers by improving the matching between the evidence and the indication. The psychometric hall never looks at drug pipelines. Neither asks whether the ratio is a range-restricted coefficient whose movement over time is a reading of the industry's own adoption of genetics — which is a question about the number's behaviour rather than its value, and one that the data to answer already exist for.

the admission-restriction hypothesis

Does the measured enrichment of human genetic support among approved targets behave as a range-restricted validity coefficient — declining across eras of pipeline entry in step with the adoption of genetics-led target choice, and standing higher and steadier in the channel of target-indication pairs admitted for reasons unrelated to genetics — so that a correction using the admission ratio yields an unrestricted validity that is stable across eras where the raw ratio is not?

why it would matter

Because three groups currently read the same number and would have to read it differently. A sponsor deciding how much weight to put on genetic support is using a figure estimated on a pipeline built by sponsors already using it, and the figure is therefore worth least exactly where it is quoted most, on the pairs most like the ones already being chosen. Anyone arguing that genetics has been oversold would gain a specific and falsifiable version of that argument: not that the ratio is wrong but that it must fall as adoption rises, so a stable or rising series is evidence against the account rather than for it. And the correction, if it can be made, changes the operational advice in a direction nobody currently gives — it implies that the marginal value of genetic support is concentrated in the pairs the field is least inclined to run it on, because those are the ones still admitted for other reasons, and that a portfolio which admits some pairs deliberately without regard to genetic support is buying, at the price of those programmes, the only estimate of its own selection rule that anyone can have.

V.How It Could Be Proven — or Broken

a candidate must be breakable, or it is merely poetry

An intercognix candidate earns nothing until it names its own test. This one is unusually cheap: the series it needs can be built from sources that several groups already hold, and its central prediction is about a slope that nobody has plotted.

the honest status

Note the narrowness. The claim is about one statistic and its behaviour over time. It does not say that human genetic evidence fails to predict clinical success; the mechanism, if it holds, says the opposite, that the published figures understate by an amount that grows with adoption. It proposes no new evidence class, no change to how associations are matched to indications, and no revision of any published estimate — only that the estimates should be dated by pipeline entry and read as a series rather than as a value, and that the series has a predicted shape. It is also worth saying what the import from the other hall does not carry. Personnel selection is a field with its own unsettled questions about fairness and about what its criterion measures in the first place, and none of that transfers; what transfers is a piece of algebra about conditioning and the eighty years of argument about when it may be undone. The whole thing rests on an assumption that is itself checkable and might fail at once: that the pipeline's admission of a target-indication pair is meaningfully selective on genetic support at all, rather than dominated by chemistry, competition and commercial judgement, in which case the range was never restricted and there is nothing here to correct. Until the entry dates are joined, the eras plotted and the unselected channel estimated separately, this remains exactly what its stamp says: a candidate. A structural echo between two halls of the library, waiting for someone with the tools of both to walk the corridor.