The Delay That Looks Like Death

I.The First Known Thing — the partner that has gone quiet

distributed computing · unreliable failure detectors

A set of computers must agree on a single value — which of two orders to apply first, which of them is the leader, whether a transaction happened — and they can talk to one another only by sending messages that take an unknown time to arrive. Some of them may crash. In 1985 Fischer, Lynch and Paterson proved a result the field has lived with since: in such a system, if there is no bound on how long a message may take and even one computer may fail by stopping, no deterministic procedure can guarantee that the rest will ever agree. The reason is not that the failure is hard to survive. It is that the failure cannot be seen. A computer that has crashed and a computer that is merely slow, or whose messages are merely delayed, present exactly the same evidence to everyone else, which is silence; and any procedure that waits for the slow one may wait for ever, while any procedure that gives up on it may give up on a computer that was about to speak.

the detector that is allowed to be wrong

The way out, found by Chandra and Toueg in work published in 1996, was to stop trying to make the silence readable and instead to name precisely how unreadable it may be. Each computer is given a failure detector, a module that maintains a list of the computers it currently suspects of having crashed. The detector is permitted to be wrong: it may suspect a computer that is alive, and it may later take the suspicion back. What matters is which of two properties it satisfies, and how well. Completeness says that every computer that really has crashed is eventually suspected, permanently, by every computer still running. Accuracy says that computers still running are not suspected — in its strong form never, in its weak form at least one of them never, and in the eventual forms only after some unknown time has passed. The two pull against each other for an obvious reason. A detector becomes more complete by suspecting sooner, and suspecting sooner is what makes it suspect the living. Combining the two properties in their strengths gives eight classes, and the field's central theorems say what each class buys. A perfect detector, complete and never wrong, cannot be built where delays are unbounded, since building it would amount to reading the silence. But agreement can be reached with a far weaker one: a detector that is complete and merely eventually weakly accurate — that after some finite but unknown time stops suspecting at least one computer that is alive — suffices, provided a majority of the computers do not crash; and Chandra, Hadzilacos and Toueg showed that this class is the weakest that can do it.

how the protocol survives its own suspicions

The protocols that use such a detector have a shape worth stating plainly, because it is the part that transfers. They never act irreversibly on a suspicion. A computer that suspects the current coordinator does not destroy anything; it moves on to the next coordinator in a fixed rotation, and if the suspicion was wrong the passed-over coordinator simply loses its turn. The decision itself — the one step that cannot be undone — is taken only when a majority have acknowledged the same proposed value, and that condition is checked without consulting the detector at all. The consequence is a division of labour that the field states as a slogan: the detector is responsible for liveness, for the protocol eventually finishing, and for nothing else; safety, the guarantee that no two computers decide differently, holds whatever the detector says, however often it is wrong. A wrong suspicion costs time. It can never cost correctness.

the only lever that moves the trade

The last thing the field learnt is how such detectors are actually built, and it is the least glamorous part. They are built from time-outs. A computer that has not heard from a partner within some interval suspects it; if the partner then speaks, the suspicion is withdrawn and the interval is lengthened, so that the same partner is not wrongly suspected for the same delay again. In a system that is partially synchronous — where bounds on message delay exist but are not known, or hold only after some unknown moment, the model Dwork, Lynch and Stockmeyer set out in 1988 — this lengthening eventually overtakes the true delay and the detector settles into the eventually accurate class. Two things follow that the field regards as obvious and that are not obvious anywhere else. The first is that the quality of the detector is not a property of the rule but of the timing of the system: a cleverer rule cannot improve completeness and accuracy together, because the two are set by where the time-out sits against a delay distribution the rule cannot see; the only way to improve both at once is to tighten the delay bound. The second is that being wrong is not a defect to be engineered away but a budget to be spent: a detector is allowed a period of wrong suspicion, the protocol is designed so that the period costs only delay, and what is demanded is not that the suspicions be right but that they eventually stop being wrong.

II.The Second Known Thing — the programme that has gone quiet

drug discovery · portfolio direction and programme termination

A pharmaceutical portfolio is a set of programmes drawing on one budget, and its governance is, more than anything else, a machinery for stopping them. Most programmes fail; the industry's figures put the chance that a molecule entering the clinic reaches the market at something like one in ten, and the cost of development is dominated not by the successes but by how long the failures were carried before they were stopped. So the field has made termination a discipline in its own right. Stage gates fix the points at which a programme must be re-approved. Paul and colleagues, in their 2010 analysis of research productivity, gave the industry its phrase — quick win, fast fail — and argued that the largest single lever on the cost of a new medicine is moving the decisive experiment earlier so that the programmes destined to fail do so cheaply. Decision frameworks followed: the five-dimensional test AstraZeneca reported in 2014, with its right target, right tissue, right safety, right patient and right commercial potential, and its reported improvement in the rate at which projects passed through the clinic, is the best known. The literature on why termination is nonetheless hard is equally established, and mostly borrowed from behavioural economics: sunk cost, escalation of commitment, the champion who cannot let go.

the two ways a termination can be wrong

The field also knows, and records, that a termination can be wrong in two directions. The first is the zombie — the programme that is neither advanced nor stopped, that runs another study and then another, that outlives the hypothesis it was started on, and that is eventually closed for reasons its record describes as strategic. The second is the kill that should not have been made: the programme stopped for portfolio reasons, or on an inconclusive readout, that another sponsor later takes, on substantially the same evidence, to approval. Analyses of clinical failures sort discontinuations by stated cause — roughly half efficacy, a quarter safety, and a residue of a seventh or so labelled strategic or commercial — and it is in that residue, and in the programmes that were in-licensed after someone else let them go, that the second kind of error lives. The field discusses both. It discusses them in different rooms. The zombie belongs to the literature on governance and sunk cost, and its cure is a harder gate; the wrong kill belongs to the literature on licensing and the occasional retrospective on the drug that got away, and its cure is a more careful committee.

what the committee actually sees

What is seldom stated is what the committee sees when it decides. It sees a programme that has not produced its next result. Enrolment is slow, the assay is being repeated, the biology has thrown up a question, the readout is late. The programme that will never deliver and the programme that is merely late present, at the gate, the same thing, which is the absence of the evidence the gate was waiting for; and the committee's rule — however many dimensions it scores — is in the end a rule about how long to wait for that evidence before treating its absence as a verdict. The field does not describe its gates as time-outs. It describes them as decisions on the evidence. But a gate that a programme reaches without the evidence is a time-out by another name, and the length of it is set less by the framework than by the cadence of the therapeutic area: a viral infection reads out in weeks, a neurodegeneration in years, and the same committee with the same framework is patient in one and impatient in the other because the delays it is waiting against are different.

where the two errors go unjoined

The consequence is that the two kinds of error are treated as two separate failures of judgement, each to be reduced on its own. A sponsor embarrassed by a zombie tightens its gates; a sponsor embarrassed by a drug that succeeded elsewhere loosens them; and each move is reported as an improvement in decision-making, though what has moved is the length of a wait. Whether the two rates are in fact one quantity seen from two sides, whether any framework can lower both at once, and what, if not the framework, does lower both — these are questions the field has not put, because it has no reason to suppose that its committee is a time-out, or that the impossibility of reading silence is a theorem rather than a difficulty.

III.The Bridge

no consensus paper cites a stage gate; no portfolio review cites a time-out

The halls do not meet. Failure detectors belong to the theory of distributed computing, and their literature is about processes, messages and majorities; programme termination belongs to research management, and its literature is about attrition, gates and committees. No review spans them, and the one word they share — a programme, a process, being terminated — is a coincidence of vocabulary, not the bridge, since the computing hall's subject is not stopping a process but deciding, from outside, whether it has already stopped. Two neighbours should be set aside. It is not the non-derogability of an earlier specimen, in which certain constraints must be held exempt because their breach forecloses a return to normal; there the question is which properties may never be traded, whereas here the trade is unavoidable and the question is what sets its terms. And it is not the prefix refutability of another, in which a deadline converts a claim that can only be confirmed into one that can be refuted; that specimen concerns what a finite watch can establish about a course of events, whereas this one concerns what a finite watch can establish about the watcher's own partner — and the answer, in the computing hall, is that it establishes nothing, and that the system must be built to act anyway. What the halls share is a rule about deciding on the death of something whose only symptom is silence.

X: the silence is not evidence, and the wait is the whole rule

Call it crash indistinguishability. Where a thing that has failed and a thing that is merely delayed present the same evidence to an observer, which is the absence of a result, no rule the observer can apply reads the difference; every rule reduces to a wait of some length and a verdict at its end. Four things follow. Any such rule has two properties that pull against each other — completeness, that everything truly dead is eventually declared so, and accuracy, that nothing alive is — and no rule can be improved on both at once, because both are fixed by where the wait sits against a distribution of delays the rule cannot see; what looks like better judgement is movement along one curve. The curve itself is moved by one thing only, a tighter bound on how long a result can take to arrive, so that the quality of a termination policy is a property of the timing of the system it governs and not of the policy. A rule that is wrong for a while is not a broken rule: what can be demanded is not that its suspicions be right but that they eventually stop being wrong, and the withdrawal and lengthening of a wait after a false suspicion is how a rule becomes eventually right without ever becoming able to see. And a system survives such a rule by never acting irreversibly on its suspicion — suspicion is allowed to cost time and forbidden to cost anything else — with the irreversible step reserved for a condition the detector is not consulted on. Read into the portfolio hall, each has a consequence the hall does not state. A sponsor's zombies and its drugs that succeeded elsewhere are its incompleteness and its inaccuracy, and they are two readings of the wait its gates impose; a framework that scores more dimensions changes where a programme sits against the wait but not the wait, and cannot lower both. The lever that lowers both is not in the committee room at all but in the latency of the evidence — how long a decisive readout takes in that therapeutic area, and whether that time is bounded — so that a portfolio's termination quality is inherited from the cadence of its biology. And the design that survives wrong kills is not a better committee but a reversible one: a suspicion that shelves — preserves the data, the material, the rights and the option — rather than kills, with the irreversible step, the disposal of the asset, tied to a condition that does not depend on how long the programme has been silent.

IV.The Question That Becomes Possible

neither discipline would ask this alone

The portfolio hall asks how to kill the right programmes and reads each kind of wrong kill as a failure of judgement to be corrected on its own. The computing hall has no programmes, but it has established that where failure is seen only as silence, no rule can be both complete and accurate, that the two are set by a wait against a delay distribution, that only bounding the delay improves both, and that systems survive by making suspicion cheap. Neither hall asks whether a sponsor's rate of programmes kept too long and its rate of programmes killed too soon lie on one curve, whether decision frameworks move sponsors along it or off it, and whether the curve is placed by the latency of the evidence.

the crash indistinguishability hypothesis

Across sponsors and therapeutic areas, do a sponsor's incompleteness — the share of its programmes that outran their stage's expected duration without a decision and delivered nothing — and its inaccuracy — the share of its non-safety, non-efficacy terminations later taken to approval by another sponsor on substantially the same evidence — trade against each other along a single curve whose position along it is set by the sponsor's characteristic wait, the time from a programme's last milestone to its termination; is the curve displaced towards the origin by the bound on readout latency in the therapeutic area and not by the adoption of a multi-dimensional decision framework; and do sponsors whose terminations are reversible pay less for a given inaccuracy than sponsors whose terminations dispose of the asset?

why it would matter

Because the industry has spent two decades improving its committees and reports the improvement as better decisions, and if the two kinds of error are one dial, then a better committee has been turning the dial rather than removing it, and the reported gains in one error should be found again as losses in the other. If the curve rule holds, a sponsor's termination record can be scored as a position and a curve rather than as a pair of embarrassments, and the two sponsors who kill fastest and slowest may be found equally good. If the latency rule holds, the investment that improves termination is not in governance but in shortening and bounding the time to a decisive readout — the biomarker that reads in weeks, the adaptive design that reports early — and the value of such investments can be stated as a displacement of the curve rather than as a saving on one programme. And if the reversibility rule holds, the cheapest correction available to any sponsor is to stop killing and start shelving, since a suspicion that can be withdrawn is one the theory says may be made freely.

V.How It Could Be Proven — or Broken

a candidate must be breakable, or it is merely poetry

An intercognix candidate earns nothing until it names its own test. This one needs no new trial; it needs the pipeline databases the industry already subscribes to, in which every programme's stage transitions, discontinuation dates and stated reasons, and later fate in other hands are recorded.

the honest status

Note the narrowness. The claim concerns terminations made in the absence of a decisive result — the programme that is late, silent, inconclusive — and says nothing about terminations made on one: a programme killed for a clear safety signal or a failed pivotal trial has spoken, and no theory of silence bears on it. It does not say that decision frameworks are useless; it says they change where a programme sits against the wait, which is a real change, and denies only that they change what the wait can achieve. It does not say that patience is a virtue or that haste is; it says the two are positions, and that the field has been comparing positions as if they were competences. It is worth saying what does not transfer. A failure detector is one module among many identical computers, and its wrong suspicion is corrected by the others; a portfolio committee is alone, and its wrong suspicion is corrected, if at all, by a competitor, years later, at the first sponsor's expense. The computing hall's protocols have a majority to consult for the irreversible step, and it is not clear what the portfolio's majority would be — perhaps a second, independent readout, perhaps a market. And the computing hall's time-outs lengthen after a false suspicion by rule; a committee that was wrong lengthens its patience by embarrassment, if at all, and the theory's guarantee that the rule eventually becomes accurate depends on that lengthening actually happening. The whole thing rests on one tabulation that a single analyst with a pipeline database could make in a season: whether a sponsor's two kinds of wrong termination trade along one curve, and what places the curve. If they do not, this is an elaborate way of saying that committees make two kinds of mistake, and should be said so. Until that tabulation is made, this remains exactly what its stamp says: a candidate. A structural echo between two halls of the library, waiting for someone with the tools of both to walk the corridor.