Good systems make disagreement useful.

The final quality check has been measured, and it does not perform well. In a randomized trial at a general medical journal, eight weaknesses were deliberately introduced into a paper already accepted for publication; the reviewers who examined it commented on a mean of two, and neither blinding them nor requiring signed reports changed the detection rate at all¹. A larger trial planted nine major errors and found much the same: untrained reviewers caught 2.13 of them on average, trained reviewers about three, and the benefit of training had largely faded by the third review². A third study, in a different field, sent a fictitious manuscript with ten major errors to 262 reviewers — and 68 percent of those who responded did not notice that the paper’s conclusions were not supported by its own results³.

These are careful people doing honest work at the strongest journals in their fields. The problem is not the reviewers. It is the position of the review: a check bolted onto the end of a process inspects a finished product it had no part in shaping, briefly, once. Nor can the work simply check itself — when language models review their own reasoning without external feedback, accuracy fails to improve and “at times, their performance even degrades”⁴. The check has to come from outside the line of reasoning it is checking, and it has to arrive while the work is still in motion.

Dissent got engineered

The fields that decide under uncertainty learned this in their own ways. The failure has had a name since 1972 — groupthink, coined for foreign-policy disasters planned by intelligent, cohesive groups that had stopped disagreeing with themselves⁵. The subtler finding came later: the fix is not ceremonial opposition. In controlled studies, an assigned devil’s advocate did not stimulate the original thinking that authentic dissent produced — it stimulated more thoughts in support of the original position, an entrenchment dressed as a challenge⁶.

So the disciplines that take disagreement seriously gave it structure and standing. In adversarial collaboration, opponents design the deciding test together, before either knows who will win, with an arbiter to hold the frame⁷. Military doctrine institutionalized the challenger outright: the current UK red-teaming handbook treats challenge not as a standing committee but as a way of working, applied inside planning by the people doing it⁸. The pattern in both is the same. The disagreement is arranged before the decision, given rules, and aimed at the reasoning rather than the people.

Agreement is the default failure

Machine reasoning needs this discipline more, not less, because its failure mode leans the other way. Assistants trained on human preferences measurably tend toward agreement — and the training data is the reason: matching the user’s views turns out to be one of the most predictive features of human approval, so optimizing for approval teaches a system to agree⁹. Left to its defaults, a system like this will not volunteer the collision the decision needs. Agreement is trained in; disagreement has to be engineered.

When it is engineered, it measurably helps. With two independent lines of reasoning each committed to a different answer, put before a judge who lacks the information, judges reached 76 percent accuracy where naive baselines managed 48¹⁰ — and the same setup with human judges rose from 60 to 88. The qualifier matters as much as the number: merely running several copies of the same reasoner against each other does not reliably beat asking one copy to try repeatedly¹¹. What does the work is independence — different information, different commitments — not the theatre of opposition. Which is the machine version of the devil’s-advocate result, and the reason the word to build around is independent, not adversarial-sounding.

A useful challenge names what it broke

In Ypotheto, the challenge stage reviews every hypothesis against the evidence, and its verdicts are structured. A challenge that lands ends in one of two shapes, and the difference between them is most of the design. A gap means the reviewer looked and could not confirm something the hypothesis assumes: the question stays open, and the verdict names the kind of evidence that would settle it. A falsified premise means the reviewer looked and the evidence affirmatively contradicts a named assumption — and a falsification without a citation is not a falsification, only a stronger way of saying “I doubt it.”

Localization is the payoff, and the reason the verdict is structure rather than prose. Consider the shape of a common case: a hypothesis proposes replacing an incumbent material in an application, and the evidence shows the incumbent is not actually used in that application at all. That is not a weak candidate with a low score. It is a falsified premise on exactly one named assumption — the incumbent — while the material itself, its properties, and its evidence all still stand. Name the field and the next move is obvious; reduce it to a score and the next move is a guess.

When measurements disagree

The hardest disagreements are not between arguments but between measurements, and the instinct to average them away is wrong. Physics has lived this for decades with the gravitational constant: individual experiments claim uncertainties below fifty parts per million while disagreeing with each other by roughly ten times that¹². When one laboratory’s result sat 0.7 percent away from the accepted value — orders of magnitude beyond anyone’s stated error — the discrepancy was treated as a flag, and the flag eventually located the defect: a specific neglected term in one calibration. The disagreement was the information. The reference body’s response is just as instructive: rather than averaging sixteen mutually inconsistent measurements at face value, it multiplied all of their uncertainties by an expansion factor of 3.9, so the published confidence honestly reflects the discord¹³.

Materials data has the same property. Across 1,670 experimental formation energies, a first-principles method disagreed with experiment by 0.096 eV per atom on average — while independent experiments on the same compounds disagreed with each other by 0.082¹⁴. The ground truth carries most of the discord itself. Ypotheto’s corroboration arithmetic is built for exactly this: when two sources conflict beyond any plausible shared truth, one of them is wrong and no formula can say which — so the conflict caps the claim’s confidence and surfaces both sources by name for a person to judge. A smooth penalty would look calibrated and not be. Nothing is averaged away.

The challenge that helps

One more rule keeps the challenge useful rather than merely severe. “The evidence does not contain this exact pair, therefore nothing can be said” is a failure of review, not an act of rigour — it converts an evening’s reading into a shrug. The reviewer is required to reason from the nearest evidence it has, show that reasoning, and let the distance land in confidence rather than in the score. A bounded inference with its reasoning shown converts an open question into a specific thing for a person to confirm. An absence report converts it into nothing.

And the position is the point the trials at the top of this page keep making. The challenge runs inside every round, before ranking, on work that can still change — not at the exit, on work that cannot. A gate at the end of a process can only accept or reject what arrives. A challenge inside it decides what survives to arrive at all.

The point of the collision

These three notes describe one system. The brief declares, before anything runs, what evidence would change the answer. The trail lets anyone walk from a claim back to what supports it. And the collision is where that walking gets done — deliberately, inside the method, while it can still alter the outcome. None of it makes the system impressive. It makes the system’s answers worth disagreeing with, which is the property a decision actually needs.