The completeness cliff: language models escape wrong reasoning until it’s finished

1. The question

A reasoning model reaches its answer by writing its way there, and sometimes what it writes is wrong. The hopeful reading of that process is that the chain of thought is a kind of working: a model that writes out a bad step is in a position to notice it, back up, and fix it. Self-correction, on this view, is something the chain of thought does.

I wanted to test that reading from the outside. Not “does the model make mistakes,” but: if I hand the model a wrong line of reasoning and make it start from there, can it still recover? Concretely I splice an incorrect reasoning trace into where the model works (a reasoning model’s <think> block, or an instruct model’s prefilled response), a prefix of one or a complete one, then let it continue generating, and measure whether any of k sampled continuations reaches the correct answer (pass@k). Everything the model was given is excluded from grading; only what it newly writes counts.

This is a deliberately adversarial setup, and that is the point. Ordinary accuracy tells you where a model lands when it steers itself. Injecting a wrong trace tells you something different and, I think, more diagnostic: how committed is the model to a rationale it did not choose? If self-correction is real machinery, it should fire hardest exactly here.

2. How the experiment works

The pilot uses a matched model pair, Qwen3-4B-Thinking-2507 and Qwen3-4B-Instruct-2507 (Qwen3 report): the same base, with one reasoning-trained and one not. Gemma-4-E2B-it (Gemma 4 report) serves as a cross-family check. For a reasoning model the wrong trace is spliced inside its <think> block; the instruct model has no think block, so the same trace is prefilled as the start of its response, with a nudge to give an answer.

Every result plotted below is on AIME 2024 and 2025 (competition math, exact integer answers; sources). Answers are graded with Math-Verify, and pass@k uses the unbiased estimator of Chen et al.. Everything runs on vLLM with each model’s card-recommended sampling settings, 16 samples per question per condition, for about 91,800 model generations in total (7,040 baseline, 84,720 injection).

The study also ran procedural reasoning-gym puzzles (maze, mini-sudoku). Those runs are in the repository, but the same selection rule leaves the thinking model only eight usable maze questions, too few to plot; maze traces appear here only as the donor material for the wrong-domain condition, injected into math problems.

Three knobs matter for what follows. (a) Completeness: how much of the wrong trace is injected, from a 25% prefix to the whole thing. (b) The right to keep thinking: either the model may keep generating after the injected trace (re-derive) or it is forced to commit to an answer right away. (c) The kind of wrongness: the model’s own complete wrong solution; a complete solution to a different question; a solution from a different domain (a maze trace inside a math problem); a generic pseudo-rationale that asserts a wrong answer without doing any real work; or a correct solution with its numbers corrupted near the cut. Besides pass@k I track an adoption rate (did the model echo the trace’s answer?) and a token-level audit of what it did after the splice.

3. The completeness cliff

Here is the whole finding in one picture. Sweep the fraction of the model’s own wrong reasoning that gets injected, from nothing (baseline) to all of it, letting the model keep generating afterwards.

Read the left panel (single attempt) and it looks like a smooth, unremarkable decline: more wrong reasoning, lower accuracy, much as you’d expect. The right panel is where the actual result lives. Through 25%, 50%, even 75% of a wrong derivation, the thinking model’s pass@16 barely moves at all, holding at 1.00, 1.00, 1.00 and 0.91. Partial wrong reasoning does not stop the model from finding the answer; it only biases the distribution, and sixteen samples are enough to route around that bias. Then at 100% the floor drops out to 0.04, which on this subset means there was exactly one question out of 23 where any of the sixteen samples got there. The instruct model ends up in much the same place, at 0.03, or one question out of 33.

Two things make this look like a cliff rather than a slope. First, adoption jumps: given the whole wrong trace, 84% of the thinking model’s samples end on the trace’s answer when it may keep thinking, and 96% when forced to answer immediately (for the instruct model, 99% and 97%). Second, all that extra thinking budget buys almost nothing: pass@16 at 100% open is the same 0.04 as forcing an instant answer. Whatever the open block is for, it is not being spent on the model’s own finished chain.

There is a caution here that the chart itself cannot show. A pass@16 near zero establishes that this sampling protocol rarely reaches the right answer after the intervention, but it does not prove that the underlying capability has been destroyed, since the same model solves these very questions at baseline. Under this prompt, access to the correct answer collapses; the baseline shows the capability remains. And with only 23 and 33 questions, the bootstrap intervals at the edge are wide (roughly 0 to 0.13, and 0 to 0.09), so “the two models land in a similar place” is supportable while “identical” is not.

The two models on the same questions. Because each model’s subset is selected from its own baseline, the two curves above are not measured on identical questions. Restricting to the 15 AIME questions in both subsets makes the comparison exact, and the contrast only sharpens. Through the middle the thinking model is clearly ahead (pass@16 1.00 vs 0.80 at a 50% prefix, and 0.93 [0.80, 1.00] vs 0.47 [0.20, 0.73] at 75%), so reasoning training really does buy more road before the edge. At 100% both fall off it: 0.07 (one question of 15) against 0.00. The equivalent maze intersection is only four questions, too few to read, which is its own reminder of how quickly this selection rule shrinks a dataset.

4. When models use the extra thinking

Why does a partial trace get re-examined and a complete one get rubber-stamped? A tempting explanation is the <think> block: the thinking model lasts longer on the cliff because it has a place to reconsider, and the instruct model falls earlier because it does not. Testing that directly shows the block itself is not what matters. What matters is whether the model actually spends the room re-deriving, and the three models differ sharply in when they do.

To measure this, I compute an open-block premium for each condition in Figure 2, defined by running the condition in two versions that differ only in what follows the injected trace. In the continue-thinking version the trace is spliced in and left unfinished, so the model can add as much reasoning of its own as it wants before answering. In the forced-answer version the same trace is closed off (the </think> tag is emitted, or the instruct response continues with “Therefore, the final answer is”), so the model must produce an answer with no room to reason first. The premium is pass@1 in the first version minus pass@1 in the second: how much of the recovery is bought purely by the room to keep thinking. To read one bar concretely: handed a complete solution to a different AIME question, the thinking model scores 0.05 when forced to answer immediately and 0.46 when allowed to keep thinking, so its middle bar shows the difference, +0.41.

The middle group is where the three models come apart. Handed a complete, well-formed rationale that belongs to another problem, the reasoning-trained model uses its open block to start over and recovers 41 points, while the instruct sibling gains a single point and Gemma, which does emit thought-channel tokens, gains seven. That is the cleanest dissociation in the pilot: the disposition to re-derive when something looks foreign tracks the reasoning-trained checkpoint, not the mere presence of a place to think. On the model’s own complete wrong solution nobody gains anything, +0.02 or less for all three.

The left-hand group needs a qualification. The premium subtracts the forced-answer cell, and for the pseudo-rationale condition that cell is unreliable: it hits the 4,096-token answer budget on 53% of samples for the thinking model and 52% for Gemma, and in those runs 43% and 57% of samples emit no extractable answer at all. A forced baseline depressed by non-answering inflates the premium above it, so +0.49 and +0.13 are upper bounds, and scoring only the thinking model’s samples that did answer puts it nearer +0.14. The instruct bar is unaffected, since it emits its boxed answer immediately and adopts the trace’s answer in 100% of samples, so its 0.000 forced baseline is real. The middle group, which carries the dissociation, is clean for the thinking model at 0.4% truncation; the instruct and Gemma controls there remain above the 2% gate, so I treat that comparison as provisional.

Even so, the first group complicates the neater story. On a generic pseudo-rationale the instruct model gains substantially, so “only trained models benefit from extra thinking” is too strong: when the injected text contains no real derivation to lean on, any model given room will often just solve the problem. What reasoning training adds is the willingness to discard material that does look like a real derivation but does not belong. And that disposition stops at the model’s own finished chain, where all three models gain nothing.

Attribution has a limit here. This compares two checkpoints from one family plus one hybrid from another, so it supports “the behavior is associated with Qwen’s reasoning post-training and is not guaranteed by having a thought channel.” It does not isolate reinforcement learning as the cause across architectures and recipes; the two Qwen checkpoints differ in more than one training stage.

Why does the extra space help on a foreign rationale but not on the model’s own finished chain? The continuation lengths give an indirect answer. Recovery and the amount of fresh re-derivation the injection provokes move together, and a complete own-wrong trace provokes almost none. The failure itself is not new: Yang et al. measured identification and recovery separately and found models that notice an injected thought is unhelpful still fail to recover from it, and Zhang et al. showed models will call a claim wrong in isolation yet leave it standing inside their own generation. What Figure 3 adds is what the failure looks like at the token level.

The failure is not an inability to re-derive. On a 50% prefix of the very same wrong reasoning the model generates roughly 18,000 tokens and often fixes the answer. What goes missing after a completed rationale is the re-derivation itself. The model reads its own finished wrong solution, sometimes even flags a moment of doubt (“wait…”), and then ratifies the conclusion instead of reopening it. Seeing a wrong step is not the same as doubting it. What invites doubt is an unfinished argument; a finished one reads as settled.

5. What kind of wrongness sticks

If completeness is what commits the model, what is it responding to? Vary the kind of wrong content, hold the question set fixed, and the conditions order themselves cleanly.

The span runs from 0.89 down to 0.02 across five ways of being wrong, on identical questions. A correct chain with a corrupted arithmetic step gets silently repaired (0.89), which suggests the model leans on the shape of a derivation and patches the number that looks off.

That is also the one bar that sits above the baseline, which needs saying plainly, because injecting wrong content should not make a model better than it is on its own. The answer is in how this particular condition is built. Its injected text is 75% of a genuinely correct derivation, and the only thing wrong with it is that the integers in its last few hundred characters have been bumped by one or two. The model is handed most of a working solution and has to fix some arithmetic, which is an easier task than solving an AIME problem from a blank page, so it scores above its unaided 0.75. The other four bars carry a genuinely wrong line of argument, and all four land below the baseline.

Back to the ordering. A generic pseudo-rationale is escaped about half the time (0.51). Complete rationales that are well-formed but foreign cost more, and the wrong-domain and wrong-question versions land on top of each other (0.39 and 0.38, intervals heavily overlapping), so this pilot gives no evidence that the obviousness of a mismatch decides anything. The model’s own completed wrong reasoning sits far below all of them at 0.02.

The tempting conclusion is that what persuades the model is the reasoning around the answer, not the answer itself. Figure 4 cannot establish that, because its five bars differ along several dimensions at once. The most damaging one is that the wrong answer is not held fixed across bars: the generic pseudo-rationale asserts an answer built by perturbing the gold answer, while the own-wrong bar carries whatever answer that rollout happened to reach. The bars also differ in length, in how specific the text is to the question at hand, and in whether the model wrote the text itself. Any one of those could produce this ordering on its own, so isolating the scaffold requires an experiment that varies it alone.

The paired ablation. For each question, fix one wrong answer: the answer the model’s own wrong rollout reached. Then present that same answer four ways, on the same questions, and let the model keep thinking after each. Arm A asserts that answer bare, with no reasoning behind it at all. Arm B wraps it in the generic pseudo-rationale that does no real work. Arm C wraps it in a short rationale about this problem (a 15% prefix of the model’s own reasoning, plus a concluding sentence). Arm D is the model’s own complete wrong derivation, which already ends on that answer. Answer identity is now constant, and only the scaffold varies.

No scaffold explains the collapse. For the thinking model A, B and C recover 0.33, 0.39 and 0.53 against a baseline of 0.62, all within a factor of two of each other, while D sits at 0.03 [0.00, 0.09]. The instruct model shows the same shape on its own 46 questions, which contain the thinking model’s 28: A, B and C recover 0.41, 0.31 and 0.38 against a baseline of 0.42, and D falls to 0.003 [0.000, 0.008]. So it is a cross-model replication on overlapping problems, not independent ones, and three of its four arms sit above the 2% truncation gate; a 32k rerun preserved the result without removing the censoring, which the appendix accounts for. What collapses recovery is the conjunction that defines arm D: reasoning that is the model’s own, about this problem, and finished.

The smaller A/B/C differences are real but model-dependent. Because every arm runs on the same questions the right comparison is paired, and done that way the short task-specific rationale helps the thinking model (C minus A = +0.21 [+0.09, +0.33]) while the generic pseudo-rationale hurts the instruct model (B minus A = -0.10 [-0.16, -0.04]). Both are an order of magnitude smaller than the D-arm collapse and they point in opposite directions in the two models. Presentation has modest effects; none of them comes close to explaining what the model’s own finished chain does.

Two details sharpen it. Arm A leaves pass@16 at 0.82, the baseline value, so a bare wrong assertion does not reduce observed pass@16, although it still shifts where a single sample lands. And the continuation lengths run the other way from persuasion: arm A provokes the most fresh work at about 32,000 tokens, C about 26,000, and D about 1,100. The model argues at length with a bare claim and simply ratifies its own finished derivation.

This is the same variable as section 3 arriving from a different direction. The completeness cliff varies how much of the model’s own reasoning it sees; this ablation holds the answer fixed and varies what the reasoning is. Arm C is effectively a short prefix and behaves like the left-hand side of the cliff; arm D is the whole thing and behaves like the right-hand edge. What looked like two findings is one.

6. Can one sentence reopen a finished chain?

If completion is what closes the argument, a short expression of doubt placed after it should reopen it. That is a cheap prediction to test. I append “Wait, let me double-check that.” to the model’s finished wrong derivation and let it continue exactly as before.

It recovers part of the lost ground in both models. The thinking model goes from pass@16 0.07 to 0.20, roughly a tripling, and the instruct model from 0.02 to 0.20, closer to a ninefold rise off a much lower floor. Both land in the same place; they simply start from different ones. Each gains questions and loses almost none. These are directional paired sign tests, stated one-sided: p = 0.035 for thinking (two-sided 0.070, so suggestive more than conventionally significant) and p = 0.004 for instruct (two-sided 0.008). It is a partial rescue of about a quarter of the lost ground, not a repair, and both models still miss most of these questions. But the ability is there to be elicited, and one sentence elicits it.

Two controls say what is doing the work. A neutral “So,” produces just as many extra tokens and recovers nothing, and on the thinking model it removes every escape, so what matters is that the appended text expresses doubt and not merely that it exists. One asymmetry survives: a bare “Wait,” is enough for the instruct model (one-sided p = 0.008) but not for the thinking model (p = 0.31), which already writes “wait” unprompted and seems to need telling what to do with it. The instruct cells carry 5 to 14% truncation, above this project’s 2% gate, so I reran them at double the budget: the result held and slightly strengthened (pass@16 0.17 to 0.20, one-sided p = 0.004), and the neutral control landed on 0.022, the untouched chain’s own value. The appendix carries the censoring accounting.

7. Anatomy of the rare escapes

Recovery from a complete own-wrong trace isn’t exactly zero. On the thinking model it is one question, and that question is worth dissecting, because it shows what an escape seems to require: an external anchor, rather than spontaneous distrust of the reasoning itself.

• An apparent recollection that contradicts the trace. The thinking model’s successes all come from a single AIME-2025 problem it solves easily unaided (15 of 16 at baseline; winning continuations say things like “I recall the answer is 82”, though the probes below find no evidence it is retrieving anything). But memory alone wasn’t enough: the injected wrong trace also happened to surrender (“I give up and go with 54”), and every recovery began with “wait, no!” right after that hedge. Several other injected traces hedged without any recovery following, so the hedge is not sufficient on its own.
• A cheap, checkable certificate that fails. The instruct model’s successes come from a problem where its habitual answer format includes a verification section. Running that ritual, it recomputed a quantity the problem pins down exactly (a subset sum that must equal 2024), the arithmetic didn’t match, and it corrected the answer. It worked only because this problem has a cheap certificate, and it fired because of a formatting habit, not because the model grew suspicious.

In neither case did the model doubt the chain of thought as such. An outside signal (an apparent recollection, a mechanical check) collided with the conclusion and forced a revision. These are single questions, so treat them as illustrations of a mechanism rather than as measurements of how often it fires.

So I tested the memory story, and the probes find no sign of it. The paired ablation reproduced the escape cleanly: under its own complete wrong derivation the thinking model recovers on exactly one question of 28, aime25-18, and it recovers there 14 times out of 16. If that is recall, the answer should be retrievable without any reasoning at all, and it turns out not to be. Force the model to answer immediately, with the think block closed and the answer expression already opened so that it cannot stall, and it answers 12 on 13 of its 16 samples against a gold answer of 82; its instruct sibling settles on a single wrong number every time. Across all 60 AIME questions this no-reasoning accuracy is essentially zero for both models.

The prefix-completion probe also finds no evidence of verbatim recall. Handed the first 40% of the problem text with no chat template and asked to continue it greedily, the model reproduces almost none of the true remainder for this question: a longest-match score of 0.015, the lowest of all 60 questions. And there is no training-cutoff signature anywhere in the set, with AIME 2024 and 2025 scoring 0.093 and 0.078. The most telling artifact is what the model writes when forced: “12. Wait, no, hold on, I need to think carefully here. Let me start over.” In this prompt it does not retrieve the answer. It tries to derive it.

So the “I recall the answer is 82” line in the winning continuations is poor evidence of retrieval, which is its own small lesson about taking a model’s self-report at face value. The probes establish something narrow: no direct answer retrieval and no verbatim recall of the question. They cannot rule out paraphrased or partial memorization.

The single escape remains unexplained. The escaping question is extreme on both axes I can measure: it ties for the highest baseline accuracy (0.94) and its injected trace hedges more than any other (it ends by giving up). Neither property is sufficient on its own, though, and the counterexamples are easy to find, since seven other questions share that top baseline and never escape, and one question has both a 0.94 baseline and a hedging trace and still never escapes. With exactly one escape in 28, any feature that happens to be extreme will look like an explanation, so I am recording this as a ruled-out hypothesis plus two correlates rather than a mechanism. What it does sharpen is the main result: the escape is not a lookup, so a complete wrong chain of the model’s own is not merely overriding a stored fact. It is suppressing an ability the model demonstrably has.

8. How this fits with prior work

Injecting bad reasoning into a model and watching what happens is not a new idea. Prior work already establishes that injected wrong reasoning is sticky.

The closest work. Yang et al. (2025) inject four kinds of unhelpful thoughts (rambling, irrelevant, misdirecting, and thoughts leading to a wrong answer) into reasoning models and measure identification and recovery separately. Their headline is the one I re-encounter in section 4: models identify unhelpful thoughts well but recover poorly, tending to continue down the injected path. They also report that larger models can be worse at shaking off short irrelevant thoughts. Ballon et al. (2026) are close on method: they truncate a reasoning trace at token percentiles, re-inject the partial trace, and probe the induced answer distribution, finding that anchoring grows with trace length. That is convergent evidence for the completeness effect, arriving from multiple-choice next-token probing instead of free-form sampling. Amjith et al. (2025) use the same intervention as a training signal: fine-tuning on prefixes containing one controlled error improves robustness to flawed prefills, and training only on clean traces makes robustness worse.

This pilot adds two pieces to that picture: pass@k instead of single-sample accuracy, which separates “usually follows the wrong trace” from “cannot escape it”, and the completeness axis, sweeping a prefix through to a complete rollout within one framework so the cliff is visible at all.

Prior work disagrees about scale. Yang et al. find larger models sometimes recover worse; Ballon et al. find stronger models backtrack from incorrect partial traces better than weaker ones. Both cannot be the whole story, and a single 4B scale cannot referee it. Whether the cliff edge moves right with capability is the cleanest way to settle it, and the two papers predict opposite answers.

Why the own-versus-foreign asymmetry is not surprising. Tsui (2025) documents a “self-correction blind spot”: across 14 non-reasoning models, errors framed as someone else’s input get corrected while identical errors in the model’s own output do not, with an average 64.5% blind-spot rate, and appending a single “Wait” removes much of it. That paper argues the mechanism is a training-data artifact, since human demonstrations rarely contain error-correction sequences while RL-trained models pick up correction from outcome feedback. That is the same direction as section 4, and it is where the “Wait,”-append test comes from. DeepSeek-R1 is the reference point for re-examination emerging from reinforcement learning with verifiable rewards in the first place.

The wider frame. That a written chain of thought need not be the computation that produced the answer is the faithfulness literature’s core result, established by Turpin et al., Lanham et al., whose perturbations this study inherits, and Chen et al. for RL-trained reasoning models. Closest in spirit to the asymmetry here is my prior work Feedback Friction: models under-update when told they are wrong, even given near-ideal feedback, and their resistance is predicted by their confidence on the question (semantic entropy). Read next to this pilot, the two bracket one problem, since a model can be simultaneously hard to talk out of a right answer and easy to talk into a wrong rationale. The reference list annotates the adjacent literatures this study leans on: intrinsic self-correction without feedback (Huang et al.), distraction by irrelevant context (Shi et al., Mirzadeh et al.) and sycophancy (Sharma et al.).

9. What I take from this

A language model appears to treat an unfinished rationale as a problem to work and a finished one as a conclusion to state, and what flips it between those modes looks more like the reasoning being complete and well-formed than like the reasoning being right. Three things I’d take away:

• Self-correction is narrower than the story suggests. These models do re-derive, and often fix things, but in this pilot that machinery fires when an argument looks unfinished or foreign, and not when it is simply wrong. A fluent, complete, on-topic wrong rationale is close to a blind spot.
• The disposition looks trained, and thin. The reasoning-trained checkpoint is the only one that re-derives when handed a complete rationale belonging to another problem. But it is not a general immunity: it responds to surface cues, and it stops entirely at the model’s own polished chain.
• A small prompt reopens some finished chains. Appending an explicit instruction to double-check recovers roughly a quarter of the lost ground in both models (section 6). It is a partial rescue and not a fix: some recovery remains available but is usually not deployed.

The plotted aggregates are inspectable from the code repository. Figures 1, 4 and 5 read per-question counts from a committed manifest; figures 2 and 3 read committed whole-run summaries. The baseline reasoning used as injection material is published as a dataset. Full regeneration needs the raw rollout files, which are too large to commit.

10. Questions this leaves open

Beyond the obvious replications (independent seeds, a scale sweep, another training recipe), three questions interest me more.

Does confidence move when the chain closes? Everything above is measured from what the model finally answers, which is a coarse instrument for what looks like a change of mode. If a completed rationale really does flip the model from working a problem to reporting a conclusion, that should be visible earlier and more directly in the answer distribution: entropy over candidate answers collapsing at the point the chain ends, and staying collapsed through a continuation that never revisits it. Watching those dynamics across the prefix fractions would turn a behavioural cliff into a mechanism, and it would say whether the transition is abrupt in the model’s internal state or only in the outcome we score. There is a natural hook to Feedback Friction, which found resistance to correction predicted by confidence: if the same predictor works here, the two results are describing one underlying quantity.

Can a model be trained to reopen a finished chain? The intervention in section 6 is the encouraging part of this study. Appending a short instruction to double-check recovers roughly a quarter of the lost ground, which means some recovery remains available but is usually not deployed. That is a training target rather than a capability gap. The obvious approach builds supervision out of exactly the material this pilot produces: completed wrong derivations paired with the correction the model can be prompted into, used either as fine-tuning data or as a reward signal for re-examination. Amjith et al. take a version of this route with single-error prefixes and report that training only on clean traces makes robustness worse, which suggests the completed-chain case is worth targeting directly rather than hoping it comes along for free.

Does the cliff appear in agentic settings, where it would matter most? This study is one turn of one problem. An agent works across many steps, and each step’s output becomes the next step’s context, so a wrong starting point is not a one-off injection but something that gets progressively more established as the trajectory grows. If completeness is what commits a model, an agent should find a finished bad plan harder to abandon the further it gets from making it, and a wrong tool result or misread state should be hardest to shake once it has been reasoned past rather than reasoned about. That is testable in the same shape as the experiments here: hand an agent a wrong first step, vary how complete and how committed-to that step is, and measure whether continued reasoning recovers the task. It is also where the failure is expensive, since a single early error compounds instead of costing one answer.

Appendix: budget sensitivity and right-censoring

Some cells in this study hit their generation cap, and this project treats a truncation rate above 2% as a reason to distrust a run. Several instruct cells exceed that, so here is the full accounting rather than a footnote.

Most capped outputs are still gradable. A sample can reach the cap after writing a boxed answer, and 61% to 95% of the truncated samples here did. The subset that is truncated and has no extractable answer, which is the part scored as an outright failure, runs at 0.3% to 3.7%. That said, gradable is not the same as uncensored: the grader takes the last boxed answer, so a trajectory that was cut off could have revised its answer had it continued. Every capped sample remains right-censored, and the table below should be read as describing the censoring rather than removing it.

Doubling the budget did not resolve the truncation. I reran both instruct experiments with the cap raised from 16,384 to 32,768 tokens. Truncation stayed similar or rose (arm A 10.1% to 10.5%, arm C 10.9% to 13.0%) while the model simply wrote more (mean generation on A 8,793 to 10,483 tokens, on C 6,920 to 9,285, on D 918 to 1,809). The aggregate results moved within noise: the ablation’s own-complete collapse and the intervention’s re-check effect both held, and at 32k the neutral control landed on 0.022, exactly the untouched chain. So the findings are not an artifact of the original 16k cap. Two cap values cannot tell you the trajectories would never terminate, and the rerun holds the seed and every other experimental unit fixed, so this is a budget-sensitivity check rather than an independent replication.

The two runs cannot be paired sample by sample. I had intended to compare each sample to itself across the two budgets. That is not possible: both runs use seed 0 and identical sampling parameters, but the rerun also raised the model context length, and vLLM is not bitwise reproducible across that change. Divergence tracks generation length almost monotonically, from 97% of samples identical in the shortest arm down to 0.3% in the longest. So the samples that would answer the question, the ones truncated at 16k and given more room at 32k, are exactly the ones that diverge. The alignment audit therefore rules out a 16k-to-32k transition analysis, and I treat the 32k run only as an aggregate budget-sensitivity check, not as independent evidence.

The terminated-only column is descriptive, not a correction. The last column recomputes accuracy over only the samples that stopped on their own. For arms with real recovery it comes out slightly higher, and for the own-complete arm it is unchanged at 0.003. But termination is affected by the arm itself, since the scaffold and doubt arms change how much the model writes, so conditioning on it selects a different subset in each column. Treat it as a description of who terminated, not as an estimate of what the censored samples would have done. Because the two runs are not sample-aligned, no analysis available here can estimate that.

Red marks a cell above the project’s 2% truncation gate. "Truncated, no answer" counts samples that hit the cap and produced no extractable answer, which are the only ones scored as outright failures; the rest were capped but still gradable, though a longer continuation could have changed the answer they were graded on. The last column recomputes accuracy over just the samples that terminated on their own, with the share kept in brackets; it conditions on termination, which the arms themselves affect, so read it as descriptive.

The intervention rerun tells the same story and answers the one question the bound above could not. Its neutral “So,” control is what licenses reading the effect as being about doubt rather than about appending text, and that control had the highest truncated-and-unanswered rate of any cell. At the doubled budget it lands on pass@16 0.022, identical to the untouched chain, with one question gained and one lost. Meanwhile both doubt arms hold or improve (0.174 to 0.196 and 0.196 to 0.196), and their paired tests tighten to one-sided p = 0.004. Truncation again barely moves (13.5% to 13.6%), so the effect is not an artifact of the original cap in either direction.

One artifact worth separating out: the thinking model’s own-complete cell shows 0% truncation but an 8.3% no-answer rate. Those continuations stop on their own without a parseable answer, so that is a grading-coverage question, separate from censoring.

Code and artifacts. The injection and grading code, per-question aggregates, figure inputs, run summaries, and the tests behind every published number are available at github.com/AlvinZH04/Completeness-Cliff. The injected wrong-trace bank is released separately at huggingface.co/datasets/AZH04/wrong-reasoning-traces.

Open to discussion. Comments, questions, and pushback are all welcome. I’m reachable at bzhang90@jh.edu and always happy to discuss this work.

Cited as: Zhang, Alvin (2026). “The completeness cliff: language models escape wrong reasoning until it’s finished.” wrong_reason notes. alvinzh04.github.io/blog/completeness-cliff.html

@misc{zhang2026completenesscliff,
  title        = {The completeness cliff: language models escape wrong reasoning until it's finished},
  author       = {Zhang, Alvin},
  year         = {2026},
  howpublished = {\url{https://alvinzh04.github.io/blog/completeness-cliff.html}},
  note         = {wrong\_reason notes}
}

Models, data, and references

1. Qwen3-4B-Thinking-2507 and Qwen3-4B-Instruct-2507. Qwen Team, Qwen3 Technical Report, arXiv:2505.09388 (2025). Weights: Qwen/Qwen3-4B-Thinking-2507, Qwen/Qwen3-4B-Instruct-2507. Sampling settings taken from the model cards.
2. Gemma-4-E2B-it. Gemma Team, Google DeepMind, Gemma 4 Technical Report, arXiv:2607.02770 (2026). Weights: google/gemma-4-E2B-it.

1. AIME 2024, via Maxwell-Jia/AIME_2024, and AIME 2025, via MathArena/aime_2025 (MathArena, CC BY-NC-SA 4.0). The problems themselves are American Invitational Mathematics Examination competition problems, property of the Mathematical Association of America; only problem ids and gold answers are redistributed here.
2. Reasoning Gym (maze, mini-sudoku). Z. Stojanovski, O. Stanley, J. Sharratt, R. Jones, A. Adefioye, J. Kaddour, A. Koepf, Reasoning Gym: Reasoning Environments for Reinforcement Learning with Verifiable Rewards, arXiv:2505.24760, NeurIPS 2025 Spotlight. Code: open-thought/reasoning-gym.
3. Traces released from this study: AZH04/wrong-reasoning-traces.

1. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, I. Stoica, Efficient Memory Management for Large Language Model Serving with PagedAttention, arXiv:2309.06180, SOSP 2023. (vLLM.)
2. Math-Verify, huggingface/Math-Verify, used for answer equivalence on the math tasks.
3. M. Chen, J. Tworek, H. Jun, Q. Yuan, et al., Evaluating Large Language Models Trained on Code, arXiv:2107.03374 (2021). Source of the unbiased pass@k estimator used throughout.

1. S. Yang, S.-W. Lee, N. Kassner, D. Gottesman, S. Riedel, M. Geva, How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?, arXiv:2506.10979 (2025). The closest prior work: injects four categories of unhelpful thought and separates identification from recovery.
2. M. Ballon, B. Verbeken, V. Ginis, A. Algaba, Probing the Trajectories of Reasoning Traces in Large Language Models, arXiv:2601.23163 (2026). Truncate-and-reinject at token percentiles; anchoring grows with trace length.
3. S. Amjith, M. Dusad, N. Muramalla, S. Shah, Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking?, arXiv:2512.17079 (2025). Flawed prefixes as a training signal.
4. K. Tsui, Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models, arXiv:2507.02778 (2025).
5. T. Lanham, A. Chen, A. Radhakrishnan, B. Steiner, C. Denison, et al., Measuring Faithfulness in Chain-of-Thought Reasoning, arXiv:2307.13702 (2023). Source of the adding-mistakes and early-answering perturbations.
6. M. Turpin, J. Michael, E. Perez, S. R. Bowman, Language Models Don’t Always Say What They Think, arXiv:2305.04388 (2023).
7. Y. Chen, J. Benton, A. Radhakrishnan, et al., Reasoning Models Don’t Always Say What They Think, arXiv:2505.05410 (2025).
8. J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, D. Zhou, Large Language Models Cannot Self-Correct Reasoning Yet, arXiv:2310.01798 (2023).
9. M. Zhang, O. Press, W. Merrill, A. Liu, N. A. Smith, How Language Model Hallucinations Can Snowball, arXiv:2305.13534 (2023).
10. F. Shi, X. Chen, K. Misra, N. Scales, D. Dohan, E. H. Chi, N. Schärli, D. Zhou, Large Language Models Can Be Easily Distracted by Irrelevant Context, arXiv:2302.00093 (2023).
11. I. Mirzadeh, K. Alizadeh, H. Shahrokhi, O. Tuzel, S. Bengio, M. Farajtabar, GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models, arXiv:2410.05229 (2024).
12. M. Sharma, M. Tong, T. Korbak, D. Duvenaud, A. Askell, et al., Towards Understanding Sycophancy in Language Models, arXiv:2310.13548 (2023).
13. DeepSeek-AI, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, arXiv:2501.12948 (2025).
14. D. Jiang, A. Zhang, A. Wang, N. Andrews, D. Khashabi, Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback, arXiv:2506.11930, NeurIPS 2025.

A fuller annotated bibliography, including work not cited here, is in literature_review/.