OLMo-3 reasoning redundancy · RQ1 and RQ3

• What is fitted? One row represents one problem and its independent natural reasoning trace from a particular model. The RQ3 coefficient table fits each outcome to the same four predictors: Ahat, op, irrelevant-node count, and raw mean NLL. It describes conditional associations, not causal effects.
• Predictor 1 — Ahat: observed benefit of the reasoning policy. For each eligible problem, generate 8 DIRECT responses and 8 independent THINK_A responses—16 responses, not 8 total. Score each final answer against the problem's gold answer. Ahat = (THINK_A correct count − DIRECT correct count) / 8. For example, 6/8 versus 2/8 gives Ahat = 0.5, a 50-percentage-point accuracy gain. This estimates the expected policy contrast A0; it is not directly measured subjective benefit. It has 17 possible values from −1 to 1 in increments of 0.125 and remains noisy. THINK_A and DIRECT differ in their prompting/output policies, so this does not isolate the causal effect of extra tokens.
• Predictor 2 — op: required-operation count. Taken from the frozen GSM-DC dataset's op field, rather than inferred from the model's generated trace. It is a structural computational-difficulty proxy: the task's required operations, not the number of steps the model actually writes. A larger value need not imply a proportionally harder problem for every model.
• Predictor 3 — irrelevant-node count: distractor load. Taken from the frozen dataset's irrelevant_nodes_count field: the annotated number of irrelevant nodes in the problem structure. It measures input distraction, not mistakes or redundant steps in the generated solution. One node is not necessarily one sentence or one extra reasoning operation; it stays separate from op in the RQ3 coefficient table.
• Predictor 4 — raw mean NLL: input-text predictability. Each target model scores its own original problem-only tokens using the Hugging Face reference teacher-forcing scorer, with the pinned tokenizer and BOS-shift scoring mask; gold answers and generated reasoning are not scored. For token t, compute −ln p(token t | BOS, preceding problem tokens), then average over that problem's scored tokens. This is a within-problem token average, not an average across the analysis rows, and uses neither residualization nor ranks. Higher NLL means less predictable text; it is a familiarity proxy, not direct training-exposure measurement. Perplexity = exp(mean NLL).
• Shared measurement — N and k*. Generate one separate, independent THINK_RHO response for each of the 600 original problems. It supplies the natural reasoning trace and the initial correctness/step-count screen; its correctness never enters Ahat. Segment its reasoning with the frozen HMTE segmenter; N is the number of reasoning steps, not op or a token count. The fixed external judge, gpt-4o-mini-2024-07-18, sees the problem plus different reasoning prefixes and returns a numerical answer. Test all positive prefixes if N ≤ 20, otherwise the frozen 20-point grid. k* is the first tested positive prefix length that yields the correct answer; later prefixes need not all remain correct. An extra k=0 call is diagnostic only and never sets k*.
• Outcome 1 — relative prefix redundancy (rho). Computed from that independent trace as rho = (N − k*) / N = 1 − k*/N. It is the fraction of steps after the first successful tested prefix. Example: N = 100 and k* = 20 give rho = 0.8. This is HMTE cross-judge prefix redundancy.
• Outcome 2 — absolute redundant steps. Using the same N and k*, compute N − k*. The same example gives 80 steps. We fit this raw count—not log1p, not tokens—and do not change the predictors or samples. Absolute amounts can increase simply because traces are longer; two traces may share rho = 0.8 but have 80 versus 8 redundant steps. It is an absolute prefix-tail measure, not an additional deletion experiment.
• Cohorts and selection: all three models start from the same 600 problems, but only each model's correct natural traces with at least two steps receive the 8+8 gain draws. In order OLMo-3 32B Think / OLMo-3 7B Think / R1-Distill-Qwen-7B, eligible reference sets contain 273 / 247 / 135 problems; complete-case analysis sets contain 273 / 245 / 129. The 0 / 2 / 6 traces without a critical point are excluded from both outcome fits, never assigned zero. Within a model, the two outcomes use exactly the same complete-case rows.
• Units and joint preprocessing: Table 1 separately z-scores each of its four predictors within that model's complete-case analysis set: z(x) = (x − mean(x)) / sample SD(x). The outcome remains in its original units, so beta is a change in rho or in steps per one predictor SD, conditional on the other three. Standardization does not make counts continuous, make distributions normal, or remove Ahat sampling noise. The additive model assumes linear conditional associations.
• Inference and prediction: joint OLS uses HC3 heteroskedasticity-robust standard errors. Both outcomes reuse the same model-specific groups and frozen five-fold CV splits, with scaling learned only from training rows and a training-mean baseline. Holm4 adjusts the current cell's four joint coefficient tests; supplementary Holm24 covers the 24 coefficient tests across all six cells. HC3 does not correct selection, predictor measurement error, or nonlinearity. Different model cohorts mean a significant result in one model and a nonsignificant result in another is not itself a test of a model difference.
• B and the three associations (auxiliary, not a fifth joint predictor): B and its scaling are defined in each view above; each is correlated with the outcome alone (Pearson r, Holm3 within the cell; supplementary Holm18 within the view). The answer ≠ 0 view is a separate Holm24/Holm18 family from “All problems”.

These are exploratory, not causal or independently confirmatory. Absolute tails can reflect longer total traces: no log transform, normalization by N, or adjustment for N was added. Inverse perplexity is text predictability, not a direct measure of training familiarity.

HMTE metric source · GSMDC code · OLMo-3 32B · OLMo-3 7B · R1 Distill 7B