Matrix-CODI Reasoning Is Rank-Indifferent: Flat Rank-k Ablation Curves
ORIGINAL / The Gradient Does Not See Rank: Rank-Indifference in Matrix-CODI on ProsQA
Ablation studies on matrix-valued chain-of-thought models reveal that rank truncation of latent matrices has negligible effect on performance across ProsQA, challenging the hypothesis that parallel reasoning paths are encoded in matrix rank.
01 ABSTRACT
The authors test whether performance of Matrix-CODI models correlates with the effective rank of latent matrix Z. Across multiple training runs and readout variants, rank-k truncated projections yield flat accuracy curves within 0.6 pp. Linear probing underperforms baseline hidden states. A control experiment suggests the flat curve may reflect position-irrelevance rather than rank-insensitivity.
02 KEY FINDINGS
- Rank-k ablation curves are flat within 0.6 pp accuracy changes
- Results are similar across training settings and readout variants
- Linear probe on Z underperforms pretrained hidden states
- Control experiment suggests conflation with position-irrelevance
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.LG