The scripts are in the repository, not in the installed package. Clone it,
then
cd recipes/papers/context-lm before running the commands below. Browse this recipe on GitHub.ContextFile(gate="complete")). A third arm runs the paper’s rule as written (gate="paper"), and the baseline runs neither (gate="off").
Recipe
- Task:
wai.methods.KVLog, a seeded key-value log (ContextBench’s KV Store, cut down [1]). It has 5 chunks of 8set key = valuelines over 8 keys, and the asked key is set at least twice. Each step the model seescontext.mdand one chunk; then the chunk is gone and it writes the whole newcontext.md. The last step shows onlycontext.mdand the question, and the model boxes a number. 512 train logs, 200 held out from a disjoint seed range. - Harness and credit:
wai.methods.ContextFile(...).trainer(GRPOTrainer). Every step of a trajectory gets reward minus its group’s mean (group of 8). The efficiency arms add0.25 xEq. 6 on the five edits, not on the answer step (the paper’s edit mask). Costc_icounts prefix-reuse tokens: each step pays for the prompt past what is still cached, plus its reply. - Base:
Qwen/Qwen2.5-1.5B-Instruct. LoRA r=32, 60 steps, 4 logs x 8 trajectories a step, lr 1e-4, on-policy, no KL. Seeds 17, 18, 19 and 20 on every arm, all run fresh. - Eval: 4 trajectories on each of the 200 held-out logs. pass@1 with a paired 95% interval (
wai.compare,train_runs=all four seeds), and tokens a trajectory, paired by log.
Run
Result
Training worked on all 12 runs. Stepwise GRPO in the file harness takes a 1.5B model from 0.07 to between 0.84 and 0.99 on held-out logs in 60 steps.
The paper’s Eq. 6 is the most accurate arm and spends 15% fewer tokens than plain GRPO. Most of that saving comes from seed 20, where the baseline let its file grow to 758 characters of prose. On the other three seeds the token deltas go both ways.
Every verdict is flat. The baseline’s own seeds span 0.91 to 0.98, which is wider than any gap between arms.
The recipe arm is this recipe’s own change, and it lost to the paper’s rule. It was built against a shortcut that one seed found in the first two-seed run: copying only the latest chunk, which answers about 70% of logs. That shortcut did not come back in any of the 12 runs here. The completeness check is also stricter than the task needs. Trained files often drop a key that was set once and never asked, and still answer right. So the gate paid few successes, and Eq. 6 mostly went unpaid.
wai.methods.ContextFile now defaults to gate="paper".
Checks
Every number in this section is read fromresults.json.
Climb
Learned
- A small model learns to keep its own context. Qwen2.5-1.5B goes from 0.07 to 0.95 on 40-line logs it never sees whole, on every seed, in under 40 GPU minutes. It learns a
key: valuetable rather than a copy of the log. - Use the paper’s Eq. 6: +0.02 pass@1 and 15% fewer tokens over plain GRPO at four seeds. The saving is uneven across seeds, because the file bloat it prevents is too.
- A guard built against one seed’s failure cost accuracy across four. Next: a harder log (more keys, longer chunks) where the file must be compressed to fit, which is where the paper’s 28K budget puts its pressure.
References
- Context Language Models. arXiv:2609.37725, 2026. Code: https://github.com/facebookresearch/context-language-models
- Shao, Z. et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300, 2024.
- Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter Evaluation.
- Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML 2023. arXiv:2210.10760.