Skip to main content
The scripts are in the repository, not in the installed package. Clone it, then cd recipes/papers before running the commands below. Browse this recipe on GitHub.
One directory per paper. A recipe here is a recent post-training paper’s idea cut down to a run that fits in under an hour on one GPU, with the number it moved and the number it did not. The building blocks are the step recipes next door (../04-train/grpo, ../04-train/dpo, ../04-train/text-to-sql, ../04-train/hosted-loop); a paper recipe copies one of them and changes one thing. Every recipe answers the same five questions in the same order: which paper, what it claims, the steps, one command, what happened. The table is generated: python recipes/papers/check.py --write reads every results.json. Do not edit it by hand. One paper here is a loop, not a trained arm: meta-harness (Lee et al. 2026, arXiv:2603.28052) searches over harness code, scores every candidate on one frozen task set, and gates the pick on held-out tasks and held-out models. It is in the step-recipe shape (run.py --dry-run, smoke.sh, a flags table), runs offline with scripted candidates, and check.py leaves it out of the table above because it has no results.json: the live run is the replication and no number from it is claimed until one is measured. harness-and-weights (Luo et al. 2026, arXiv:2607.03935; Hebbar et al. 2026, arXiv:2605.27276; Karten et al. 2026, arXiv:2608.23552) puts that loop and a GRPO arm under one optimizer: four arms (neither, harness, weights, both) on one holdout of seeded quant coding tasks. It is in the same step-recipe shape (recipe.py --dry-run, smoke.sh, a flags table) and writes a results.json only when the full live run has been measured; its README says what ran.

​
Run one

No smoke.sh in this directory: recipe.py is one file and imports modal at the top, which CI does not install, so --selftest is the offline check and you run it yourself.

​
The contract

  • README.md in the shape of _template/README.md: Paper, Claim, The change, numbered steps, one command, the Result table, the Climb table, three Learned bullets, the Verified line, the References list.
  • recipe.py: one file. Data, then train, then eval, then results.json. Two arms on the same holdout: the baseline and the paper’s change. Paired delta with a 95% interval (wai.compare).
  • results.json: the numbers the table above reads. Shape in _template/results.json.
  • Default run: under 60 GPU minutes, under $10. Bigger runs behind a flag.
  • Public data or a seeded environment that lives in the recipe directory. No customer data.
  • A flat result is a result. Say so in the table.
  • One training seed per arm is never “moved”: the verdict is “unresolved” until both arms have MIN_TRAIN_SEEDS (2) independent seeds, and the README’s result line says in one sentence what would resolve it. results.json records the seeds per arm in checks.train_seeds.
  • python recipes/papers/check.py --write passes (tests/recipes/test_papers.py runs it in CI).
  • post.md: the result as a post, once the recipe is verified. Under 280 characters, plain words, the metric with its interval, the arXiv link and the recipe link, nothing invented and nothing rounded. A flat result is posted as flat. Replicated papers are how we market (CONSTITUTION.md, belief 2); the post is the last artifact of a recipe, not a separate job.

​
The science bar

Every recipe is held to the same science bar. The README names the source each check rests on, and the ## Checks table is run, not ticked:

​
Maintenance

A daily agent re-runs the recipe with the oldest verified date, refreshes its numbers, fixes what broke, and adds one new recipe from recent post-training research. The default pick is a paper from the last 60 days; an older paper is allowed when the PR says what it is the baseline for (SimpleRL-Zoo, March 2025, is the zero-RL baseline). The table’s Paper column dates every one. Everything arrives as a pull request. One comment per run on the issue titled “Recipe log”. Several agents can work at once: each recipe is its own directory and the table is generated, so two new recipes never touch the same line.

​
References

  1. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter Evaluation.
  2. Miller, E. Adding Error Bars to Evals. arXiv:2411.00640, 2024.
  3. Lambert, N. et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124, 2024.
  4. Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML 2023. arXiv:2210.10760.

adaptive-clip

Paper: Group Adaptive Clipping Policy Optimization, Sheng Jia et al., arXiv:2609.00444, August 2026.

bpco-bounded-critic

Paper: Best Practice Critic Optimization, Penghui Qi, Xiangxin Zhou, Wee Sun Lee (NUS / Tencent Hunyuan), arXiv:2608.23566, August 2026.

endpoint-sft

Paper: Revisiting Complete Reasoning Traces for Post-Training, Jaehui Hwang et al., arXiv:2609.07103, September 2026.

filter-metric

Paper: The Filter Metric is Safety-Critical: Phantom Advantages in Group-Relative RL under Shaped Rewards, Juntao Yu, arXiv:2609.13866, September 2026.

flash-reinforce

Paper: FlashREINFORCE: Critic-Free Single-Rollout Asynchronous RL for Agentic Language Models, Yifan Hu et al., NVIDIA, September 2026 (no arXiv id yet).

gmts-token-select

Paper: GMTS: Gradient Magnitude-based Token Selection Improves RLVR Training, Outongyi Lv et al., arXiv:2608.30632, August 2026.

harness-and-weights

Paper: Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions (HASE), Haochen Luo et al., arXiv:2607.03935, July 2026.

meta-harness

An outer loop over harness code.

sao-single-rollout

Paper: Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning, Hou, Li, Tang, Dong (Tsinghua), arXiv:2607.07508, July 2026.

zero-rl-format-reward

Paper: SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild, Weihao Zeng et al., arXiv:2503.18892, March 2025.
Last modified on September 22, 2026