> ## Documentation Index
> Fetch the complete documentation index at: https://docs.while.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Install with `uv add whileai`; import as `import whileai as wai`.
> Run the offline path first (`simulator=False`, `wai.seeded_agent`, a callable judge); no key is needed for it.
> Report every pass rate with its interval and n, as `scored.pass_at` prints it.

# Team talk: train a model to reason as a team talking to itself

> Paper: Self-Organizing Agent Teams Learn to Reason Together, Aneesh Pappu et al., arXiv:2609.22682, September 2026.

<Note>The scripts are in the repository, not in the installed package. Clone it,
then `cd recipes/papers/team-talk` before running the commands below. [Browse this recipe on GitHub](https://github.com/whilehq/whileai-sdk/tree/main/recipes/papers/team-talk).</Note>

**Paper:** Self-Organizing Agent Teams Learn to Reason Together, Aneesh Pappu et al., arXiv:2609.22682, September 2026. [https://arxiv.org/abs/2609.22682](https://arxiv.org/abs/2609.22682)
**Book:** RL on a verifiable reward pays only for the answer, so any structure left in the trace after training is structure that helped get the answer right \[1]\[2].
**Claim:** a team that proposes, challenges and settles in conversation reaches answers none of its members reach alone (Pappu et al., no training); reasoning traces already look like that conversation, and RL on accuracy makes a base model talk more (Kim et al. \[3]). Put the team in one model's trace and train it there.
**The change:** the prompt the recipe arm trains under. The baseline is asked to solve the problem. The recipe is asked to solve it as Solver, Checker and Lead taking turns in one reply. The reward is the same answer check in both arms; the talk is never paid for.

## Recipe

1. Base: `Qwen/Qwen2.5-1.5B-Instruct`. Data: GSM8K, 512 train prompts from the train split, 120 held out from the test split.
2. Reward, both arms: the binary outcome, `MathEqual` against the GSM8K gold number. A program, not a judge.
3. Baseline: TRL GRPO + LoRA r=32, 40 steps, 12 prompts x 4 rollouts per step, lr 1e-4, on-policy, no KL, reward minus the group mean with no std division. Plain prompt.
4. Recipe: the same trainer and settings under the team prompt (three speakers, one turn per line, Lead writes `\boxed{}`). Both arms get the same 512-token completion cap, so a conversation has to fit where a monologue does.
5. Eval: pass\@1 on the 120 held-out tasks, 4 samples per task, two training seeds per arm (17 and 18), paired delta with a 95% interval (`wai.compare`, `train_runs=`). Twice: each arm under its own prompt, then both under the plain prompt. The second is the memory test: with the team prompt gone, whatever talk and accuracy remain is in the weights.

Talk is counted, not judged: a reply talks when it has three or more turns, two or more speakers, and a Checker turn (`talks()` in `recipe.py`).

## Run

```bash theme={"theme":"vitesse-dark"}
python recipe.py --selftest                 # the talk counter and the grader, offline, no GPU and no key
python recipe.py                            # both arms, two seeds each, four L40S in parallel
python recipe.py --arm recipe --steps 80    # one arm, longer
```

## Result

| Arm                                                     | pass\@1 | 95% CI        | pass\@k | Talk rate | Steps | GPU min |
| ------------------------------------------------------- | ------- | ------------- | ------- | --------- | ----- | ------- |
| Base, plain prompt                                      | 0.58    | \[0.52, 0.65] | 0.84    | 0.00      | 0     | 0       |
| Base, team prompt (no training)                         | 0.45    | \[0.39, 0.52] | 0.75    | 0.21      | 0     | 0       |
| Baseline, GRPO plain prompt (seed 17; seed 18: 0.67)    | 0.65    | \[0.58, 0.72] | 0.83    | 0.00      | 40    | 21.2    |
| Recipe, GRPO team prompt (seed 17; seed 18: 0.67)       | 0.65    | \[0.58, 0.73] | 0.79    | 0.00      | 40    | 23.9    |
| Recipe, evaluated with the plain prompt (seed 18: 0.64) | 0.66    | \[0.59, 0.74] | 0.86    | 0.00      | 40    |         |

Recipe vs baseline: **+0.000 \[-0.044, +0.046]** across both training seeds. Verdict: **flat**. With the team prompt taken away: **+0.015 \[-0.025, +0.052]**, flat.

The talk did not survive. Asked to talk without training, the base model talked in one reply in five and lost 13 points doing it. After 40 steps of GRPO under the same prompt, it talked in none: 0 turns in 480 held-out replies, and 0 in the last training batch on both seeds. Read by eye, the trained replies are the plain step-by-step solution the baseline writes, with the team prompt still in front of them. The reward paid only for the answer, the talk cost answers, and RL took it out.

## Checks

Nothing in this table is ticked by hand: every cell is written by `recipe.py` into `results.json`.

| Check                                                            | Source                                       | Result                                                                                                                         |
| ---------------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| Eval noise: the base evaluated 3 times, `eval_variance` run\_std | \[6]                                         | run\_std 0.006 from 3 re-runs (0.58, 0.57, 0.57); the two training seeds differ by 0.02 in both arms, more than the eval noise |
| Holdout is clean: `decontaminate(train, against=holdout)`        | \[6]                                         | 0 of 512 train rows dropped                                                                                                    |
| Reward is a program, not a judge                                 | \[6]                                         | `MathEqual` (Math-Verify) against the public GSM8K gold number; talk is never rewarded                                         |
| Proxy vs target: `wai.compare(proxy=)`                           | \[7]                                         | `proxy=None`: the training reward is the target; over\_optimized false                                                         |
| Length: mean completion length before -> after, per arm          | \[7]                                         | 865 chars base -> 1025 baseline, 996 recipe. Same 512-token cap in both arms                                                   |
| Hack scan on the last training batch: `hack_scan`                | \[7]                                         | top feature `contains:+ AND contains:5`, arithmetic surface; nothing endorsed                                                  |
| Pinned: seed, torch, transformers, trl, peft                     | [the contract](/recipes/papers#the-contract) | training seeds 17 and 18, `--seed 0` for the data; torch 2.7.1, transformers 4.54.0, trl 0.19.1, peft 0.16.0                   |

## Climb

| Round | What changed                                          | pass\@1                                  | vs previous                                            |
| ----- | ----------------------------------------------------- | ---------------------------------------- | ------------------------------------------------------ |
| 1     | team prompt, correctness-only reward, 2 seeds per arm | baseline 0.65 / 0.67, recipe 0.65 / 0.67 | +0.000 \[-0.044, +0.046], flat; talk rate 0.21 -> 0.00 |

## Learned

* A prompt cannot keep a 1.5B model talking under an answer-only reward. The talk cost 13 points before training, so GRPO removed it within 40 steps and the arm converged on the baseline's monologue. Kim et al. see talk grow under the same reward, on models whose talk already pays; this one's did not.
* Nothing was lost either: the team arm ends where GRPO ends, with and without its prompt. The prompt was a detour, not a handicap.
* Next: make the talk pay by giving it a listener. Separate copies post to a shared message board and read each other's notes before answering, and the reward counts the team's answer, so a note matters only if another copy uses it (Park et al. \[5], Pappu et al. \[4]). Or a model large enough that its own talk already helps.

Verified 2026-09-23, whileai 0.125, TRL 0.19.1 + PEFT 0.16.0 on torch 2.7.1. 134.1 GPU minutes over four containers, \$4.47 on L40S. Run page: [https://while.ai/platform/training/run\_18e22bfb2035a68d](https://while.ai/platform/training/run_18e22bfb2035a68d). Experiment: [https://while.ai/platform/experiments/team-talk](https://while.ai/platform/experiments/team-talk)

## References

1. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter *Reasoning*.
2. Shao, Z. et al. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300, 2024.
3. Kim, J., Lai, S., Scherrer, N., Agüera y Arcas, B., Evans, J. Reasoning Models Generate Societies of Thought. arXiv:2601.10825, 2026.
4. Pappu, A. et al. Self-Organizing Agent Teams Learn to Reason Together. arXiv:2609.22682, 2026.
5. Park, C. et al. MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learning. arXiv:2502.18439, 2025.
6. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter *Evaluation*.
7. Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML 2023. arXiv:2210.10760.