The scripts live in the repository, not in the installed package. Clone it
first:
git clone https://github.com/whilehq/whileai-sdk.git, then cd to the recipe
directory named at the top of its page.Simulate
Make rollouts: an agent, situations, a reward that is a program.bring-your-own-agent
the
agent(message) -> {steps, final_text} contract, what a run says when the agent raises, why an evaluate() score must not become the rewardswarm-rescue
on the tasks where all 8 rollouts fail, whether a particle swarm of rollouts that share their best attempts finds a passing answer that resampling does not; rescue rate paired by task, four arms at one budget
verifiers
rewards that are programs:
MathEqual, All (answer and format), CodeExec against hidden tests, JSONSchema, each honoring the judge contractMeasure
Say what the numbers mean: pass@k, headroom, reward hacking, safety.character-to-the-wall
whether a persona holds when two of its own values collide: situations built so no reply can honor both principles, the spec’s authority ordering as the answer key, held_wall and kept_lower graded apart to tell caving from rigidity, a judge checked against the set’s own labels, before/after on held-out walls
compare-judges
six judges on the same 300 labeled rollouts, one ranked table: agreement with its interval, kappa, leak rate, unsure and unjudged counts, seconds per row; Jev, the hosted judge, Claude, and the policy judging itself
eval-your-agent
evals for the agent you already have: wrap it, write the policy as a judge, pass@1 with an interval per policy branch, the coverage warnings that catch a hollow run, a CI gate
is-your-eval-any-good
whether a number your eval produced means anything: ceiling, headroom, criteria that cannot fail, self-noise, the judge, contamination, and the three checks that void a base-vs-tuned comparison outright
pass-at-k
pass@1 with its interval, pass^k, pass@k, the per-ask histogram the mean hides, and headroom = what a grouped update can learn
public-benchmark
a public benchmark (200 GSM8K test questions) into the measurement:
wai.rows with MathEqual as the reward, pass@1 with its interval, the eval’s own noise over three passes, holdout_size, select dropping the groups that carry no gradient, and a compare reportreward-hacking
reward hacking caught before, during and after training: the within-ask scan, the judge probes, the trajectory flags, the proxy-vs-target verdict
safety-evals
a safety suite for a tool-using agent: prompt injection, exfiltration, secret leakage, unauthorized writes, benign controls; four trajectory markers as the judge, pass^k per attack class, a before/after that fails the fix which got safe by refusing
safety-evals-marketplace
the same eval where the untrusted text is user-generated content and the private data is per tenant;
live.py runs it on a local model through OllamaSelect
Turn graded rows into training data: SFT rows, pairs, RL groups.character
a constitution to traits, graded replies per trait, a judge checked against the spec’s own labels, length-matched pairs and masked SFT rows, before/after on an adversarial holdout
prime-intellect-rl
simulate(mode="rl") for uniform groups, the gradient gate (diagnose.py) that catches a reward the policy can game before you train, prompts in the verifiers shapeschema
one row file projected into eval, SFT, preference, GRPO prompts, OPSD and OPD targets; the
Task/Rollout/Judgment/Marker split that makes that possibleTrain
Train it, hosted or on your own GPU, and watch the run page.dpo
on-policy pairs from
build_preference_pairs, TRL DPOTrainer, the reward margin on the run page, iterated rounds with --from-run, constructed negativesfireworks
export SFT rows and DPO pairs in Fireworks’ shapes (
format="fireworks"), the firectl commands that train on Fireworks GPUs and serve the result, the paired before/after through wai.Fireworks and wai.comparegrpo
TRL
GRPOTrainer with LoRA on a verifiable rule, HackMonitor and reward/KL on the run page, paired pass@1 before/after with per-category deltas, loss variants and --balance as flagshosted-loop
push graded rows,
wai.train SFT on Qwen3-4B, wai.serve the adapter, one chat completion from the endpointidentity
a leak-free SFT set that teaches a name and maker, with Modal scripts for the LoRA and for the identity/leak eval
prime-rl
GRPO, OPSD and OPD on one taskset on prime-rl from
wai.prime_rl_config, a launcher over Prime Intellect’s published image, per-prompt held-out deltas with intervals from wai.compare; run e2e1: OPD matched GRPO with no reward, OPSD moved a fifth as farreport-run
the typed objects the platform tracks (a tracked agent with its harness, behaviors, runs, live traffic), why a harness is versioned by its fingerprint, and why a version is scored on every behavior
resist-planted-instruction
a behaviour rubric decided by code, the criterion promoted into the reward on probe evidence, rejection sampling from the base itself, a pre-registered random-selection control, three arms from one vLLM process with attack and clean halves apart
sft
LoRA SFT with TRL
SFTTrainer on the rows lesson 7 exports (select(mode="sft").export), three base passes for the noise floor, one trained pass, the paired wai.compare(run_std=) on the held-out set; the step the course used to skiptext-to-sql
hill-climb a model on a schema with a verifier as the reward: a seeded Postgres, 741 execution-checked tasks,
SQLExec, benchmarks through simulate(tasks=), self-distillation, GRPO rounds on Modal with vLLM generation and Postgres in the container, every round measured on the same holdoutExport
Ship the data and the adapter.bedrock-import
merge a LoRA adapter on Modal, import the weights into your AWS account with Bedrock Custom Model Import, measure the served model on the same held-out tasks as the vLLM run; a paired interval says the weights survived the move
hugging-face
rows to a Hub dataset repo (one split per purpose, commit tagged by dataset id), any Hub split onto the account with a profile, a run’s adapter to a model repo
Papers
One directory per paper.adaptive-clip
Paper: Group Adaptive Clipping Policy Optimization, Sheng Jia et al., arXiv:2609.00444, August 2026.
bpco-bounded-critic
Paper: Best Practice Critic Optimization, Penghui Qi, Xiangxin Zhou, Wee Sun Lee (NUS / Tencent Hunyuan), arXiv:2608.23566, August 2026.
endpoint-sft
Paper: Revisiting Complete Reasoning Traces for Post-Training, Jaehui Hwang et al., arXiv:2609.07103, September 2026.
filter-metric
Paper: The Filter Metric is Safety-Critical: Phantom Advantages in Group-Relative RL under Shaped Rewards, Juntao Yu, arXiv:2609.13866, September 2026.
flash-reinforce
Paper: FlashREINFORCE: Critic-Free Single-Rollout Asynchronous RL for Agentic Language Models, Yifan Hu et al., NVIDIA, September 2026 (no arXiv id yet).
gmts-token-select
Paper: GMTS: Gradient Magnitude-based Token Selection Improves RLVR Training, Outongyi Lv et al., arXiv:2608.30632, August 2026.
harness-and-weights
Paper: Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions (HASE), Haochen Luo et al., arXiv:2607.03935, July 2026.
meta-harness
An outer loop over harness code.
sao-single-rollout
Paper: Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning, Hou, Li, Tang, Dong (Tsinghua), arXiv:2607.07508, July 2026.
zero-rl-format-reward
Paper: SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild, Weihao Zeng et al., arXiv:2503.18892, March 2025.
Community
Recipes contributed by people trying the SDK on their own problems, published so the next person starts from someone’s real run instead of from zero.airline-voice-concise-under-probe-outcome-filter
Behaviour: the agent runs long when the caller’s own text carries a planted instruction that buys length (“explain your reasoning step by step”, “take as long as you need”).
can-the-judge-be-trusted
Grade the same rollouts two ways — with rubric_judge() and with a deterministic verifier — and measure where the LLM judge disagrees with the rule.
can-the-markers-be-trusted
Check every behavioral marker against ground truth. seeded_agent plants six behaviors and writes what it planted on each row, so a marker’s recall and precision are computable rather than assumed.
force-the-branch
Seat: a post-training engineer at a startup that ships one production agent, trying to find out whether a small open model can take over the boring half of it.
grpo-group-size-at-fixed-budget
Sweep GRPO’s group size (num_generations) across G ∈ {2, 4, 8} while every arm spends the same 768 rollouts, and ask whether the knob moves held-out GSM8K accuracy past the noise floor.
hosted-grpo-vs-sft
Seat: an open-model fine-tuner who lives in TRL and Hugging Face, here for clean exports, a LoRA run, a hosted result and a cost line.
how-much-contamination-survives
Measure what fraction of real held-out contamination wai.decontaminate() actually removes, using human-labelled paraphrase pairs as ground truth instead of a planted copy.
identity-spec-no-unasked-maker-aas
Seat: a post-training engineer on a team that ships one production agent with a written identity and spec, trying to teach it who it is without it mentioning who it is for the rest of the day.
process-vs-outcome-reward
arXiv:2607.02869 (Palandye et al., July 2026) trains Qwen2.5-0.5B with GRPO on GSM8K under five reward regimes and reports that scoring the steps beats scoring the answer: 63.73% test accuracy for process-only against 5.
same-entrypoint-before-after
Seat: a post-training engineer at a startup that ships one production agent, trying to find out whether a small open model can take over the boring half of it.
the-step-the-course-skips
The eight-lesson course for people who have never trained a model stops one line short of the training: # 2.
what-trl-does-with-the-loss-mask
wai.export(…, format=“trl”) writes a loss_mask on every row and reports mask_mode: “assistant”.
which-half-can-a-small-model-take
I have one production agent and a cost line to defend.
who-protects-the-holdout
decontaminate() applies four rules in order.
Write one
Copyrecipes/_template into the step it belongs
to, replace the parts in angle brackets, and open a pull request. The contract is in
CONTRIBUTING.md.