> ## Documentation Index
> Fetch the complete documentation index at: https://docs.while.ai/llms.txt
> Use this file to discover all available pages before exploring further.
> ## Agent Instructions
> Install with `uv add whileai`; import as `import whileai as wai`.
> Run the offline path first (`simulator=False`, `wai.seeded_agent`, a callable judge); no key is needed for it.
> Report every pass rate with its interval and n, as `scored.pass_at` prints it.
# Quickstart
> Sixty seconds, offline: simulate a stand-in agent, grade it with a one-line judge, read pass@1 with an interval, keep the rows worth training on.
**What you learn:** simulate a stand-in agent, grade it with a one-line judge, read pass\@1 with an interval, keep the rows worth training on. **Needs:** nothing. **Takes:** sixty seconds.
Running your own model? [Your model and your key](/get-started/your-model-and-key)
is the two-line version: the model as a string, the key in the
provider's usual environment variable.
No key, no network. `seeded_agent` is a stand-in agent. It answers
honestly most of the time and, on a labeled fraction of rollouts, does one
thing wrong on purpose: hedges, flatters, or claims success after a tool
failed. Each row records what it did in `seeded`, so you can check that
your judge catches exactly those rows before you trust it on real ones.
A tool is a typed function. The signature is the schema, the
docstring is the description. One is enough.
```python theme={"theme":"vitesse-dark"}
import whileai as wai
@wai.tool
def get_order(order_id: str) -> dict:
"""Look up an order by id."""
...
```
`simulator=False` writes the customers from templates, so no model is
called. `mode="rl"` with `repeats=4` plays every ask four times, which
is what pass\@k needs.
```python theme={"theme":"vitesse-dark"}
data = wai.simulate(
wai.seeded_agent([get_order]),
tools=[get_order],
system_prompt="Help customers with orders.",
simulator=False, # no model
mode="rl",
repeats=4,
repeat_policy="fixed",
budget=64,
)
```
Any callable that takes a row and returns a reward is a judge. This
one uses the label the stand-in agent left behind.
```python theme={"theme":"vitesse-dark"}
scored = data.grade(judge=lambda row: {"reward": int(not row["seeded"])})
print(scored.pass_at)
```
```text theme={"theme":"vitesse-dark"}
pass@1 0.67 [0.55..0.78] | pass^4 (pass_pow_k) 0.19 [0.00..0.38] | pass@4 1.00 [1.00..1.00] | headroom 0.33 (16 groups, k=4)
```
`select` runs the gates: replies that quote the answer key the grader
was given, the 20 to 80% difficulty band, unanimous groups,
duplicates, truncation, a scan for what the reward is really tracking.
Print it and it says what each gate dropped and why. Four of the
stand-in's planted mistakes recite the answer key, and they never
reach the training file.
```python theme={"theme":"vitesse-dark"}
rows = scored.select(mode="rl")
print(rows)
rows.export("train.jsonl")
```
```text theme={"theme":"vitesse-dark"}
rl selection: kept 27 of 64 rows
band 20%..80% pass rate: 0 asks dropped (0 too easy, 0 too hard)
unanimous groups dropped: 6; duplicates dropped: 27; truncated drop: 0
privileged leaks dropped: 4
groups kept: 10
hack scan: train
warning: 27 duplicate rollout(s) within 13 ask(s) dropped
warning: reward punishes reply length (corr -0.74); check the judge before training
warning: reward punishes boilerplate (corr -0.32); check the judge before training
warning: reward punishes hedging (corr -0.37); check the judge before training
warning: 3 rollouts per ask at the median; the floor is coarse below 4, re-scan at repeats>=8 before acting on a close call
warning: Difficulty was measured from 4 rollouts per task, so a task's band assignment can be off by about ±0.3. Use repeats=16 for a firmer band (the count the 20-80 band is measured from, Lambert 2025, chapter Reasoning).
```
The warnings are part of the answer. The judge here reads the label
the stand-in left behind, and the planted mistakes add words, so
shorter replies really are better; on a real judge that line means
check whether it grades the job or the word count.
pass\@1 is the pass rate over tasks with a bootstrap interval. pass^4 is
how often all four rollouts of a task pass. Headroom is pass\@4 minus
pass\@1, the gap an RL update could close.
## Your model, your key
Every role in a run is a model behind an endpoint. Say which with a
backend object; its repr tells you where the call goes and which key it
uses. Set it once for the process, or pass it on the call.
```python theme={"theme":"vitesse-dark"}
wai.configure(
agent=wai.OpenAI("gpt-4.1-mini", api_key="sk-..."), # or OPENAI_API_KEY
judge=wai.Anthropic("claude-haiku-4-5"), # ANTHROPIC_API_KEY
)
print(wai.settings)
data = wai.simulate(tools=[get_order], system_prompt="Help customers with orders.", mode="rl", repeats=8)
scored = data.grade(wai.Judge(rubric="Look up the order before answering."))
```
```text theme={"theme":"vitesse-dark"}
Settings(agent=openai:gpt-4.1-mini, judge=anthropic:claude-haiku-4-5, simulator=default (While hosted), api_key=unset, keys=openai)
```
`wai.Endpoint("Qwen/Qwen3-4B", url="http://localhost:8000/v1")` is any
OpenAI-compatible server you run; `wai.Ollama("llama3")` needs no key.
With nothing configured, every role uses the model While hosts, on the
key from `wai login`. The full table is on
[Connect your agent](/get-started/connect-your-agent).
## Next
A callable, an endpoint, a backend object, or the hosted model. Plus
traces to aim the run.
A pass rate with an interval, a table of where the agent fails, and a
CI check that turns red when it gets worse.
How the engine picks situations, plays the customer, and breaks the
tools on purpose.
simulate, grade, trust the judge, select, push: the run in order.