What you learn: install the package, set the two kinds of key, and set up a repo for a coding agent. Needs: Python 3.10 to 3.13. Takes: a minute.
Python 3.10 to 3.13.
Set up a repo for your coding agent
One command in the repository your agent works in. It writes a short block
into AGENTS.md (Codex, Cursor and Copilot read it), points CLAUDE.md at
it (Claude Code reads that), installs the tested skills under
.claude/skills/ (strengthen-your-evals, manage-experiments,
whileai-simulations, audit-your-judge, pick-a-method), and runs the evals skill’s check.py so the agent sees
the verdict line before it touches your code. After that, “use whileai to
build me better evals for my agent” lands on the playbook with no URL to
remember. Run it again after an upgrade; wai status says when the
block is stale.
Keys
Bring your own keys. Your model keys never leave your machine.
The quickstart needs no key at all.
A While key adds hosting: we store your datasets, run the hosted judge, train and serve.
Both keys can come from env vars instead: WHILEAI_API_KEY, OPENAI_API_KEY.
wai login saves the While key at ~/.whileai/credentials.json;
WHILEAI_HOME=/some/fresh/dir isolates a second account, and
wai status prints which key the SDK will use.
The trial. wai signup --email you@example.com makes an account
with no browser. A fresh key gets 25,000 input and 50,000 output tokens a
day, about twelve hosted situations of a four-tool agent; a run past that
stops with Hosted model daily quota exceeded. simulate(..., simulator=False) writes situations offline with no quota, and one sign-in
at the While site (the link wai status prints) lifts the limit.
Other backends: connect your agent. Last modified on September 24, 2026