Skip to main content
What you learn: install the package, set the two kinds of key, and set up a repo for a coding agent. Needs: Python 3.10 to 3.13. Takes: a minute.
Python 3.10 to 3.13.

Set up a repo for your coding agent

One command in the repository your agent works in. It writes a short block into AGENTS.md (Codex, Cursor and Copilot read it), points CLAUDE.md at it (Claude Code reads that), installs the tested skills under .claude/skills/, and runs the evals skill’s check.py so the agent sees the verdict line before it touches your code. After that, “use whileai to build me better evals for my agent” lands on the playbook with no URL to remember. Run it again after an upgrade; wai status says when the block is stale.

Extras

Keys

Bring your own keys. Your model keys never leave your machine. The quickstart needs no key at all. A While key adds hosting: we store your datasets, run the hosted judge, train and serve.
Both keys can come from env vars instead: WHILEAI_API_KEY, OPENAI_API_KEY. wai login saves the While key at ~/.whileai/credentials.json; WHILEAI_HOME=/some/fresh/dir isolates a second account, and wai status prints which key the SDK will use. The trial. wai signup --email you@example.com makes an account with no browser. A fresh key gets 25,000 input and 50,000 output tokens a day, about twelve hosted situations of a four-tool agent; a run past that stops with Hosted model daily quota exceeded. simulate(..., simulator=False) writes situations offline with no quota, and one sign-in at the While site (the link wai status prints) lifts the limit. Other backends: connect your agent.
Last modified on September 21, 2026