Command line

What you learn: sign in, check which key is in use, write an eval harness into a project, read what the platform tracks. Needs: uv add whileai; a key for the platform commands. Takes: two minutes. uv add whileai puts a wai command on your path. whileai runs the same command; wai is three letters and one token, which is why the docs, the skills and the block in AGENTS.md all spell it that way for coding agents. Everything else is Python.

Command	What it does
`wai login`	Sign in from this terminal. Opens the browser, saves a key.
`wai signup --email you@example.com`	Create an account and a key, no browser.
`wai status`	Show which key the SDK will use and where it came from.
`wai logout`	Delete the saved key.
`wai init-evals`	Write an eval harness (agent, judge, run, test) wired to this project.
`wai agents`	List the agents tracked on your account.
`wai agent <id>`	One tracked agent: record, behaviors, verdict.
`wai runs <id>`	The training runs of one agent, newest first.
`wai verdict <id>`	Does the candidate beat the served version, and is it real.
`wai promote <id> <version>`	Make a version the served one.
`wai live <id> ...`	Report one day of traffic on the served version.
`wai keys`	List the API keys on your account, names and prefixes.

wai login

The command prints a link and waits for you to approve it in the browser. Then it saves a key on this machine. --no-browser prints the link only, --no-wait returns at once so you can run it again to finish, --name labels the key on your account (default cli <host>). No browser at all, for example on a server:

wai signup --email you@example.com

That creates the account and saves its key in one step.

Which key is in use

wai status

The SDK resolves a key in this order: an explicit api_key= argument, the WHILEAI_API_KEY environment variable, then the saved credentials from login or signup. status says which one won.

Write an eval harness

wai init-evals --agent app.agent:answer --tools app.agent:TOOLS --system-prompt app.agent:SYSTEM

This writes five files into evals/ (change with --dir): the agent wrapper, a judge, a runner, a test that goes red in CI when the pass rate drops, and a README. The three flags point at the callable, the tool list and the system prompt as module:name. Leave them out and the command looks for them in the project. --force overwrites files that are already there. What the harness computes and how to read it is on the evals guide.

Read and promote what the platform tracks

The platform verbs a coding agent needs, as commands. Each is a thin call into whileai.platform.

wai agents                 # tracked agents and what each serves
wai agent <id>             # record, behaviors, verdict
wai runs <id>              # the version table, newest first
wai verdict <id> [--behavior <name>]
wai promote <id> <version>
wai live <id> --day YYYY-MM-DD --version <v> --replies N [--flagged N --p50 S --cost USD]
wai keys

All of them take --json to print the API’s JSON and --api-key to use a key other than the saved one. Errors exit 1 with the reason on stderr. verdict defaults to the latest run’s target behavior; --behavior picks another. live records a day of production traffic on the served version: how many replies, how many failed a check, the median latency and the spend, so the platform can compare production with the held-out score.

The old wai purge command is gone. wai.simulations.purge_agent("name") and wai.simulations.delete_empty_datasets(max_rows=2) remain in Python. Both delete by default, permanently; pass dry_run=True first, which only counts.

​Sign in

​Which key is in use

​Write an eval harness

​Read and promote what the platform tracks

Sign in

Which key is in use

Write an eval harness

Read and promote what the platform tracks