wai.OpenAI("gpt-4.1-mini", api_key="sk-..."), wai.Anthropic(...), wai.Fireworks(...),
wai.models.Bedrock(model_id, region=), wai.Endpoint(model, url=),
wai.Ollama(...), wai.Hosted(). Its repr says where the call goes and
which key it uses, wai.configure(agent=, judge=, api_key=) sets it once
for the process, and print(wai.settings) shows what each role resolves
to. That page is Connect your agent.
This page is the string form underneath, for configs and command lines.
The agent is the first argument of simulate(). Pass the model as a
string, and the key comes from that provider’s usual environment
variable. Every request goes straight to that provider. The situation
writer runs on the same model, so no While key is involved.
The provider and the model are separated by a colon. DSPy and LiteLLM
use a slash (
dspy.LM("openai/gpt-4o-mini")), so that spelling is the
common typo; wai.configure and simulate both refuse it on the line you
typed it and name the colon form to use instead:
"vllm:Qwen/Qwen3-4B@http://localhost:8000/v1". A bare model
name ("gpt-4.1-mini") and a provider that is not in the table above are
refused the same way, naming the seven forms.
Set the key the way you already do for that provider:
Amazon Bedrock
bedrock:<model-id> reaches any chat model Bedrock hosts on your AWS
account through its Converse API: a foundation model
(anthropic.claude-haiku-4-5-20251001-v1:0), a cross-region inference
profile (us.anthropic.claude-sonnet-5), or the ARN of a model you
imported yourself. Pin the region with @<region>; without it the SDK
reads AWS_REGION, then AWS_DEFAULT_REGION, then uses us-east-1.
Two ways in, both your own:
A model that refuses
temperature (Claude Sonnet 5 answers 400 “temperature is deprecated for this model”) is called once more without the field and samples at its own default.
wai.models.Bedrock(model_id, region=, api_key=), is the
same spec with a place for a key; it sits one dot down so the front door
stays small.
A model you trained and imported into Bedrock is the same spec with its
ARN as the model id, so the held-out measurement that scored the base
scores the served weights unchanged:
InvokeModel with the OpenAI chat body instead, so the SDK picks that
route from the ARN and you change nothing; and an idle import is
unloaded, so the first call after a pause took 96 s and returns
ModelNotReadyException until it is back, which the SDK waits through
(ten tries, fifteen seconds apart) before saying so in one sentence.
Neither route returns log-probabilities, so logprobs= has no effect.
Bedrock honors tool calling on imports only for GPT-OSS models.
Serve it on While
https://models.while.ai/v1 is one OpenAI-compatible endpoint in
front of every model your account registers, wherever it runs: a Bedrock
import in While’s account or in yours, or any /v1 server you host. Your
While key picks the account, model picks the row, and any OpenAI client
works, so a served model needs no SDK change at all:
A subdomain of your own:
PUT /domain {"subdomain": "acme"} makes
https://acme.models.while.ai/v1 answer for your account and no other;
a call there with another account’s key is refused. GET /v1/models lists
what you registered, DELETE /models/{name} removes one. Streaming works. An import that sat idle is restored on the
first call; until it is back the endpoint answers 503 with
code: model_starting and a Retry-After, which the OpenAI SDKs honor.
What While keeps: per model per day, the count of calls, errors and tokens,
and which five-minute windows the model answered in, and nothing else.
Never a prompt, a completion or a request log; Bedrock invocation logging
is off and the endpoint logs errors without content. Each account gets 600
requests a minute across its keys, answered with a 429 and a Retry-After
past that.
What it costs, for a model While hosts (one you handed over with
publish): 5 per unit per month while it is kept,
and $10 per import. A unit is a Bedrock Custom Model Unit; an 8B model is
2. A model in your own AWS account, or a server you run, is routed for
free. Add a card once under Account on while.ai; without one, publish
and calls to a hosted model answer 402 with code: billing_required.
Tools are functions
@wai.tool turns a typed function into the tool: the signature is the
schema, the docstring is the description, Annotated[str, "note"] or a
Google-style Args: block gives a parameter its note, and a parameter
with a default is optional. The mock world answers the calls, faults
first. To have the bodies answer instead, pass
execute=wai.Tool.dispatch([get_order, ...]). Raw OpenAI schema dicts
still work in the same list.
No tools at all yet
draft_tools writes plausible schemas from one sentence about the agent,
on the same key:
drafted, so you can tell it from a
declared tool. Replace them with your real schemas when you have them.
The judge
Same shape as the agent: a callable over a row, a verifier (wai.verify.MathEqual(), wai.verify.CodeExec(tests=...)), or a model
string on its own key. The judge is never the model it is judging; the
evals guide shows how to check it against people before you
trust it.
Three models, three arguments
Three models can take part in a run. A model string can name each one, and each goes in a different place.
Set them once for a machine with
WHILEAI_AGENT, WHILEAI_SURROGATE
(the writer) and WHILEAI_JUDGE instead. If agent and judge end up the
same model, data.degraded carries same_model and the warning says so.
The While key
Only the hosted parts need it. The SDK looks in this order and stops at the first it finds:
Keys start with
zp_.
What reaches While
Three things, and only when you ask for them:
Everything else runs on your machine.
wai status prints which key
the SDK will use and where it came from; wai login or
wai signup --email you@example.com gets one when you want the hosted
parts. The platform reference covers that side.