> ## Documentation Index > Fetch the complete documentation index at: https://docs.while.ai/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Install with `uv add whileai`; import as `import whileai as wai`. > Run the offline path first (`simulator=False`, `wai.seeded_agent`, a callable judge); no key is needed for it. > Report every pass rate with its interval and n, as `scored.pass_at` prints it. # Your model and your key > How to name the model the agent runs on, which environment variable each provider reads, where the requests go, and the three things that reach While only when you ask. **What you learn:** the model as a string per provider, which variable holds its key, where the requests go, and what reaches While only when you ask. **Needs:** the key of the provider you name; nothing to read. **Takes:** five minutes. The shortest answer is a backend object: `wai.OpenAI("gpt-4.1-mini", api_key="sk-...")`, `wai.Anthropic(...)`, `wai.Fireworks(...)`, `wai.models.Bedrock(model_id, region=)`, `wai.Endpoint(model, url=)`, `wai.Ollama(...)`, `wai.Hosted()`. Its repr says where the call goes and which key it uses, `wai.configure(agent=, judge=, api_key=)` sets it once for the process, and `print(wai.settings)` shows what each role resolves to. That page is [Connect your agent](/get-started/connect-your-agent). This page is the string form underneath, for configs and command lines. The agent is the first argument of `simulate()`. Pass the model as a string, and the key comes from that provider's usual environment variable. Every request goes straight to that provider. The situation writer runs on the same model, so no While key is involved. ```python theme={"theme":"vitesse-dark"} import whileai as wai @wai.tool def get_order(order_id: str) -> dict: """Look up an order by id.""" ... data = wai.simulate( "openai:gpt-4.1-mini", # the agent; key from OPENAI_API_KEY tools=[get_order], # the function is the tool system_prompt="Help customers with orders.", mode="rl", repeats=4, budget=64, ) ``` | Agent | Key | Requests go to | | ------------------------------------------------------------ | ------------------------------------------------------------------- | ---------------------------------------- | | `"openai:"` | `OPENAI_API_KEY` (`OPENAI_BASE_URL` for a compatible server) | api.openai.com, or the base URL you set | | `"anthropic:"` | `ANTHROPIC_API_KEY` | api.anthropic.com | | `"fireworks:accounts/fireworks/models/"` | `FIREWORKS_API_KEY` | api.fireworks.ai | | `"bedrock:[@]"` | `AWS_BEARER_TOKEN_BEDROCK`, else your AWS credentials through boto3 | bedrock-runtime.``.amazonaws.com | | `"vllm:@"` | `OPENAI_API_KEY`; none for localhost or plain http | `` | | `"ollama:"` | none | localhost:11434 | | `"typesafe:"` (judge only) | `TYPESAFE_API_KEY` | TypeSafe's API | | `my_agent(message) -> {"steps": [...], "final_text": "..."}` | yours | wherever your code goes | | `wai.seeded_agent([get_order])` | none | nowhere: an offline stand-in | The provider and the model are separated by a **colon**. DSPy and LiteLLM use a slash (`dspy.LM("openai/gpt-4o-mini")`), so that spelling is the common typo; `wai.configure` and `simulate` both refuse it on the line you typed it and name the colon form to use instead: ```python theme={"theme":"vitesse-dark"} >>> wai.configure(agent="openai/gpt-4.1-mini") ValueError: agent='openai/gpt-4.1-mini' separates the provider from the model with a slash, the spelling DSPy and LiteLLM use; whileai uses a colon, so pass agent='openai:gpt-4.1-mini' (or 'vllm:openai/gpt-4.1-mini@' if that is a repo id on a server you run) ``` A model name may itself contain slashes after the colon, as a Hugging Face repo id does: `"vllm:Qwen/Qwen3-4B@http://localhost:8000/v1"`. A bare model name (`"gpt-4.1-mini"`) and a provider that is not in the table above are refused the same way, naming the seven forms. Set the key the way you already do for that provider: ```bash theme={"theme":"vitesse-dark"} export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY=... ``` ## Amazon Bedrock `bedrock:` reaches any chat model Bedrock hosts on your AWS account through its Converse API: a foundation model (`anthropic.claude-haiku-4-5-20251001-v1:0`), a cross-region inference profile (`us.anthropic.claude-sonnet-5`), or the ARN of a model you imported yourself. Pin the region with `@`; without it the SDK reads `AWS_REGION`, then `AWS_DEFAULT_REGION`, then uses `us-east-1`. Two ways in, both your own: | You have | Set | Installs | | -------------------------------------------------------- | ----------------------------------------------------------------- | ----------------------------------------------------------------- | | a Bedrock API key (console, *API keys*) | `AWS_BEARER_TOKEN_BEDROCK`, or `api_key=` on `wai.models.Bedrock` | nothing; the calls go over `requests` | | AWS credentials (`aws configure`, `AWS_PROFILE`, a role) | nothing more | `pip install "whileai[bedrock]"` for boto3, which signs the calls | A model that refuses `temperature` (Claude Sonnet 5 answers 400 "`temperature` is deprecated for this model") is called once more without the field and samples at its own default. ```python theme={"theme":"vitesse-dark"} import whileai as wai wai.configure( agent="bedrock:us.anthropic.claude-haiku-4-5-20251001-v1:0@us-west-2", judge="bedrock:us.anthropic.claude-sonnet-5@us-west-2", ) ``` The object form, `wai.models.Bedrock(model_id, region=, api_key=)`, is the same spec with a place for a key; it sits one dot down so the front door stays small. A model you trained and imported into Bedrock is the same spec with its ARN as the model id, so the held-out measurement that scored the base scores the served weights unchanged: ```python theme={"theme":"vitesse-dark"} served = "bedrock:arn:aws:bedrock:us-east-1:123456789012:imported-model/abc123def456" after = wai.simulate(served, tools=TOOLS, system_prompt=POLICY, seed=0) ``` Three facts about imported models, measured on 2026-09-20 with a Llama 3.1 8B adapter: the import takes merged weights in the Hugging Face layout (a LoRA adapter is merged into its base first, then uploaded to S3; the [Bedrock import recipe](/recipes/05-export/bedrock-import) does it on Modal); Bedrock refuses Converse for an imported model and answers `InvokeModel` with the OpenAI chat body instead, so the SDK picks that route from the ARN and you change nothing; and an idle import is unloaded, so the first call after a pause took 96 s and returns `ModelNotReadyException` until it is back, which the SDK waits through (ten tries, fifteen seconds apart) before saying so in one sentence. Neither route returns log-probabilities, so `logprobs=` has no effect. Bedrock honors tool calling on imports only for GPT-OSS models. ## Serve it on While `https://models.while.ai/v1` is one OpenAI-compatible endpoint in front of every model your account registers, wherever it runs: a Bedrock import in While's account or in yours, or any `/v1` server you host. Your While key picks the account, `model` picks the row, and any OpenAI client works, so a served model needs no SDK change at all: ```python theme={"theme":"vitesse-dark"} import whileai as wai served = wai.Endpoint("nemotron-8b-t2s-r1", url="https://models.while.ai/v1", api_key="zp_...") ``` Register a model once, with the same key: ```bash theme={"theme":"vitesse-dark"} curl -X POST https://models.while.ai/models \ -H "Authorization: Bearer $WHILEAI_API_KEY" -H "Content-Type: application/json" \ -d '{"name": "nemotron-8b-t2s-r1", "arn": "arn:aws:bedrock:us-east-1::imported-model/", "roleArn": "arn:aws:iam:::role/WhileModelsInvoke"}' ``` | you register | While calls | credentials | | -------------------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `{name, arn, region?, roleArn?}` | a Bedrock import, custom deployment, provisioned model or inference profile | While's own role, or a role in your account named `WhileModelsInvoke*` that trusts account `940333627479` with external id `while-models`, assumed per call; While never holds a key of yours | | `{name, url, model, auth?}` | any OpenAI-compatible `/v1` server | `auth: "caller"` forwards your While key to it, `"none"` sends nothing | A subdomain of your own: `PUT /domain {"subdomain": "acme"}` makes `https://acme.models.while.ai/v1` answer for your account and no other; a call there with another account's key is refused. `GET /v1/models` lists what you registered, `DELETE /models/{name}` removes one. Streaming works. An import that sat idle is restored on the first call; until it is back the endpoint answers 503 with `code: model_starting` and a `Retry-After`, which the OpenAI SDKs honor. What While keeps: per model per day, the count of calls, errors and tokens, and which five-minute windows the model answered in, and nothing else. Never a prompt, a completion or a request log; Bedrock invocation logging is off and the endpoint logs errors without content. Each account gets 600 requests a minute across its keys, answered with a 429 and a `Retry-After` past that. What it costs, for a model While hosts (one you handed over with `publish`): $0.12 per unit-minute while it answers, counted in the same five-minute windows Bedrock bills, $5 per unit per month while it is kept, and \$10 per import. A unit is a Bedrock Custom Model Unit; an 8B model is 2\. A model in your own AWS account, or a server you run, is routed for free. Add a card once under Account on while.ai; without one, `publish` and calls to a hosted model answer 402 with `code: billing_required`. ## Tools are functions `@wai.tool` turns a typed function into the tool: the signature is the schema, the docstring is the description, `Annotated[str, "note"]` or a Google-style `Args:` block gives a parameter its note, and a parameter with a default is optional. The mock world answers the calls, faults first. To have the bodies answer instead, pass `execute=wai.Tool.dispatch([get_order, ...])`. Raw OpenAI schema dicts still work in the same list. ## No tools at all yet `draft_tools` writes plausible schemas from one sentence about the agent, on the same key: ```python theme={"theme":"vitesse-dark"} from whileai.simulations.generate.generator import draft_tools TOOLS = draft_tools( "a support agent that looks up orders and issues refunds", backend_spec="openai:gpt-4.1-mini", ) ``` Each drafted schema is marked `drafted`, so you can tell it from a declared tool. Replace them with your real schemas when you have them. ## The judge Same shape as the agent: a callable over a row, a verifier (`wai.verify.MathEqual()`, `wai.verify.CodeExec(tests=...)`), or a model string on its own key. The judge is never the model it is judging; the [evals guide](/evals) shows how to check it against people before you trust it. ## Three models, three arguments Three models can take part in a run. A model string can name each one, and each goes in a different place. | Role | What it does | Where the string goes | Default | | ---------- | ---------------------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------- | | **Agent** | Answers the customer and calls tools. The thing under test. | First argument of `simulate()`, or `backend=` | The hosted Qwen, on your While key | | **Writer** | Writes the situations and plays the customer in follow-up turns. | `simulator=` on `simulate()`; `user_model=` for the customer alone | Same model as the agent; `simulator=False` is the offline template writer | | **Judge** | Grades the finished conversation. | `data.grade(llm_spec=...)`, or `wai.grade(rows, spec=...)` | Hosted Phi-4, a different family from the agent | Set them once for a machine with `WHILEAI_AGENT`, `WHILEAI_SURROGATE` (the writer) and `WHILEAI_JUDGE` instead. If agent and judge end up the same model, `data.degraded` carries `same_model` and the warning says so. ## The While key Only the hosted parts need it. The SDK looks in this order and stops at the first it finds: | Order | Where | How | | ----- | ---------------------- | ---------------------------------------------------------------- | | 1 | `api_key=` on the call | `wai.simulate(..., api_key="zp_...")`, `data.grade(api_key=...)` | | 2 | Environment | `export WHILEAI_API_KEY=zp_...` | | 3 | Saved credentials | `wai login`, or `wai signup --email you@example.com` | Keys start with `zp_`. ## What reaches While Three things, and only when you ask for them: | You do this | What happens | | ------------------------------------------------- | --------------------------------------------------------------------- | | leave `agent=` out | the agent is the Qwen the platform hosts, on the key from `wai login` | | leave `simulator=False` out with a callable agent | the situation writer is hosted, on the same key | | call `push`, `train` or `serve` | rows go to your account for hosted training and serving | Everything else runs on your machine. `wai status` prints which key the SDK will use and where it came from; `wai login` or `wai signup --email you@example.com` gets one when you want the hosted parts. The [platform reference](/reference/platform) covers that side.