whileai.simulations.generate

18 public names. import whileai.simulations as wai, then wai.name.

Name	What it does
`AgentProfile`	no docstring
`ModelSimulator`	Invent what a person might send, ask, or discuss; it never grades.
`World`	The mock world for a callable agent, faults first.
`adaptive_allocator`	Adaptive mix.
`allocator_slot_counts`	Integer explore/expand/verify slots from mix shares.
`build_dimensions`	Coverage axes from this agent.
`claude_code`	no docstring
`connect`	no docstring
`hosted_model`	The default simulation brain: hosted Qwen wearing these tools.
`inspect`	Read tools and system prompt off the agent; caller extras are merged in.
`local_model`	Build an agent that talks to any OpenAI-compatible endpoint for `simulate(agent=...)`.
`novelty`	Min cosine distance from a candidate embedding to every tested row.
`open_ended_probes`	Taxonomy-free probes.
`policy_sections`	Split policy text into short rule clauses used as coverage cells.
`scenario_regions`	Weighted target regions over a pairwise covering set of the dimensions.
`seeded_agent`	Build a demo agent whose mistakes are on purpose and recorded on the row.
`world`	A `World` for a callable agent: `world.call(tool, arguments)`.
`write_scene_brief`	One cheap LLM pass per simulate().

adapters

Framework adapters. Every runner returns {steps, final_text}.

AgentProfile

class AgentProfile(
    tools: list[dict] = <factory>,
    policy: str = '',
    capabilities: dict = <factory>,
    constraints: dict = <factory>,
    transport: str = 'callable',
    name: str = '',
    rubric: str = '',
) -> None

Defined in whileai/simulations/generate/adapters.py.

claude_code

claude_code(
    extra_args: tuple | list = (),
    cwd: str | None = None,
    max_turns: int | None = None,
    timeout: float = 300.0,
) -> Callable

Defined in whileai/simulations/generate/adapters.py.

connect

connect(
    agent: Any,
    tools: list[dict] | None = None,
    system_prompt: str | None = None,
    policy: str | None = None,
    transport: str | None = None,
    execute: Callable | None = None,
    model: str | None = None,
) -> ConnectedAgent

Defined in whileai/simulations/generate/adapters.py.

inspect

inspect(
    agent: Any,
    tools: list[dict] | None = None,
    system_prompt: str | None = None,
    policy: str | None = None,
    transport: str | None = None,
) -> AgentProfile

Defined in whileai/simulations/generate/adapters.py. Read tools and system prompt off the agent; caller extras are merged in.

agents

OpenAI-compatible chat loop for hosted and local simulation backends.

hosted_model

hosted_model(
    tools: list[dict],
    system: str = '',
    fault_plans: dict | None = None,
    **kwargs,
) -> Callable

Defined in whileai/simulations/generate/agents.py. The default simulation brain: hosted Qwen wearing these tools.

local_model

local_model(
    base_url: str,
    model: str,
    tools: list[dict],
    system: str = '',
    api_key: str | None = None,
    max_turns: int | None = None,
    avg_turns: float = 12.0,
    min_user_turns: int = 1,
    turn_stats: dict | None = None,
    temperature: float = 0.8,
    logprobs: bool | str = False,
    fault_plans: dict | None = None,
    result_shapes: dict | None = None,
    opening_rate: float = 0.0,
    human_tools: set | None = None,
    execute: Callable | None = None,
    timeout: float = 300.0,
    max_tokens: int | None = None,
    user_model: str | None = None,
    thinking: bool | None = None,
    patience: Patience | None = 'normal',
    user_temperature: float | None = None,
    world_options: WorldOptions | Mapping[str, Any] | None = None,
) -> Callable

Defined in whileai/simulations/generate/agents.py. Build an agent that talks to any OpenAI-compatible endpoint for simulate(agent=...). Reach for it when the policy under test is a served model: a trained adapter behind vLLM, a local server, any chat endpoint. It returns a callable that plays the multi-turn agent (tool calls, the simulated user, faults) against model at base_url with tools and the system prompt, and every rollout comes back as a row.

base_url, model, api_key: where the model is served and what to call it.
thinking: for reasoning bases such as Qwen3. False sends chat_template_kwargs={"enable_thinking": False} so the reply is the answer, not the reasoning, the way the hosted Qwen path already does; True asks for it; None (the default) sends nothing and leaves the server’s default. The same field goes to the simulated user when the agent’s own model plays it (the default) or user_model sits on the same endpoint, so the customer is asked not to reason either; a user_model on another endpoint keeps that server’s default. Either way <think> markup never reaches step["text"], final_text, or a user turn (step["user"] and the messages history): what the user model still emits as reasoning is stripped before it becomes speech, and a turn that was reasoning with no spoken line is retried, then dropped. The run reports those under search["user_think"]: user_turns, stripped and unclosed as counts, stripped_share and unclosed_share as shares of the user turns, zeros when none.
result_shapes: pins what a tool returns, as {tool_name: example result dict}. The sandbox fills the example on every call instead of inventing a record, so a policy branch that only exists for some tool results (a credit over $200 must be escalated) is reached on purpose rather than by luck. Field names and free text stay as written; ids, dates and people are re-drawn per call, and a number moves by up to about a third of itself (900.0 lands in roughly 600 to 1200, 90.0 in 60 to 120), so pick a template value whose whole range sits on the side of the threshold you want. An argument that shares a key with the template is echoed back (invoice_id in, same invoice_id out). To measure a branch, run the same pinned tasks under two shapes, one per side of the rule. Without it the situation writer drafts an example per tool (write_result_shapes) and the branch is exercised at random.
fault_plans: schedules faults per ask, as {message: {tool_name: {"mode": "timeout", "rate": 1.0}}}, keyed by the exact user message, with mode one of timeout, malformed, stale or permission_denied and rate the chance the fault fires on a call. The plan may also carry world_state, stance, tone and texture, which are popped off and shape the world and the simulated user for that ask. simulate() writes these itself from fault_rate=; pass your own only to replay a known plan (tasks= does this for you).
timeout: seconds per completion, LOCAL_MODEL_TIMEOUT (300) by default: a served model that scaled to zero takes two to three minutes to answer its first request, and a timeout under that drops every rollout of the first pass. When a call still times out the run says so in data.warnings with the fix (raise timeout=, or send one throwaway request first so the endpoint is warm).
patience: a level name (PATIENCE_LEVELS, "normal" by default) or a table {"second": p, "later": q}: the chance the person leaves at the agent’s second question and at every later one, fitted from your own traces (see PATIENCE_HAZARDS).
user_model and user_temperature: the simulated person’s model (the agent’s own by default) and the sampling temperature of every simulated-user line, follow-ups (USER_TURN_TEMPERATURE) and human-tool answers (HUMAN_TOOL_TEMPERATURE) alike; None keeps those two defaults.
world_options: the mock world’s dials (a WorldOptions or the same fields as a dict: fault modes, hit counts, name pools, …); simulate(advanced={"world": {...}}) lands here. None is the defaults in defaults.py.
execute: your own world (tool, arguments) -> result in place of the mock one. max_turns / avg_turns (12.0) cap and shape the conversation length; avg_turns=1 is one user line and one reply, and the follow-up branch never runs; temperature (0.8), max_tokens and logprobs are the agent’s own sampling, recorded on every row.

agent = wai.local_model("http://localhost:8000/v1", "my-adapter",
                        tools=TOOLS, system=POLICY, thinking=False)
data = wai.simulate(agent, tools=TOOLS, system_prompt=POLICY, budget=100)

diversity

Sparse generic writer knobs and annealing helpers.

adaptive_allocator

adaptive_allocator(
    time_budget: float | None,
    until: str = 'compute',
    elapsed: float | None = None,
) -> dict[str, Any]

Defined in whileai/simulations/generate/diversity.py. Adaptive mix. Short remaining clock is messier; saturation walks more cards. Shares are explore / expand / verify. n_req and k are caps so expand and verify can actually run. Not a pinned n=1 k=1 policy.

allocator_slot_counts

allocator_slot_counts(take: int, plan: dict | None) -> dict[str, int]

Defined in whileai/simulations/generate/diversity.py. Integer explore/expand/verify slots from mix shares.

generator

Model-driven scenario generation. Templates are bootstrap and offline fallback.

ModelSimulator

class ModelSimulator(
    backend_spec: str | None = None,
    tools: Sequence[dict] = (),
    policy: str = '',
    candidates_per_round: int = 160,
    seed: int = 0,
    dimensions: dict | None = None,
    timeout: float = 30.0,
    cells_per_request: int | None = None,
    completions: int | None = None,
    distinct_cards: bool = False,
    extra_cards: int = 1,
    texture_rate: float | None = None,
    kind: str | None = None,
    scene_brief: str = '',
    out_tokens: int | None = None,
    time_budget: float | None = None,
    run_started: float | None = None,
    mode: str | None = None,
    prefer_success: bool | None = None,
    steering_weight: float | None = None,
    hard_share: float | None = None,
    writer_temperature: float | tuple[float, float] | None = None,
    world: Any = None,
)

Defined in whileai/simulations/generate/generator.py. Invent what a person might send, ask, or discuss; it never grades.

write_scene_brief

write_scene_brief(
    tools: Sequence[dict] = (),
    policy: str = '',
    backend_spec: str | None = None,
    kind: str = '',
    timeout: float = 8.0,
) -> str

Defined in whileai/simulations/generate/generator.py. One cheap LLM pass per simulate(). Private writer context. Empty on failure.

offline_agent

The free path, with something to catch.

World

class World(tools: Sequence[dict] | None = None, seed: int = 0) -> None

Defined in whileai/simulations/generate/offline_agent.py. The mock world for a callable agent, faults first. Built once; call reads the rollout being answered from current_rollout (thread-local, set by simulate before each rollout) so the row’s own faults and world_state apply. Outside a run it answers with no faults and no world state.

World.call

call(self, tool: str, arguments: dict | None = None) -> dict[str, Any]

The world’s answer to one tool call. Scheduled faults apply first.

seeded_agent

seeded_agent(
    tools: Sequence[dict],
    rate: float = 0.35,
    seed: int = 0,
    behaviors: Sequence[str] | None = None,
) -> Callable[[str], dict]

Defined in whileai/simulations/generate/offline_agent.py. Build a demo agent whose mistakes are on purpose and recorded on the row. Reach for it to try the whole loop offline, with no key and no model: it gives a run something to catch, and each row says what was planted, so a grader or a marker can be checked against the truth. It returns a callable message -> trajectory for simulate(agent=...). Honest by default: it picks the tool the ask names, calls it through world() (faults fire), and reports what came back. On rate of rollouts, drawn deterministically from seed, the prompt and the rollout index, it does one thing from behaviors: hedging, sycophancy, apology and boilerplate add the phrase style_report looks for; ignore_fault claims success although the tool faulted; leak quotes the row’s privileged context. Each row it answers carries seeded: what it did on purpose, [] when it behaved.

tools: the tool schemas the agent may call.
rate (SEEDED_RATE, 0.35): the share of rollouts with one planted mistake, high enough that a 20-row demo run catches every behavior kind at least once; a convention, not a real failure rate.
seed (0): fixes which rollouts misbehave and how.
behaviors (SEEDED_BEHAVIORS): the subset of mistakes to draw from.

agent = wai.seeded_agent(TOOLS, rate=0.4, seed=3)
data = wai.simulate(agent, tools=TOOLS, simulator=False, budget=20)
print(sum(1 for row in data.trajectories if row["seeded"]), "planted")

world

world(tools: Sequence[dict] | None = None, seed: int = 0) -> World

Defined in whileai/simulations/generate/offline_agent.py. A World for a callable agent: world.call(tool, arguments). Pass the same tools you pass simulate. Inside a run the row’s scheduled faults fire on it; a callable that answers its tool calls through this sees them, and the row’s faults stop being a label with no effect.

scenarios

Coverage cells from the agent’s tools and policy, plus offline fallback wording.

build_dimensions

build_dimensions(
    tools: list[dict],
    policy: str = '',
    rule_cap: int | None = 16,
) -> dict[str, list[str]]

Defined in whileai/simulations/generate/scenarios.py. Coverage axes from this agent. Length and vagueness are writer-only. rule_cap is the most policy clauses on the rule axis: the grid’s RULE_CAP by default, None for every clause (what a report over an existing suite passes, RULE_AXIS_CAP_REPORT).

novelty

novelty(candidate_vector, tested_matrix) -> float

Defined in whileai/simulations/generate/scenarios.py. Min cosine distance from a candidate embedding to every tested row.

open_ended_probes

open_ended_probes(
    tools: list[dict],
    policy: str = '',
    per_round: int = 10,
    seed: int = 0,
) -> list[str]

Defined in whileai/simulations/generate/scenarios.py. Taxonomy-free probes. Wording rotates with seed.

policy_sections

policy_sections(policy: str, cap: int | None = 16) -> list[str]

Defined in whileai/simulations/generate/scenarios.py. Split policy text into short rule clauses used as coverage cells. Identity / system-prompt preambles are not clauses. A long unsplit paragraph is dropped rather than truncated mid-word into rule. cap is the most clauses returned, in document order; None is every clause. rule_axis says how many a cap left out.

scenario_regions

scenario_regions(
    tools: list[dict],
    policy: str = '',
    strength: int = 2,
    observed_counts: dict[str, int] | None = None,
    novelty: Callable[[dict], float] | None = None,
    behavior_value: Callable[[dict], float] | None = None,
    alpha: float = 0.35,
    beta: float = 0.35,
    gamma: float = 0.2,
    delta: float = 0.1,
    dimensions: dict | None = None,
    mode: str | None = None,
    prefer_success: bool | None = None,
) -> list[dict]

Defined in whileai/simulations/generate/scenarios.py. Weighted target regions over a pairwise covering set of the dimensions. prefer_success defaults off in mode="rl" so fault cells survive for covering-grid RL data. Explicit True/False always wins.

​adapters

​AgentProfile

​claude_code

​connect

​inspect

​agents

​hosted_model

​local_model

​diversity

​adaptive_allocator

​allocator_slot_counts

​generator

​ModelSimulator

​write_scene_brief

​offline_agent

​World

​World.call

​seeded_agent

​world

​scenarios

​build_dimensions

​novelty

​open_ended_probes

​policy_sections

​scenario_regions

adapters

AgentProfile

claude_code

connect

inspect

agents

hosted_model

local_model

diversity

adaptive_allocator

allocator_slot_counts

generator

ModelSimulator

write_scene_brief

offline_agent

World

World.call

seeded_agent

world

scenarios

build_dimensions

novelty

open_ended_probes

policy_sections

scenario_regions