whileai.simulations.ingest

35 public names. import whileai.simulations as wai, then wai.name.

Name	What it does
`agents`	Every agent on your account with counts: traces, sets by purpose, public cards.
`catalog`	The public catalog: `{"datasets": [card, ...], "agents": [...]}`.
`datasets`	List your datasets plus storage used, newest first.
`delete_dataset`	Permanently delete a dataset from your account.
`delete_empty_datasets`	Delete datasets with no stored bytes, or with `max_rows` rows or fewer when that is set (smoke runs).
`dimensions_from_traces`	Coverage axes aimed at behaviors seen in `rows`.
`drop_leaky_rows`	Kept rows plus the report.
`flaw_rows`	Rows with an observed fault or a 0 label: the next round’s traces.
`format_trace_report`	The trace report as a text block, aiming stated in plain words.
`hf_publish`	Push one of your datasets to a Hugging Face dataset repo you own.
`hf_publish_run`	Push a finished training run’s LoRA adapter to a Hugging Face model repo you own, with a model card (base model, metrics, the dataset repo when the data was pushed too).
`hf_status`	Is a Hugging Face account connected to this account, and which namespaces (you plus your orgs) can it publish under?
`import_hf`	Bring one split of a Hugging Face dataset onto your account as rows, so it gets a profile (pass rate, gradient support, mixed prompts) before you train on it.
`infer_harness`	Draft a harness from observed trace rows.
`issue_delegated_credential`	Create a short-lived delegated credential for SDK or backend use.
`leakage_report`	Near-copy check of generated prompts against source traces.
`load_traces`	Normalize any supported trace source to the one trajectory schema the SDK reads.
`mine_traces`	What the deployed agent actually did, counted for grid focusing.
`opening_share`	Share of traces whose conversation opens with the assistant.
`preview`	Three sample rows and the analyzer report for one of your datasets.
`profile`	The trainer’s numbers for one of your datasets: pass rate, gradient support, tasks with both a pass and a fail, tool use, tokens, per-task pass rates.
`publish`	Publish one of your datasets as a public card on huggingface.co/while-ai.
`pull`	Download a dataset.
`purge_agent`	Remove an agent and everything under it: its traces, its datasets, and its registry record.
`push_file`	Upload an existing JSONL file.
`push_rows`	Upload rows as JSONL to your While account.
`refresh_delegated_credential`	Refresh a delegated credential before it expires.
`register_agent`	Create or update an agent record: the name, and optionally what it is (a line), its tool schemas, and its system prompt.
`rows_from_otel`	Conversations from an OTLP JSON batch or an iterable of spans.
`simulate_from_traces`	Alias for `simulate(agent, traces=...)`: same grid focus and leakage gate, for callers who start from the traces.
`split_pseudo_production`	Set aside a pseudo-production slice; the rest stays for training.
`tools_from_traces`	The agent’s tool surface, read off the calls the traces contain.
`trace_report`	What these traces contain and what they will aim generation at.
`unpublish`	Take a dataset off the public catalog.
`update_dataset`	Change what a dataset is for, its mode, agent or description.

otel

OTel GenAI spans to trajectory rows: simulate(traces=...) input.

rows_from_otel

rows_from_otel(
    source: Any,
    reward_keys: Sequence[str] = ('whileai.reward',),
) -> list[dict]

Defined in whileai/simulations/ingest/otel.py. Conversations from an OTLP JSON batch or an iterable of spans. One row per conversation id (trace id when no conversation id is emitted): the first user message becomes prompt, tool spans in time order become steps, later user turns become user steps, and the last assistant output becomes final_text. reward_keys names the span attributes a 0..1 reward is read from (REWARD_KEYS).

platform

While platform client: dataset upload, listing, download.

agents

agents(api_key: str | None = None) -> list[dict]

Defined in whileai/simulations/ingest/platform.py. Every agent on your account with counts: traces, sets by purpose, public cards. An agent exists the moment a push names it (data.push(name, agent=...)) or a trace arrives with gen_ai.agent.name; register_agent is for attaching the spec or a description ahead of that.

catalog

catalog() -> dict

Defined in whileai/simulations/ingest/platform.py. The public catalog: {"datasets": [card, ...], "agents": [...]}. No key needed.

datasets

datasets(api_key: str | None = None) -> dict

Defined in whileai/simulations/ingest/platform.py. List your datasets plus storage used, newest first.

delete_dataset

delete_dataset(dataset_id: str, api_key: str | None = None) -> dict

Defined in whileai/simulations/ingest/platform.py. Permanently delete a dataset from your account.

delete_empty_datasets

delete_empty_datasets(
    max_rows: int = 0,
    dry_run: bool = False,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Delete datasets with no stored bytes, or with max_rows rows or fewer when that is set (smoke runs). Permanent. Returns the ids.

hf_publish

hf_publish(
    dataset_id: str,
    namespace: str | None = None,
    repo: str | None = None,
    private: bool = False,
    wait: bool = True,
    timeout: float = 600.0,
    poll: float = 2.5,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Push one of your datasets to a Hugging Face dataset repo you own. The set’s purpose (train, holdout, eval) is the split; pushing the holdout set into the same repo adds a second split. Every push is a commit tagged zp-<dataset id> and the repo carries whileai.json (split -> dataset, numbers, history). Defaults: your username and a slug of the dataset name. With wait (the default) this returns the finished state {"repo", "url", "commit", "tag", "split", ...}; otherwise the pushing stamp. This is the platform route, for a set that lives on your account; for a file, an adapter directory or rows on your machine, wai.hub.push and export(push_to=) upload with your own HF_TOKEN and never call the platform.

hf_publish_run

hf_publish_run(
    run_id: str,
    namespace: str | None = None,
    repo: str | None = None,
    private: bool = True,
    wait: bool = True,
    timeout: float = 900.0,
    poll: float = 3.0,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Push a finished training run’s LoRA adapter to a Hugging Face model repo you own, with a model card (base model, metrics, the dataset repo when the data was pushed too). Private by default: it is a checkpoint, not a release. Tagged zp-<run id>.

hf_status

hf_status(api_key: str | None = None) -> dict

Defined in whileai/simulations/ingest/platform.py. Is a Hugging Face account connected to this account, and which namespaces (you plus your orgs) can it publish under? Connect one on any dataset page at while.ai/platform/datasets. Returns {"connected", "username", "namespaces", "scopes"}.

import_hf

import_hf(
    repo: str,
    split: str = 'train',
    revision: str | None = None,
    config: str | None = None,
    name: str | None = None,
    purpose: str = 'train',
    mode: str | None = None,
    agent: str | None = None,
    description: str | None = None,
    max_rows: int = 100000,
    wait: bool = True,
    timeout: float = 900.0,
    poll: float = 3.0,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Bring one split of a Hugging Face dataset onto your account as rows, so it gets a profile (pass rate, gradient support, mixed prompts) before you train on it. Parquet, JSONL, CSV and Arrow all come in the same way. Public repos need no connected account; private ones use the Hugging Face account connected on the platform. Returns the dataset row. With wait (the default) the row is ready (or this raises with the import error); otherwise it is importing and wai.datasets() shows it settle.

issue_delegated_credential

issue_delegated_credential(
    clerk_token: str | None,
    ttl_seconds: int = 3600,
    name: str = 'sdk-default',
    timeout: float = 120,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Create a short-lived delegated credential for SDK or backend use. clerk_token must be a valid Clerk session token or other authenticated backend token. This helper sends that token as a bearer token to the auth endpoint to mint the delegated credential.

preview

preview(dataset_id: str, api_key: str | None = None) -> dict

Defined in whileai/simulations/ingest/platform.py. Three sample rows and the analyzer report for one of your datasets.

profile

profile(
    dataset_id: str,
    force: bool = False,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. The trainer’s numbers for one of your datasets: pass rate, gradient support, tasks with both a pass and a fail, tool use, tokens, per-task pass rates. Cached on the platform until the set changes; force=True recomputes.

publish

publish(
    dataset_id: str,
    agent: str,
    description: str | None = None,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Publish one of your datasets as a public card on huggingface.co/while-ai. Cards are grouped by agent (a short name such as "airline-support"). The dataset must be finalized and hold rows. Returns the card. Anyone can then pull it with no key.

pull

pull(
    dataset_id: str,
    path: str | None = None,
    api_key: str | None = None,
) -> str | list[dict]

Defined in whileai/simulations/ingest/platform.py. Download a dataset. Writes JSONL to path and returns the path, or returns the parsed rows when path is omitted. Public catalog datasets need no key; your own need the usual one. A dataset is stored as one or more parts, and the grant lists every one of them. Datasets pushed with push_rows are a single part, which is why reading only downloadUrl looked correct for so long; a dataset filled by trace ingest is one part per trace, and that path returned the first row of a 60-row dataset without saying so.

purge_agent

purge_agent(
    agent: str,
    dry_run: bool = False,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Remove an agent and everything under it: its traces, its datasets, and its registry record. Permanent. dry_run=True only counts. Returns {"agent", "traces", "datasets", "deleted"}.

push_file

push_file(
    path: str,
    name: str | None = None,
    api_key: str | None = None,
    parent: str | None = None,
    gate: bool = True,
    mode: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Upload an existing JSONL file. name defaults to the file name. gate=True (default) parses the file, runs publish_gate (rows get their calibration stamp; RL-shaped rows that are ungraded or have no mixed group are refused), and uploads the stamped rows. The report comes back as entry["gate"]. gate=False uploads the bytes exactly as they are on disk.

push_rows

push_rows(
    rows: list[dict],
    name: str,
    api_key: str | None = None,
    parent: str | None = None,
    gate: bool = False,
    mode: str | None = None,
    purpose: str | None = None,
    agent: str | None = None,
    description: str | None = None,
    endorsed: Sequence[str] = (),
    strict_hacks: bool = False,
    prove_effect: float = 0.05,
    holdout: float | None = None,
    publish: bool = False,
    timeout: float | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Upload rows as JSONL to your While account. Returns the registry entry, including datasetId. Pass parent (a ds_... id) when this dataset is an iteration of an existing one, so lineage shows on the platform. purpose is what the set is for on the Datasets page: "train" (the default), "holdout" or "eval"; mode is the simulation mode that made it, and is also recorded. gate=True runs publish_gate first (calibration stamp; RL-shaped rows refused when ungraded or without a mixed group) and returns its report as entry["gate"]. endorsed names what the reward should track for the gate’s hack_scan; strict_hacks=True refuses a set whose reward is best explained by something else. SimulationData.push gates by default; this row-level entry point does not, because the caller may already have run optimize. A purpose="holdout" push warns when the set is too small to prove a prove_effect gain (5 points) at 80% power. holdout=0.2 keeps a fifth of the tasks (by scenario_id) out of the set and pushes them as a second, linked dataset with purpose "holdout"; the entry carries it as ["holdout"]. publish=True with an agent name also puts the set on the public catalog as a card (["card"]). Both are what SimulationData.push takes, so a graded RL push (scored.push) has the same route to a linked holdout (#408). timeout caps the upload in seconds; the default grows with the payload (put_timeout_for).

refresh_delegated_credential

refresh_delegated_credential(
    clerk_token: str | None,
    credential: str,
    ttl_seconds: int = 3600,
    timeout: float = 120,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Refresh a delegated credential before it expires.

register_agent

register_agent(
    name: str,
    description: str | None = None,
    tools: list | None = None,
    system_prompt: str | None = None,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Create or update an agent record: the name, and optionally what it is (a line), its tool schemas, and its system prompt. Returns the record.

unpublish

unpublish(dataset_id: str, api_key: str | None = None) -> dict

Defined in whileai/simulations/ingest/platform.py. Take a dataset off the public catalog. The data stays on your account.

update_dataset

update_dataset(
    dataset_id: str,
    purpose: str | None = None,
    mode: str | None = None,
    agent: str | None = None,
    description: str | None = None,
    api_key: str | None = None,
) -> dict

Defined in whileai/simulations/ingest/platform.py. Change what a dataset is for, its mode, agent or description. purpose moves it between the Train, Holdout, Eval and Raw sections of the Training data page. Only the arguments you pass change.

traces

Production traces to a focused coverage grid. Simulation, version two.

dimensions_from_traces

dimensions_from_traces(
    rows: Sequence[dict],
    tools: list[dict],
    policy: str = '',
    broaden: bool = True,
    fault_to_axis: Mapping[str, tuple[str, str]] | None = None,
) -> dict[str, list[str]]

Defined in whileai/simulations/ingest/traces.py. Coverage axes aimed at behaviors seen in rows. Starts from build_dimensions for this agent so every value is one the writer and sandbox understand. The tool axis puts observed failing tools first; broaden=False drops tools the traces never touched (keeping the base specials such as unrelated), so a run spends its budget near the flaws instead of boiling the ocean. Fault and world axes always keep their clean value: contrast needs passing rows too. fault_to_axis maps an observed fault chip to the axis value that reproduces it (FAULT_TO_AXIS by default).

drop_leaky_rows

drop_leaky_rows(
    rows: Sequence[dict],
    sources: Sequence[Any],
    threshold: float = 0.9,
    embedder: Any = 'hash',
    examples: int = 20,
) -> tuple[list[dict], dict[str, Any]]

Defined in whileai/simulations/ingest/traces.py. Kept rows plus the report. Flagged rows are removed, not rewritten. sources takes the same shapes as decontaminate(against=...): a list of rows, a list of row lists ([holdout], flattened), a list of prompt strings, or a single row. A row byte-identical to a source is always dropped, under every shape and whatever the embedder thinks.

>>> holdout = [{"prompt": "Cancel order 9911.", "task_id": "h1"}]
>>> train = [{"prompt": "Cancel order 9911.", "task_id": "t1"}]
>>> kept, report = drop_leaky_rows(train, sources=[holdout])
>>> len(kept), report["n_leaky"], report["max_similarity"]
(0, 1, 1.0)

flaw_rows

flaw_rows(rows: Sequence[dict]) -> list[dict]

Defined in whileai/simulations/ingest/traces.py. Rows with an observed fault or a 0 label: the next round’s traces. The hill-climb loop feeds a round’s failures back into simulate(traces=flaw_rows(evaluated)) so the next batch aims at what the agent still gets wrong.

format_trace_report

format_trace_report(report: dict[str, Any]) -> str

Defined in whileai/simulations/ingest/traces.py. The trace report as a text block, aiming stated in plain words.

infer_harness

infer_harness(rows: Sequence[dict]) -> dict[str, Any]

Defined in whileai/simulations/ingest/traces.py. Draft a harness from observed trace rows. Mechanical, no model. Tool schemas come from what the agent actually sent: every argument key seen for a tool becomes a property, its JSON type read off the observed values, and a key present on every call becomes required. The policy cannot be inferred — exporters do not ship system prompts — so it comes back empty for the caller to fill in. A drafted schema is a starting point to edit, not a spec to trust: it can only describe arguments the traces happened to exercise.

leakage_report

leakage_report(
    generated: Sequence[Any],
    sources: Sequence[Any],
    threshold: float = 0.9,
    embedder: Any = 'hash',
    examples: int = 20,
) -> dict[str, Any]

Defined in whileai/simulations/ingest/traces.py. Near-copy check of generated prompts against source traces. A generated row whose prompt sits at or above threshold cosine similarity to any source prompt is flagged (0.9 by default: the 8-gram exact-overlap test of Lambert 2025, chapter Evaluation, with a small paraphrase allowance). Exact matches always flag, whatever the embedder thinks. leaky lists the first examples offenders; n_leaky is the full count. sources takes the same shapes as decontaminate(against=...): a list of rows, a list of row lists ([holdout], flattened), a list of prompt strings, or a single row. Each shape gives the same report.

load_traces

load_traces(source) -> list[dict]

Defined in whileai/simulations/ingest/traces.py. Normalize any supported trace source to the one trajectory schema the SDK reads. Reach for it when you have traces from somewhere else (a production log, an eval harness, an OpenAI-style messages export) and want simulate(traces=...), evaluate or decontaminate to read them. It returns a list of dicts in the canonical schema: prompt (the first user ask), steps (a list of {"user": str}, {"tool": str, "arguments": dict, "result": Any} and {"text": str} agent turns), final_text (the agent’s last message) and, optionally, reward (0 or 1). Every other key carries through untouched, and ungraded traces are first-class.

source: a JSONL path or an iterable of dicts. Rows carrying tool_trace/trace instead of steps, final/output/response instead of final_text, or only OpenAI-style messages are converted (PROMPT_KEYS, STEP_KEYS, FINAL_KEYS, ARG_KEYS and RESULT_KEYS list the spellings read); reward is kept only when it coerces cleanly to 0 or 1, and its absence is fine. Rows that are not dicts or carry neither an ask nor any steps are dropped.

>>> rows = wai.load_traces([{"question": "Where is order 4473?", "output": "Shipped."}])
>>> rows[0]["prompt"], rows[0]["final_text"]
('Where is order 4473?', 'Shipped.')

mine_traces

mine_traces(rows: Sequence[dict]) -> dict[str, Any]

Defined in whileai/simulations/ingest/traces.py. What the deployed agent actually did, counted for grid focusing. flaw_rows is any row with an observed fault or a 0 label. Those are the behaviors worth simulating more of.

opening_share

opening_share(rows: Sequence[dict]) -> float

Defined in whileai/simulations/ingest/traces.py. Share of traces whose conversation opens with the assistant. Reads the raw messages field (kept by load_traces); rows without messages count as user-opened. This is the evidence the opening="auto" topology axis resolves against.

simulate_from_traces

simulate_from_traces(
    traces: Sequence[dict],
    agent: Any = None,
    tools: list[dict] | None = None,
    policy: str = '',
    mode: str = 'rl',
    **kwargs: Any,
)

Defined in whileai/simulations/ingest/traces.py. Alias for simulate(agent, traces=...): same grid focus and leakage gate, for callers who start from the traces. With no agent, tools or policy, the tool surface is read from the traces themselves, so handing over graded telemetry is enough to start.

split_pseudo_production

split_pseudo_production(
    rows: Sequence[dict],
    fraction: float = 0.2,
    seed: int = 0,
) -> tuple[list[dict], list[dict]]

Defined in whileai/simulations/ingest/traces.py. Set aside a pseudo-production slice; the rest stays for training. The split is by task, not by row: every row sharing a task_key (the scenario_id, else the prompt) lands on the same side, so the held-out slice is disjoint from the training side in the unit every report groups by, not just prompt-disjoint. Splitting by row is not enough — under mode="rl" with repeats=k each prompt has k rows, and scattering siblings across the two sides trains the student on every prompt it is then evaluated on. Every unique flaw signature (fault name plus behavior shape) sends its task to the production side first, so the held-out slice contains each distinct failure at least once. fraction is still counted in rows, but whole tasks are added, so the slice can overshoot it by up to the size of one task. Deterministic in seed.

tools_from_traces

tools_from_traces(traces: Sequence[dict]) -> list[dict]

Defined in whileai/simulations/ingest/traces.py. The agent’s tool surface, read off the calls the traces contain. Argument names are unioned across every observed call, so a tool called with different arguments in different traces ends up with all of them.

trace_report

trace_report(
    traces,
    tools: list[dict] | None = None,
    policy: str = '',
) -> dict[str, Any]

Defined in whileai/simulations/ingest/traces.py. What these traces contain and what they will aim generation at. Run before simulate(traces=...). With tools (and optionally policy) the report also computes the actual grid emphasis: which axis values move forward in the coverage grid because of these traces. Reward stays optional; ungraded counts are reported, never required. advisory_labels counts rows carrying only a qwen_reward: those labels do steer trace mining, so they are disclosed, not hidden under “ungraded”. dropped counts input rows that carried no usable signal and were discarded by normalization.

​otel

​rows_from_otel

​platform

​agents

​catalog

​datasets

​delete_dataset

​delete_empty_datasets

​hf_publish

​hf_publish_run

​hf_status

​import_hf

​issue_delegated_credential

​preview

​profile

​publish

​pull

​purge_agent

​push_file

​push_rows

​refresh_delegated_credential

​register_agent

​unpublish

​update_dataset

​traces

​dimensions_from_traces

​drop_leaky_rows

​flaw_rows

​format_trace_report

​infer_harness

​leakage_report

​load_traces

​mine_traces

​opening_share

​simulate_from_traces

​split_pseudo_production

​tools_from_traces

​trace_report