whileai.simulations.schema

10 public names. import whileai.simulations as wai, then wai.name.

Name	What it does
`Calibration`	Measured difficulty of one task for one student.
`Dataset`	Split membership is a dataset decision, so one task can be holdout in one dataset and training in another.
`Judgment`	no docstring
`Marker`	no docstring
`Rollout`	One episode.
`Task`	The situation.
`calibration_of`	The typed `Calibration` a row carries, or `None` when absent or malformed.
`from_row`	Split one flat row into its four objects.
`to_row`	The flat v1 wire row.
`validate`	Problems with one row, empty when it is fine.

Calibration

class Calibration(
    task_id: str,
    student: PolicyRef,
    n: int,
    pass_rate: float,
    mean_kl: float | None = None,
    pass_rate_ci95: tuple[float, float] | None = None,
) -> None

Defined in whileai/simulations/schema.py. Measured difficulty of one task for one student. Optional; only calibrate produces it. pass_rate_ci95 is the Wilson 95% interval on pass_rate from n rollouts (about +/-0.3 wide at n=8), so a band assignment can be read with its uncertainty. mean_kl is the sampled KL to a reference policy per generated token; simulate(logprobs=True) captures the student side and calibrate(rows, ref=...) fills it in.

Dataset

class Dataset(
    dataset_id: str,
    spec_id: str = '',
    splits: dict[str, tuple[str, ...]] = <factory>,
) -> None

Defined in whileai/simulations/schema.py. Split membership is a dataset decision, so one task can be holdout in one dataset and training in another.

Judgment

class Judgment(
    rollout_id: str,
    scorer: ScorerRef,
    reward: float | None,
    status: "Literal['ok', 'missing_reward', 'invalid_result', 'error', 'timeout']" = 'ok',
    reason: str = '',
    failure_class: str | None = None,
    evidence: dict = <factory>,
) -> None

Defined in whileai/simulations/schema.py.

Marker

class Marker(
    rollout_id: str,
    name: str,
    value: float,
    evidence: dict = <factory>,
) -> None

Defined in whileai/simulations/schema.py.

Rollout

class Rollout(
    rollout_id: str,
    task_id: str,
    policy: PolicyRef = <factory>,
    index: int = 0,
    steps: list[Step] = <factory>,
    final_text: str = '',
    ledger: list[FaultEvent] = <factory>,
    usable: bool = True,
    unusable_reason: str | None = None,
    extra: dict[str, Any] = <factory>,
) -> None

Defined in whileai/simulations/schema.py. One episode. Mutable so grading paths can attach to it in place; the no-verdict invariant is enforced at the boundary, not here.

Task

class Task(
    task_id: str,
    spec_id: str = '',
    prompt: str = '',
    prefix: tuple[Message, ...] = (),
    world: World = <factory>,
    privileged: Privileged = <factory>,
    axes: dict[str, Any] = <factory>,
    behaviors: tuple[str, ...] = (),
    lineage: Lineage = <factory>,
) -> None

Defined in whileai/simulations/schema.py. The situation. Identity-bearing, so nothing computed lives here: splits belong to a Dataset, difficulty to a Calibration.

calibration_of

calibration_of(row: dict) -> Calibration | None

Defined in whileai/simulations/schema.py. The typed Calibration a row carries, or None when absent or malformed. publish_gate / calibrate write it as a flat dict under calibration; this is the read side. mean_kl is optional.

from_row

from_row(row: dict) -> tuple[Task, Rollout, list[Judgment], list[Marker]]

Defined in whileai/simulations/schema.py. Split one flat row into its four objects. Any version, any shape.

to_row

to_row(
    task: Task,
    rollout: Rollout,
    judgments: Sequence[Judgment] = (),
    markers: Sequence[Marker] = (),
) -> dict

Defined in whileai/simulations/schema.py. The flat v1 wire row. Inverse of from_row on engine rows; on other shapes it is the canonical row load_traces would produce, with the source row’s unknown keys carried along.

validate

validate(
    row: Any,
    kind: "Literal['row', 'training', 'preference']" = 'row',
) -> list[str]

Defined in whileai/simulations/schema.py. Problems with one row, empty when it is fine. Never raises. Stamped rows must carry the required fields with the right types. Unstamped kind="row" rows are version 0 and only have to be non-empty dicts: nothing that works today is rejected there. Two things are reported regardless of version, because “0 validation failures” is read as a guarantee and neither case is one:

an empty dict, which carries no prompt, no messages, no verdict;
a training or preference row with no messages or no chosen/rejected. Those two kinds ask “is this a training sample”, and an unstamped dict with no conversation in it is not one whatever version it claims.

​Calibration

​Dataset

​Judgment

​Marker

​Rollout

​Task

​calibration_of

​from_row

​to_row

​validate