whileai.simulations.export

​

export_dataset

export_dataset(
    source,
    output: str | None = None,
    system_prompt: str | None = None,
    tools: Sequence[dict] | None = None,
    strip_think: bool = True,
    validate: bool = True,
    mask_mode: str = 'assistant',
    unroll: bool = False,
    max_tool_output_chars: int | None = None,
    format: str = 'openai',
    push_to: str | None = None,
) -> dict[str, Any]

• source: a SimulationData (system prompt and tools come from its profile), a row list, or a JSONL path; for the last two pass system_prompt and tools.
• output: the file to write. With a path source and no output it writes <name>.train.jsonl next to it. It never overwrites the source.
• validate (True): refuse to write a dataset whose tool calls do not round-trip to structured arguments, or whose assistant turns quote the row’s own privileged block (the export scrubs the key, not the reply that recited it); False exports anyway and leaves the report to read. The leak check reads the source before the scrub, so pass the SimulationData or its trajectories; rows that already came through rows(), save() or a file carry nothing to check, and report["privileged_leaks"] says so.
• format: "openai" (the default) writes the OpenAI chat-completions wire row: the full messages list, function.arguments as a JSON string, and the ask carried alongside as prompt. "trl" writes what trl can actually load: conversational SFT rows (arguments as dicts, the ask under prompt_text). TRL decides “is this conversational?” from the column set, so a prompt string beside messages makes it skip the chat template without an error and train on the bare ask, which is why the TRL rows do not carry one. The report says which format and which argument encoding the round-trip gate checked.
• mask_mode with format="trl": the TRL rows carry no loss_mask, because trl 0.19.1’s SFTTrainer never reads one. Its collator (DataCollatorForLanguageModeling) labels every token and unlabels only from two token-level columns the trainer builds itself: completion_mask from prompt/completion rows, and assistant_masks from assistant_only_loss=True, which needs a {% generation %} block in the chat template (Qwen2.5-Instruct has none). So mask_mode="final" and unroll=True write prompt/completion rows (prompt up to the last assistant turn, that turn as the completion), and TRL trains on exactly what the mask asked. mask_mode="assistant" (the default) writes messages rows, and TRL trains on every token of them unless assistant_only_loss=True is set. The report’s mask_mode states which of the two TRL will do, and trained_messages / masked_messages count what TRL trains, not what the SDK intended; a format="trl" export on 94 rows was measured at 9x the tokens its loss_mask marked (#507).
• push_to: a Hub repo ("me/my-set") to upload the written file to, with your own token: HF_TOKEN from the environment or the login hf auth login cached, through huggingface_hub (pip install 'whileai[hf]'). Private by default; no platform call. wai.hub.push is the same upload for a file, a directory or rows you already hold, with token= and private=. The report gains hub (repo_id, url, commit).
• strip_think, mask_mode, unroll, max_tool_output_chars: passed through to training_rows, which explains each.

report = wai.export_dataset(data, "train.jsonl", format="trl")
print(report["n"], report["mask_mode"], report["tool_call_roundtrip"])

​

export_preference

export_preference(
    pairs: Sequence[dict],
    output: str | None = None,
    system_prompt: str | None = None,
    tools: Sequence[dict] | None = None,
    strip_think: bool = True,
    validate: bool = True,
    drop_ties: bool = True,
    format: str = 'openai',
) -> dict[str, Any]

​

export_training

export_training(
    source,
    output: str | None = None,
    system_prompt: str | None = None,
    tools: Sequence[dict] | None = None,
    strip_think: bool = True,
    validate: bool = True,
    mask_mode: str = 'assistant',
    unroll: bool = False,
    max_tool_output_chars: int | None = None,
    format: str = 'openai',
    push_to: str | None = None,
) -> dict[str, Any]

• source: a SimulationData (system prompt and tools come from its profile), a row list, or a JSONL path; for the last two pass system_prompt and tools.
• output: the file to write. With a path source and no output it writes <name>.train.jsonl next to it. It never overwrites the source.
• validate (True): refuse to write a dataset whose tool calls do not round-trip to structured arguments, or whose assistant turns quote the row’s own privileged block (the export scrubs the key, not the reply that recited it); False exports anyway and leaves the report to read. The leak check reads the source before the scrub, so pass the SimulationData or its trajectories; rows that already came through rows(), save() or a file carry nothing to check, and report["privileged_leaks"] says so.
• format: "openai" (the default) writes the OpenAI chat-completions wire row: the full messages list, function.arguments as a JSON string, and the ask carried alongside as prompt. "trl" writes what trl can actually load: conversational SFT rows (arguments as dicts, the ask under prompt_text). TRL decides “is this conversational?” from the column set, so a prompt string beside messages makes it skip the chat template without an error and train on the bare ask, which is why the TRL rows do not carry one. The report says which format and which argument encoding the round-trip gate checked.
• mask_mode with format="trl": the TRL rows carry no loss_mask, because trl 0.19.1’s SFTTrainer never reads one. Its collator (DataCollatorForLanguageModeling) labels every token and unlabels only from two token-level columns the trainer builds itself: completion_mask from prompt/completion rows, and assistant_masks from assistant_only_loss=True, which needs a {% generation %} block in the chat template (Qwen2.5-Instruct has none). So mask_mode="final" and unroll=True write prompt/completion rows (prompt up to the last assistant turn, that turn as the completion), and TRL trains on exactly what the mask asked. mask_mode="assistant" (the default) writes messages rows, and TRL trains on every token of them unless assistant_only_loss=True is set. The report’s mask_mode states which of the two TRL will do, and trained_messages / masked_messages count what TRL trains, not what the SDK intended; a format="trl" export on 94 rows was measured at 9x the tokens its loss_mask marked (#507).
• push_to: a Hub repo ("me/my-set") to upload the written file to, with your own token: HF_TOKEN from the environment or the login hf auth login cached, through huggingface_hub (pip install 'whileai[hf]'). Private by default; no platform call. wai.hub.push is the same upload for a file, a directory or rows you already hold, with token= and private=. The report gains hub (repo_id, url, commit).
• strip_think, mask_mode, unroll, max_tool_output_chars: passed through to training_rows, which explains each.

report = wai.export_dataset(data, "train.jsonl", format="trl")
print(report["n"], report["mask_mode"], report["tool_call_roundtrip"])

​

loss_mask

loss_mask(messages: Sequence[dict], mode: str = 'assistant') -> list[int]

​

to_trl

to_trl(rows: Sequence[dict], kind: str = 'training') -> list[dict]

​

tool_call_roundtrip

tool_call_roundtrip(
    rows: Sequence[dict],
    format: str = 'openai',
) -> dict[str, Any]

• format="openai" (encoding: "json_string"): arguments must parse back to a dict. Arguments that survive as un-parseable strings get re-quoted by chat templates and teach the model to emit string-wrapped arguments, which then spiral on tool rejections.
• format="trl" (encoding: "dict"): arguments must already be dicts. A JSON string here is valid OpenAI wire and still wrong for a chat template, which would render it quoted twice, so it counts as invalid rather than passing on a technicality.

​

training_rows

training_rows(
    source,
    system_prompt: str | None = None,
    tools: Sequence[dict] | None = None,
    strip_think: bool = True,
    mask_mode: str = 'assistant',
    unroll: bool = False,
    max_tool_output_chars: int | None = None,
) -> list[dict]

• source: a SimulationData (system prompt and tools come from its profile), a row list, or a JSONL path. For lists and paths, pass system_prompt= and tools= explicitly; a row exported without its policy trains an agent that never saw its rules.
• strip_think (True): remove <think> blocks from the assistant turns, so a thinking rollout model never teaches a non-thinking student to emit them. On a reasoning base such as Qwen3 that teaches the adapter to emit an empty <think></think> and answer at once, so at eval it answers while the untrained base is still reasoning under the same max_tokens. Pass strip_think=False when the student should keep reasoning, and set thinking= the same on both arms of the eval either way.
• mask_mode ("assistant"): which assistant turns carry loss (see loss_mask): all of them, or "final" for the last turn only.
• unroll (False): True turns an N-turn conversation into N samples, the k-th ending at the k-th assistant turn with loss on that turn only (Lambert 2025, chapter Instruction Tuning). Every earlier agent turn then trains once with exactly the context it had, instead of only the last one (mask_mode="final") or all of them at once ("assistant", where later turns see context the policy never produced). Each sample carries unroll (turn, turns) and lineage.unrolled_from (the source row’s prompt hash and rollout index); mask_mode is ignored, and no group fields are stamped, since samples of one conversation are not a GRPO group.
• max_tool_output_chars (None, cut nothing): cap each tool message at that many characters, appending [... N chars of tool output truncated] and counting the cut on the row as tool_output_truncated (messages) and tool_output_chars_cut. Tool output is masked from the loss anyway; what it costs is context, and the cut is explicit rather than silent (Lambert 2025, chapter Tool Use).

rows = wai.training_rows(data, unroll=True)
print(rows[0]["messages"][0]["role"], rows[0]["loss_mask"])