> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Cookbook config reference

> Look up the type and default of every field on the post-training cookbook's GRPO, SFT, and DPO config classes.

Every [cookbook recipe](https://docs.together.ai/docs/post-training/cookbook-overview.md#recipes) passes a config object to a training function. This page lists every field on the cookbook's three config classes: `GRPOConfig` for group relative policy optimization (GRPO), `SFTConfig` for supervised fine-tuning (SFT), and `DPOConfig` for direct preference optimization (DPO). For the API fields behind them and the limits that apply, see the [post-training API reference](https://docs.together.ai/reference/post-training/model-resources-create.md) and [Limits](https://docs.together.ai/docs/post-training/limits-and-errors.md#limits).

The cookbook renders prompts in each model family's chat format through a codec for that family, and loads each model's tokenizer from Hugging Face by its catalog ID. [Supported models](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates) lists each catalog model's codec. It also names the models that need `HF_TOKEN` because their Hugging Face repository is gated.

## Load a config from YAML

Each config class is a Python dataclass, so recipes set fields as keyword arguments. Each class also loads from a YAML file whose keys are the field names:

```python theme={null}
from post_training_cookbook.grpo.config import GRPOConfig

config = GRPOConfig.from_yaml("config.yaml")
```

`from_yaml` raises `ValueError` on any key that isn't a field. Every run writes its config to `runs/<run_name>/config.yaml`, which loads back the same way.

## Shared fields

All three config classes take these fields. Where a default differs between classes, the table lists each one.

| Field | Type | Default | Description |
| - | - | - | - |
| `use_lora` | `bool` | `True` | `True` trains a low-rank adapter (LoRA). `False` trains the full weights on a resource created with `lora_enabled=False`. Not every model supports full-weight training. See [Supported models](https://docs.together.ai/docs/post-training/supported-models.md). |
| `lora_rank` | `int \| None` | `None` | Adapter rank. `None` uses the server default of `32`. |
| `lora_alpha` | `int \| None` | `None` | Adapter scaling factor. `None` uses the server default of `64`. |
| `lora_dropout` | `float \| None` | `None` | Dropout on the adapter layers. `None` uses the server default of `0.0`. |
| `adam_beta1` | `float` | `0.9` | AdamW first-moment decay. |
| `adam_beta2` | `float` | `0.95` | AdamW second-moment decay. |
| `adam_eps` | `float` | `1e-8` | AdamW epsilon. |
| `weight_decay` | `float` | `0.01` (GRPO), `0.1` (SFT, DPO) | Weight decay coefficient. Under Muon, it applies to both the Muon and AdamW parameter groups. |
| `training_checkpoint_interval` | `int` | `0` | Steps between training checkpoints. `0` saves none. To save one at the end, set it to the step count. |
| `inference_checkpoint_interval` | `int` | `0` | Steps between intermediate inference checkpoints. The final step always saves one. |
| `resume_from_checkpoint_id` | `str \| None` | `None` | ID of the training checkpoint to start the session from. |
| `load_optimizer` | `bool \| None` | `None` | Whether to restore the optimizer state from `resume_from_checkpoint_id`. `None` or `True` restores it. `False` loads the weights only and starts a new run. To continue a GRPO run, also set `resume_run_dir`. SFT and DPO restart the learning-rate schedule at step zero either way. |
| `wandb_project` | `str \| None` | `"post-training-cookbook"` (GRPO), `None` (SFT, DPO) | Weights & Biases (W\&B) project. `None` turns off W\&B logging. A run that logs needs a W\&B API key or `WANDB_MODE=offline`. Without either, it fails right after it provisions the resource. |
| `wandb_entity` | `str \| None` | `None` | W\&B team or username. |
| `wandb_run_name` | `str \| None` | `None` | W\&B run name. It also names the session and the run directory. `None` generates a name. |
| `wandb_group` | `str \| None` | `None` | W\&B group, for comparing related runs such as the arms of a sweep. |
| `timeout` | `float` | `7200.0` | Timeout in seconds for each SDK call the loop makes. |

## `GRPOConfig`

Configures `train_single_turn` and `train_multi_turn` in `post_training_cookbook.grpo.train`. Set exactly one of `model` and `model_resources_id`. It also takes the [fields every config shares](#shared-fields).

### Model and resource

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str \| None` | `None` | Catalog model ID to provision a new model resource for. The run stops the resource when it exits. |
| `model_resources_id` | `str \| None` | `None` | ID of an existing model resource to attach a session to. The resource defines the base model and keeps running after the run exits. |
| `num_generator_replicas` | `int` | `1` | Number of generator replicas to provision, at least `1`. Ignored when attaching to an existing resource. |

### Schedule

| Field | Type | Default | Description |
| - | - | - | - |
| `batch_size` | `int` | `16` | Prompts per step for single-turn runs, or tasks per step for multi-turn runs. |
| `num_steps` | `int \| None` | `None` | Total training steps, including any steps a continued run already completed. `None` runs `num_epochs` passes over the training examples, or one pass over the tasks for multi-turn runs. |
| `learning_rate` | `float` | `1e-5` | Peak learning rate. |
| `warmup_ratio` | `float` | `0.05` | Fraction of steps over which the learning rate rises linearly to its peak. |
| `min_lr_ratio` | `float` | `0.1` | Final learning rate as a fraction of the peak, at the end of the cosine decay. |

### Sampling and loss

| Field | Type | Default | Description |
| - | - | - | - |
| `temperature` | `float` | `0.7` | Sampling temperature for training completions. |
| `max_sampling_attempts` | `int` | `1` | Maximum batches a step draws to collect `batch_size` groups with varied rewards. `1` turns dynamic sampling off. |
| `loss` | `str` | `"grpo"` | Built-in loss function for each step: `"grpo"`, `"ppo"`, `"cispo"`, `"dppo"`, `"dro"`, or `"importance_sampling"`. See [Loss variants](https://docs.together.ai/docs/post-training/grpo-recipes.md#loss-variants). |
| `clip_low_threshold` | `float \| None` | `None` | Lower bound on the importance ratio, as an absolute value. Applies only to `"grpo"`, `"ppo"`, and `"cispo"`, and is rejected with any other loss function. `None` uses the server default, which is `0.8` for `"grpo"` and `"ppo"`, or `0` for `"cispo"`. Must be in `[0, 1]` for `"grpo"` and `"ppo"`, and at least `0` for `"cispo"`. |
| `clip_high_threshold` | `float \| None` | `None` | Upper bound on the importance ratio, as an absolute value. Applies only to `"grpo"`, `"ppo"`, and `"cispo"`, and is rejected with any other loss function. `None` uses the server default, which is `1.2` for `"grpo"` and `"ppo"`, or `4` for `"cispo"`. Must be at least `1` for `"grpo"` and `"ppo"`, and at least `clip_low_threshold` for `"cispo"`. |
| `loss_agg_type` | `str` | `"GRPO_LOSS_AGGREGATION_TYPE_FIXED_HORIZON"` | How per-token losses reduce to a scalar. Applies only to `"grpo"`. See [Loss variants](https://docs.together.ai/docs/post-training/grpo-recipes.md#loss-variants). |
| `ratio_type` | `str` | `"GRPO_LOSS_RATIO_TYPE_TOKEN"` | Importance-ratio granularity. `"GRPO_LOSS_RATIO_TYPE_SEQUENCE"` is GSPO. Applies only to `"grpo"`. |
| `dro_beta` | `float \| None` | `None` | Coefficient on DRO's log-ratio penalty. Required when `loss="dro"`, and rejected otherwise. |

### Token budgets

For single-turn runs, set `max_sample_tokens` or `max_prompt_length`. If both stay at their `None` defaults, the prompt budget is `0` and every prompt is dropped.

| Field | Type | Default | Description |
| - | - | - | - |
| `max_model_length` | `int` | `2048` | Maximum prompt plus completion length, in tokens. |
| `max_sample_tokens` | `int \| None` | `None` | Maximum tokens generated per completion. `None` uses `max_model_length`. |
| `min_response_tokens` | `int` | `1` | Minimum completion budget a prompt must leave room for. |
| `overlong_penalty_factor` | `float` | `1.0` | Scale of the soft penalty on completions that approach the budget. `0` turns the penalty off. |
| `overlong_buffer_len` | `int \| None` | `None` | Tokens before the budget at which the penalty starts. `None` uses `max_sample_tokens // 8`. |
| `overlong_masking` | `bool` | `False` | `True` skips truncated completions when building training samples. Works independently of the penalty. |

### Thinking

`SFTConfig` also takes these three fields. `DPOConfig` takes only `enable_thinking`.

[Supported models](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates) lists each model's thinking default, effort levels, and whether its template keeps earlier reasoning.

| Field | Type | Default | Description |
| - | - | - | - |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. `None` uses the chat template's default. On a model that always thinks, or whose template has no switch, `False` raises `ValueError`. On an adaptive-only template, `True` and `False` have no effect. |
| `thinking_effort` | `str \| None` | `None` | Reasoning budget. `None` uses the chat template's default. Setting it on a model that doesn't support it, or to a level the model doesn't accept, raises `ValueError`. |
| `preserve_thinking` | `bool \| None` | `None` | Whether earlier turns keep their reasoning when the history is rendered. |

Each `preserve_thinking` value sets how the history is rendered and [how many training sequences a conversation becomes](https://docs.together.ai/docs/post-training/concepts.md#chat-templates-and-thinking):

| Value | History rendering | Training sequences |
| - | - | - |
| `None` | The chat template's own behavior. Some templates strip earlier reasoning, and others keep it. | One sequence per turn wherever stripping changes the history. |
| `False` | Strips earlier reasoning on every model. Same as `None` on templates that strip it. | Same as `None`. |
| `True` | Keeps earlier reasoning on every model. Requires `enable_thinking` to be `None` or `True`. | Usually one sequence per conversation, which is cheaper. |

Set `preserve_thinking` to match inference. Use `True` only when the model also sees its earlier reasoning at inference. Otherwise, the run trains on history that inference never reproduces.

### Optimizer

| Field | Type | Default | Description |
| - | - | - | - |
| `optimizer` | `"adam" \| "muon"` | `"adam"` | Optimizer for the weight matrices. `"muon"` requires `use_lora=False`. |
| `muon_scaling_strategy` | `str \| None` | `None` | `"MUON_SCALING_STRATEGY_MATCH_ADAM"` or `"MUON_SCALING_STRATEGY_ORIGINAL"`. `None` uses the server default. |
| `muon_momentum` | `float \| None` | `None` | Muon momentum coefficient. `None` uses the server default. |
| `muon_newton_schulz_steps` | `int \| None` | `None` | Newton-Schulz iterations per Muon update. `None` uses the server default. |

Under Muon, the [shared](#shared-fields) AdamW fields still apply to the parameters Muon doesn't optimize: embeddings, norm layers, and the output head.

### Weight sync

| Field | Type | Default | Description |
| - | - | - | - |
| `weight_sync_type` | `str` | `"WEIGHT_SYNC_TYPE_SYNCHRONOUS"` | How new weights reach the generators after each step. Every mode except `"WEIGHT_SYNC_TYPE_SYNCHRONOUS"` requires `max_steps_off_policy`. See [Async training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training). |

### Validation

| Field | Type | Default | Description |
| - | - | - | - |
| `validation_interval` | `int` | `50` | Steps between validation runs. The loop also validates once before the first step. |
| `val_temperature` | `float \| None` | `None` | Sampling temperature for validation. `None` samples greedily. |

### Checkpoints and resume

| Field | Type | Default | Description |
| - | - | - | - |
| `resume_run_dir` | `str \| None` | `None` | Directory of the run being continued. Required when continuing, and rejected otherwise. See [Continue a run](https://docs.together.ai/docs/post-training/grpo-recipes.md#continue-a-run). |

### Logging

| Field | Type | Default | Description |
| - | - | - | - |
| `export_samples` | `bool` | `True` | Writes each step's completions or episodes to JSON files in the run directory. |

### Single-turn only

Multi-turn runs ignore these fields.

| Field | Type | Default | Description |
| - | - | - | - |
| `dataset` | `str` | `"gsm8k"` | Name of a dataset in the cookbook's registry. See [Built-in datasets](https://docs.together.ai/docs/post-training/grpo-recipes.md#built-in-datasets). |
| `num_examples` | `int` | `7000` | Training examples drawn from the dataset. |
| `num_val_examples` | `int` | `200` | Validation examples, drawn from the rows after the training examples. `0` turns validation off. |
| `group_size` | `int` | `8` | Completions sampled per prompt. |
| `num_epochs` | `int` | `1` | Passes over the training examples when `num_steps` is `None`. |
| `max_prompt_length` | `int \| None` | `None` | Maximum prompt length, in tokens. Longer prompts are dropped before training. `None` uses `max_model_length - max_sample_tokens`. |

### Multi-turn only

Single-turn runs ignore these fields, except `max_steps_off_policy`, which `train_single_turn` rejects.

| Field | Type | Default | Description |
| - | - | - | - |
| `max_turns` | `int \| None` | `None` | Maximum assistant turns per episode. Required by `train_multi_turn`. |
| `episodes_per_task` | `int` | `1` | Episodes per task per step. Set it to `2` or more, because a single episode has nothing to compare against, so it never trains. |
| `failed_parse_reward` | `float` | `-0.1` | Reward for an episode whose sampled message can't be parsed. |
| `val_max_turns` | `int \| None` | `None` | Turn limit during validation. `None` uses `max_turns`. |
| `validation_iterations` | `int` | `1` | Validation passes per validation run, averaged together. |
| `sequential_validation` | `bool` | `False` | `True` runs validation episodes one at a time instead of concurrently. |
| `max_steps_off_policy` | `int \| None` | `None` | How many steps old a rollout group can be and still train. An integer turns on [async training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training). `None` collects in lockstep. |
| `groups_in_flight` | `int \| None` | `None` | Rollout groups collected concurrently under [async training](https://docs.together.ai/docs/post-training/multi-turn-recipes.md#async-training). `None` uses `batch_size`. |

## `SFTConfig`

Configures `train` in `post_training_cookbook.sft.train`. SFT runs on a trainer-only resource, so it has no generator settings. Set exactly one of `model` and `model_resources_id`. It also takes the [fields every config shares](#shared-fields).

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str \| None` | `None` | Catalog model ID to provision a new trainer-only resource for. |
| `model_resources_id` | `str \| None` | `None` | ID of an existing model resource to attach a session to. |
| `dataset_name` | `str` | `"HuggingFaceH4/no_robots"` | Hugging Face dataset repo ID, or the path to a local directory of `.jsonl` files. Rows need a `messages` column unless you pass `to_messages` to `train`. See [Use your own dataset](https://docs.together.ai/docs/post-training/sft-recipes.md#use-your-own-dataset). |
| `dataset_config` | `str \| None` | `None` | Hugging Face dataset config name. |
| `dataset_split` | `str` | `"train"` | Dataset split to train on. |
| `batch_size` | `int` | `8` | Conversations per step. |
| `num_epochs` | `int` | `1` | Passes over the split. Each pass reshuffles the rows and drops the final partial batch. |
| `max_steps` | `int \| None` | `None` | Cap on total training steps. `None` trains on every full batch of every epoch. |
| `learning_rate` | `float` | `1e-4` | Starting learning rate, which decays linearly toward zero over the run. |
| `max_length` | `int` | `4096` | Maximum tokens per training sequence. |
| `train_on_what` | `str` | `"all_assistant_messages"` | Which assistant messages to train on, either `"all_assistant_messages"` (`TRAIN_ON_ALL`) or `"last_assistant_message"` (`TRAIN_ON_LAST`). Any other value raises `ValueError` when you build the config. |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. For data without reasoning, the value to use [depends on the model](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates). |
| `thinking_effort` | `str \| None` | `None` | Reasoning budget, on models that support it. See [Thinking](#thinking). |
| `preserve_thinking` | `bool \| None` | `None` | Whether earlier turns keep their reasoning when the history is rendered. See [Thinking](#thinking). |

`train` also takes a keyword-only `to_messages` function, which maps a dataset row to its list of messages. `train` returns the ID of the latest training checkpoint, or `None` if the run saved none.

## `DPOConfig`

Configures `train` in `post_training_cookbook.dpo.train`. `DPOConfig` has no `model_resources_id` field, so every DPO run provisions its own trainer-only resource. It also takes the [fields every config shares](#shared-fields).

| Field | Type | Default | Description |
| - | - | - | - |
| `model` | `str` | `"Qwen/Qwen3.8-27B"` | Catalog model ID to provision a trainer-only resource for. |
| `dataset_name` | `str` | `"nvidia/HelpSteer3"` | Hugging Face dataset repo ID. With the default `to_pair`, rows need HelpSteer3's `context`, `response1`, `response2`, and `overall_preference` columns. See [DPO recipes](https://docs.together.ai/docs/post-training/dpo-recipes.md). |
| `dataset_config` | `str \| None` | `None` | Hugging Face dataset config name. |
| `dataset_split` | `str` | `"train"` | Dataset split to train on. |
| `dataset_subsample` | `int \| None` | `2000` | Maximum dataset rows to load, after a seeded shuffle. `None` uses the whole split. Two filters run after loading, so training gets fewer pairs than this. Rows that `to_pair` skips, such as HelpSteer3 ties, are dropped. Pairs longer than `max_length` are truncated, and dropped only if a side has no trained token left. |
| `val_split` | `str` | `"validation"` | Dataset split to validate on. |
| `val_subsample` | `int \| None` | `100` | Maximum validation rows to load, before rows that `to_pair` skips are dropped. `None` uses the whole split. `0` turns validation off. |
| `val_interval` | `int` | `100` | Steps between validation runs. `0` or less turns validation off. |
| `batch_size` | `int` | `8` | Preference pairs per step. |
| `max_steps` | `int` | `1000` | Total optimizer steps. |
| `learning_rate` | `float` | `5e-6` | Peak learning rate. |
| `warmup_ratio` | `float` | `0.03` | Fraction of steps over which the learning rate rises linearly to its peak. |
| `min_lr_ratio` | `float` | `0.1` | Final learning rate as a fraction of the peak, at the end of the cosine decay. |
| `max_length` | `int` | `2048` | Maximum tokens per chosen or rejected sequence. |
| `enable_thinking` | `bool \| None` | `None` | Turns the model's `<think>` output on or off. For preference data without reasoning, such as HelpSteer3, the value to use [depends on the model](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates). |
| `beta` | `float` | `0.1` | Temperature on the implicit reward. Higher values keep the policy closer to the reference. |
| `label_smoothing` | `float` | `0.0` | Conservative DPO label smoothing. |
| `precompute_ref_batch_size` | `int` | `128` | Batch size for the reference log-probability passes before training and for each validation pass. |

`train` also takes a keyword-only `to_pair` function, which maps a dataset row to a `(chosen_messages, rejected_messages)` tuple, or to `None` to skip the row. The default, `helpsteer3_pair`, reads HelpSteer3's columns. Pass your own function to [train on another preference dataset](https://docs.together.ai/docs/post-training/dpo-recipes.md#use-your-own-preference-dataset).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.