Skip to main content
Supervised fine-tuning (SFT) trains a model to imitate example responses token by token, so it picks up their content, format, and style. Use it when you have high-quality demonstrations of the behavior you want, such as human-written answers, responses in a target output format, or conversations from a system that already works. Fine-tuning jobs also run SFT, with a fixed data format and no training loop to write. To choose between them, see Post-training vs. standard fine-tuning.

How Together post-training runs SFT

SFT never samples from the model, so it runs on a trainer-only model resource (num_generator_replicas=0). Each step runs two operations:
  1. forward_backward: With LOSS_TYPE_CROSS_ENTROPY, the trainer computes the weighted next-token cross-entropy over the batch and accumulates gradients.
  2. optim_step: The optimizer applies the accumulated gradients with the step’s learning rate.
The next step trains directly on the updated weights. The service receives tokenized Sample objects that hold input tokens, next-token targets, and loss weights. Your code controls data loading, preprocessing, tokenization, and which tokens train, so any dataset schema works.

Loss masks

A loss mask sets how much each target token in a sample contributes to the cross-entropy loss. It holds one float32 weight per target token. A weight of 0 excludes the token, 1 trains it fully, and a fractional value reduces its contribution. In a chat conversation, the mask gives 0 to the prompt content and role headers, including the assistant header. It gives 1 to the content and end-of-turn tokens of each assistant turn you train on. Training on end-of-turn tokens teaches the model when to stop. Samples and loss masks explains token alignment and how to build a mask.

Next steps

SFT recipes

Run SFT from a cookbook recipe on the No Robots dataset or your own conversations.

Write a training loop

Train on batches of your own samples with the Python SDK.

Combine methods

Hand off an SFT run to GRPO or DPO through a training checkpoint.

Samples and loss masks

Build samples and loss masks from chat conversations.