How Together post-training runs SFT
SFT never samples from the model, so it runs on a trainer-only model resource (num_generator_replicas=0). Each step runs two operations:
forward_backward: WithLOSS_TYPE_CROSS_ENTROPY, the trainer computes the weighted next-token cross-entropy over the batch and accumulates gradients.optim_step: The optimizer applies the accumulated gradients with the step’s learning rate.
Sample objects that hold input tokens, next-token targets, and loss weights. Your code controls data loading, preprocessing, tokenization, and which tokens train, so any dataset schema works.
Loss masks
A loss mask sets how much each target token in a sample contributes to the cross-entropy loss. It holds onefloat32 weight per target token. A weight of 0 excludes the token, 1 trains it fully, and a fractional value reduces its contribution.
In a chat conversation, the mask gives 0 to the prompt content and role headers, including the assistant header. It gives 1 to the content and end-of-turn tokens of each assistant turn you train on. Training on end-of-turn tokens teaches the model when to stop. Samples and loss masks explains token alignment and how to build a mask.
Next steps
SFT recipes
Run SFT from a cookbook recipe on the No Robots dataset or your own conversations.
Write a training loop
Train on batches of your own samples with the Python SDK.
Combine methods
Hand off an SFT run to GRPO or DPO through a training checkpoint.
Samples and loss masks
Build samples and loss masks from chat conversations.