Skip to main content
Post-training offers three training methods, each learning from a different signal: example responses, preference pairs, or scores from a reward function you write. Each method can train a low-rank adapter (LoRA) or the full model weights.
Fine-tuning jobs also run supervised fine-tuning and preference optimization, with no training loop to write. To choose between the two services, see Post-training vs. standard fine-tuning.

Choose a training method

Choose a method based on the training data you can produce: The methods also differ in what they can teach and what they cost:
  • SFT: It’s the simplest method to run, because it needs no reward function. It can teach only what your demonstrations show. To improve accuracy beyond them on a task you can score, continue training with GRPO.
  • DPO: Use it for behavior that’s easier to judge than to demonstrate, where you can say which of two responses is better but can’t write the ideal one.
  • GRPO: Use it for tasks where your code can check an answer, such as math with a known solution or code with tests, and for agents that act over several turns and call tools. It’s the only method that samples while it trains, so its model resource needs at least one generator replica, which adds GPUs and cost. SFT and DPO run on trainer-only resources.

Combine methods

To run methods in sequence, hand off through a training checkpoint. Each phase runs in its own session, typically on a new model resource for the same base model, because a GRPO phase needs a generator replica. Keep the base model and the training mode the same in every phase. A training checkpoint from a LoRA session can’t start a full-weight session, or the reverse. The handoff takes three steps:
  1. Save a training checkpoint at the last step of the first run.
  2. Start the next session from that checkpoint with load_optimizer=False. The new session loads the adapter weights with a fresh optimizer and its step counter at zero.
  3. Use the same LoRA rank and alpha in both phases. A different rank makes the checkpoint fail to load, and a different alpha silently changes how strongly the adapter applies.
A common combination teaches the output format with SFT, then optimizes accuracy with GRPO. The format matters because GRPO learns only from groups whose completions score differently. If the base model rarely produces a correct answer, most groups score the same and train nothing. A short SFT run on correctly formatted examples gets the rewards to vary, so GRPO has a signal from its first steps. Chain SFT and GRPO shows the handoff with cookbook recipes. The same handoff works from SFT into DPO. DPO measures each update against the model as it was when the DPO run started, so after an SFT handoff, the SFT model is the reference.