> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Concepts

> Understand how a post-training run works, from model resources and sessions to checkpoints and deployment.

Together AI post-training splits each training run between a training loop you write and a model resource running on Together AI GPUs. This page explains how the two divide the work, introduces the model resources, sessions, samples, and checkpoints you work with, and covers how to deploy a trained model. To learn more about the available training methods, see [Training methods](https://docs.together.ai/docs/post-training/training-methods.md).

## How it works

The [model resource](#model-resources-and-sessions), a set of Together AI GPUs with the base model loaded, does all the heavy computation required for training: generating completions, computing gradients, and updating the weights.

Your training loop controls everything else: loading the data, choosing which prompts to train on, building each batch and setting its size, scoring the model's outputs, and deciding when to stop. The loop is either a [cookbook](https://docs.together.ai/docs/post-training/cookbook-overview.md) recipe or your own code built on the [SDK](https://docs.together.ai/docs/post-training/training-loop.md). It runs on any machine with the Together SDK installed, including a laptop with no GPU.

![Post-training architecture. Your code, built on the Together SDK, calls the training API, which routes each operation to a model resource: GPUs running the base model as trainer workers, which are always present, and generator replicas, which only RL uses. The model resource writes checkpoints to the checkpoint store as adapter or merged weights. From the checkpoint store, you can download the weights to your machine or deploy them to a dedicated endpoint for production.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/architecture.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=46cb1eccf00ce43e9184f74d63118a04)

The loop starts by creating a model resource (`create`) and attaching a training session to it (`create_session`). From then on, it calls the post-training API, which routes each operation to one of the model resource's [two roles](#trainer-and-generators): the trainer or, for reinforcement learning (RL), a generator replica. As it trains, the loop also saves [checkpoints](#checkpoints), which you can download or [deploy](#deployment).

For example, one step of an RL run looks like this:

1. Your code sends a batch of prompts (`sample_batch`), and the model resource generates several completions for each one.
2. Your code checks each completion against the known solution and scores it.
3. Your code sends the scored completions back (`forward_backward`), and the model resource computes gradients that push the model toward the higher-scoring completions.
4. The model resource updates the model's weights (`optim_step`).
5. The model resource publishes the new weights to its generator replicas (`weights_sync`), so the next step generates completions from the updated model.

Supervised fine-tuning (SFT) and direct preference optimization (DPO) train on responses you already have, so their steps skip generation and scoring. Your code sends a batch, and the model resource computes gradients and updates the weights, with no `sample_batch` or `weights_sync` call.

## Model resources and sessions

A *model resource* represents the set of GPUs allocated for training or sampling, plus the frozen base model loaded onto them. You choose its base model, training mode, GPU type, and number of [generator replicas](#trainer-and-generators) when you [create the resource](https://docs.together.ai/docs/post-training/model-resources.md). You pay for it by the minute while it's ready.

A *session* is one set of trainable weights attached to a resource: a low-rank adapter (LoRA) or, on a full-weight resource, the full model weights. Along with those weights, it holds the optimizer state and a step counter. Every operation you call (`sample`, `forward_backward`, `optim_step`, `weights_sync`) targets a session.

LoRA is the default training mode. A full-weight resource hosts a single session, which updates every parameter. [LoRA vs. full-weight training](https://docs.together.ai/docs/post-training/lora-vs-full-weight.md) compares the two modes.

Stopping a session only detaches it from the resource, which keeps running and billing until you stop it. If none of its sessions runs an operation within its [idle timeout](https://docs.together.ai/docs/post-training/model-resources.md#idle-timeout) (one hour by default), the resource stops on its own and its sessions expire. [Lifecycle](https://docs.together.ai/docs/post-training/limits-and-errors.md#lifecycle) lists every state and timer for resources and sessions.

### Trainer and generators

![The two roles on a model resource. forward\_backward and optim\_step go to the trainers, which compute the loss and gradients and run the optimizer. sample and weights\_sync go to the optional generator replicas, which sample from the current policy and receive the trainers' updated weights. SFT and DPO set num\_generator\_replicas to 0, and GRPO sets it to 1 or more.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/trainer-and-generators.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=a0530e2936ce1cd905b7e0bd90069af0)

The *trainer* runs every `forward_backward`, `optim_step`, and checkpoint save for every session on the resource. Each resource has at most one trainer, a single group of GPUs with the model sharded across them. It holds one copy of the frozen base weights and, on a LoRA resource, every session's adapter and optimizer state alongside them.

A *generator replica* is a unit of sampling capacity that serves `sample` calls. It runs one or more inference engines, each with its own copy of the base model and each session's adapter loaded, so one replica serves every session on the resource. Replicas are independent, so sampling throughput scales with the replica count. The catalog sets a replica's size for each model and GPU type. Each replica adds to the [resource's hourly rate](https://docs.together.ai/docs/post-training/pricing.md#estimate-the-cost-of-a-resource).

The two roles hold separate copies of the weights. The trainer's copy changes on every `optim_step`. The generators' copy stays the same until you call `weights_sync`, which publishes the trainer's current weights to every replica, so the next `sample` comes from the updated model.

SFT and DPO run on trainer-only resources with no generator replicas and never call `weights_sync`. A [sampling-only resource](https://docs.together.ai/docs/post-training/model-resources.md#sampling-only-resources) has generators but no trainer, and samples a saved inference checkpoint.

### Sharing a resource

![One model resource, made of GPUs and the frozen base weights, hosting three sessions that train adapters A, B, and C. All sessions share the base weights, which are never updated. A resource holds up to five sessions, and each session has its own LoRA adapter, data, hyperparameters, and checkpoints.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/sessions-share-a-resource.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=4247982d4af0b4f8c4f3e65359b347b2)

Because the base weights are frozen, [several LoRA sessions can share one model resource](https://docs.together.ai/docs/post-training/multi-lora.md). Each session trains its own adapter, with separate data, hyperparameters, and checkpoints, but you pay for only one model resource.

The number of LoRA sessions a resource accepts at once [depends on the model](https://docs.together.ai/docs/post-training/supported-models.md).

## Data and samples

A *sample* is one training example. It pairs a token sequence with the per-token values that the loss function reads, such as which tokens to train on.

Cookbook recipes build samples for you. When you write your own loop, you [build each sample's arrays](https://docs.together.ai/docs/post-training/samples.md) yourself.

## Chat templates and thinking

Every conversation becomes tokens through the model's chat template. These template behaviors decide whether a conversation trains as you expect:

* **Thinking defaults differ by model:** Some templates think by default, and some always think. A template that thinks by default can render the prompt of a conversation without reasoning differently from the start of the full conversation, so the conversation doesn't train. To train on data without reasoning, set `enable_thinking` to fit the model.
* **Reasoning belongs in `reasoning_content`:** Put assistant reasoning in that field, alongside `content`, not in `<think>` tags inside `content`. Qwen3.8 27B's template reads reasoning only from `reasoning_content`.
* **Some templates strip earlier reasoning:** When these templates render conversation history, they drop the `<think>` blocks from earlier assistant turns. The rendered history then no longer matches the tokens the model produced for those turns, so the conversation can't train as a single sequence. Split it into one sequence per turn instead, which trains each turn against the history the model sees at inference. The cookbook splits conversations for you.
* **Some templates end a turn with the next message's role marker:** When one assistant turn follows another, the first has no stop token, so merge or split consecutive assistant turns before training.

[Supported models](https://docs.together.ai/docs/post-training/supported-models.md#thinking-and-chat-templates) lists each model's template behavior. The cookbook's [thinking settings](https://docs.together.ai/docs/post-training/cookbook.md#thinking) control two of these behaviors: `enable_thinking` turns thinking on or off, and `preserve_thinking` sets whether earlier turns keep their reasoning.

## Checkpoints

A session saves two kinds of [checkpoint](https://docs.together.ai/docs/post-training/checkpoints.md):

* **Inference checkpoint:** Snapshots the model weights so you can download them. On most models, it also registers the adapter (for a LoRA session) or the full model (for a full-weight session) in your project's model registry, so you can deploy it.
* **Training checkpoint:** Saves the full training state (adapter or full weights, optimizer moments, and step counter), so a later session can resume the run or start a new one from the same weights.

## Deployment

After training, you can [deploy an inference checkpoint](https://docs.together.ai/docs/post-training/deployment.md) to Together's [dedicated model inference](https://docs.together.ai/docs/dedicated-endpoints/overview.md) platform.

You can also [download the weights](https://docs.together.ai/docs/post-training/checkpoints.md#download-the-weights) and run them somewhere else, such as on your own GPUs or with another provider. The merged variant is a standard Hugging Face model directory with safetensors weights, the model config, and the tokenizer, so you can serve it with any inference framework that supports the base model.

## Next steps

> [card] [SFT quickstart](https://docs.together.ai/docs/post-training/quickstart-sft.md)
>
> Run your first supervised fine-tuning session from a cookbook recipe.

> [card] [GRPO quickstart](https://docs.together.ai/docs/post-training/quickstart-rl.md)
>
> Train a model on grade-school math with GRPO and an exact-match reward.

> [card] [Post-training cookbook](https://docs.together.ai/docs/post-training/cookbook-overview.md)
>
> Run ready-made training recipes, or reuse the cookbook's helpers in your own loop.

> [card] [Post-training SDK](https://docs.together.ai/docs/post-training/training-loop.md)
>
> Call post-training operations directly from your own Python code.

> [card] [Manage model resources](https://docs.together.ai/docs/post-training/model-resources.md)
>
> Choose a configuration, then create, attach to, and stop a resource.

> [card] [Manage checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md)
>
> Save, resume from, and download checkpoints.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.