> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Train models on Together AI GPUs from a training loop you build with the Python SDK or the post-training cookbook.

Together AI post-training lets you fine-tune models on Together AI GPUs with your own training loop. You provision a model resource, attach a training session to it, and call the service at each step to sample completions, compute a loss, and update the model weights. You can [write the loop yourself](https://docs.together.ai/docs/post-training/training-loop.md) with the Together Python SDK, or start from a ready-made recipe in the [post-training cookbook](https://docs.together.ai/docs/post-training/cookbook-overview.md).

From the checkpoints saved by a session, you can download the trained weights to your machine, fork the run into a new training session, or deploy the model to a [dedicated endpoint](https://docs.together.ai/docs/dedicated-endpoints/overview.md) for production inference.

![The post-training workflow. Load a model on GPUs, attach a training session, run your training loop, and save a checkpoint. From a checkpoint, you can fork or resume a new training session, download the weights to your machine, or deploy them to a dedicated endpoint.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/workflow.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=7c1a59bc75366be7c268c82e53413799)

Post-training supports these training methods:

* [Supervised fine-tuning (SFT)](https://docs.together.ai/docs/post-training/sft.md).
* [Direct preference optimization (DPO)](https://docs.together.ai/docs/post-training/dpo.md).
* [Reinforcement learning with group relative policy optimization (GRPO)](https://docs.together.ai/docs/post-training/grpo.md).
* [Multi-turn GRPO](https://docs.together.ai/docs/post-training/multi-turn-environments.md).

If you only need SFT or DPO, you can alternatively use [standard fine-tuning](#post-training-vs-standard-fine-tuning), which runs the whole training loop for you.

## Key features

* **Train on outcomes:** With [reinforcement learning](https://docs.together.ai/docs/post-training/grpo.md), your code scores the model's outputs, so you can train on any task where you can check the answer, even if you can't provide training examples.
* **Train agents:** Fine-tune models in [multi-turn environments](https://docs.together.ai/docs/post-training/multi-turn-environments.md), where the model acts over several turns and can call tools.
* **Control every step:** Your code chooses each batch, learning rate, and loss function. For any training method, you can [write your own loss function](https://docs.together.ai/docs/post-training/loss-functions.md) that computes the loss in PyTorch from the model's log-probabilities.
* **No infrastructure management:** Your loop runs on your laptop, while Together AI runs the sharded trainer, the generator replicas, and the weight sync between them. See [Concepts](https://docs.together.ai/docs/post-training/concepts.md) for more details.
* **Iterate cheaply, then deploy:** Train several low-rank adapters (LoRAs) on [one base model](https://docs.together.ai/docs/post-training/multi-lora.md) while paying for only that base model's GPU time, then [deploy](https://docs.together.ai/docs/post-training/deployment.md) the best one for dedicated model inference.

## Get started

> [card] [SFT quickstart](https://docs.together.ai/docs/post-training/quickstart-sft.md)
>
> Run a cookbook SFT recipe that trains a model on instruction-following examples.

> [card] [GRPO quickstart](https://docs.together.ai/docs/post-training/quickstart-rl.md)
>
> Run a cookbook GRPO recipe that trains a model on grade-school math with an exact-match reward.

> [card] [Write a training loop](https://docs.together.ai/docs/post-training/training-loop.md)
>
> Run a minimal training loop with the Python SDK, then add batching and sampling.

> [card] [Supported models](https://docs.together.ai/docs/post-training/supported-models.md)
>
> Pick a base model by training mode, context length, and GPU type.

> [card] [Migrate from Tinker](https://docs.together.ai/docs/post-training/migrate-from-tinker.md)
>
> Run a Tinker training script by changing its import line.

## Cookbook recipes vs. writing your own loop

There are two main ways to use post-training, both of which use the same underlying APIs and infrastructure:

* **[Run a cookbook recipe](https://docs.together.ai/docs/post-training/cookbook-overview.md):** Choose this if a ready-made training loop fits your task. The post-training cookbook is a Python library of recipes that handle dataset loading, tokenization, the loop itself, and, for GRPO, reward scoring. You can tune a run by editing its config.
* **[Write your own loop with the SDK](https://docs.together.ai/docs/post-training/training-loop.md):** Choose this if you need more control over data, sampling, rewards, or the loss function.

## Post-training vs. standard fine-tuning

Together AI also offers a [job-based fine-tuning service](https://docs.together.ai/docs/fine-tuning/overview.md) for standard workloads. If you only need SFT or DPO, and your data is well formed, it's much simpler to use: just upload a dataset, submit a job, and wait for the model to train.

Post-training is more powerful but more complex. It uses a session-based workflow in which your own code runs the training loop against remote GPUs.

The table summarizes the main differences:

| | Standard fine-tuning | Post-training |
| - | - | - |
| Where it runs | Together AI runs the whole loop, including tokenization, batching, loss, learning-rate schedule, and checkpoints. | Your code drives the training loop, and remote GPUs run the heavy steps: forward and backward passes, optimizer steps, and sampling. |
| Billing | Per token processed. | Per GPU-minute. |
| Data | JSONL files in fixed formats, validated at upload, or pre-tokenized Parquet files. | Token sequences and loss masks you build, in any shape, changeable mid-run. |
| Mid-run control | None. The job runs to completion unless you cancel it or configure [early stopping](https://docs.together.ai/docs/fine-tuning/early-stopping.md). | Full. Inspect per-step metrics, change the learning rate, reshuffle data, or stop on your own logic. |
| Loss functions | Built-in SFT and DPO objectives. | Built-in cross-entropy and policy-gradient loss functions, plus custom loss functions that run in your code. |
| Reinforcement learning | Not available. | GRPO and related policy-gradient methods, for both single-turn and multi-turn tasks. |
| Use when | You want standard SFT or DPO on well-formed data, with minimal code. | You need step-level control, custom loss functions or curricula, or reinforcement learning. |

## Train, evaluate, and productionize adapters

A training project usually produces several candidate adapters. Post-training lets you train them on one base model, compare them, and promote the winner:

![The three stages of a post-training project. Train: one model resource trains adapters A, B, and C (up to five per resource) while you pay for only one base model. Evaluate: deploy the adapters to a dedicated endpoint and compare them on live traffic, where adapter B wins. Productionize: merge and quantize the winner, B, then scale it across several replicas for production inference.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/train-evaluate-productionize.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=5bad9402d75c716be9b8945752c0cf7a)

1. **Train:** [Attach several training sessions to one model resource](https://docs.together.ai/docs/post-training/multi-lora.md). Each session trains an independent adapter against the same frozen base model.
2. **Evaluate:** If the base model supports multi-LoRA deployment, [attach the candidates to one deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md) of that model on dedicated GPUs, then [compare them](https://docs.together.ai/docs/post-training/evaluate.md) by naming a different adapter in each request. Each candidate is ready to attach once its session saves an inference checkpoint, which registers the adapter as a model.
3. **Productionize:** [Deploy the winner](https://docs.together.ai/docs/post-training/deployment.md#deploy-a-lora-adapter-in-merged-mode) on its own deployment, where each replica merges the adapter into the base model's weights when it starts. If the base model is [served only in FP4](https://docs.together.ai/docs/post-training/supported-models.md#checkpoints-and-deployment), [prepare the adapter for FP4 inference](https://docs.together.ai/docs/post-training/prepare-for-fp4-inference.md) instead, then deploy the prepared model.

## Learn more

> [card] [Concepts](https://docs.together.ai/docs/post-training/concepts.md)
>
> Learn how work splits between your code and Together AI GPUs, and what model resources, sessions, and checkpoints do.

> [card] [Choose a training method](https://docs.together.ai/docs/post-training/training-methods.md)
>
> Compare SFT, DPO, and GRPO by the training data each one needs, and combine methods.

> [card] [Model resources](https://docs.together.ai/docs/post-training/model-resources.md)
>
> Size, create, attach to, and stop the GPUs your sessions train on.

> [card] [Pricing](https://docs.together.ai/docs/post-training/pricing.md)
>
> Learn how per-GPU-minute billing and refunds on failure work, and estimate what a run costs.

> [card] [Manage checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md)
>
> Save, download, and resume from what you trained.

> [card] [Deploy a checkpoint](https://docs.together.ai/docs/post-training/deployment.md)
>
> Serve a trained model on dedicated model inference.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.