Requirements
Before you begin, make sure you have:- Python 3.11 or later and uv.
- The Together CLI, version 3.0.0 or later, to download the trained model. Install or upgrade it with
uv tool install "together[cli]" --upgrade. - A Together AI account with enough credit to run your model resource for four hours, the minimum for creating one.
- Your API key exported as
TOGETHER_API_KEY. Get one from API keys in the Together AI console. - A Weights & Biases API key exported as
WANDB_API_KEY. Most cookbook recipes log to Weights & Biases, including the one in this quickstart. To run without an account, exportWANDB_MODE=offlineinstead. - Optionally, a Hugging Face token exported as
HF_TOKENfor gated models or datasets.
Step 1: Clone the cookbook
Clone the cookbook and install its dependencies, including the Together Python SDK:Step 2: Review the recipe
Open the GSM8K recipe,recipes/grpo/gsm8k.py. It passes a config to train_single_turn, the cookbook’s single-turn GRPO training loop:
recipes/grpo/gsm8k.py
dataset field selects GSM8K from the cookbook’s dataset registry, which supplies the prompt format and the reward. GSM8K reference answers end with #### <number>, and the prompt asks the model to end its solution the same way.
The reward function reads the number after #### in each completion, or a \boxed{} answer when there’s no ####. It returns 1.0 if that number matches the reference and 0.0 otherwise. A completion with neither marker scores 0.0, even when it states the right number.
These other settings shape the run:
Each step of
train_single_turn samples group_size completions per prompt, scores them, and computes each completion’s advantage relative to its group. It then runs forward_backward with the GRPO loss, optim_step, and weights_sync, so the next step samples from the updated model. See How Together post-training runs GRPO for details.
Step 3: Run training
Run the recipe:[3f8b2c7d], is the first eight characters of the session ID.
Step 4: Monitor the run
The step log and thegrpo-gsm8k project in Weights & Biases report the same metrics. On a first run, watch these:
The
loss line is a poor progress signal for GRPO, so track accuracy instead. Monitor training runs covers the other metrics and how to act on them.
Step 5: Retrieve the trained model
The recipe registers each inference checkpoint in your project’s model registry and prints each checkpoint’s ID as it saves it. The run ends with the final pair:Session ready line of the training output and in the run’s manifest.json:
ml_...@rv_.... Download the final adapter’s files with it:
[step 200] inference checkpoint line to the Python SDK’s download_checkpoint function.
To serve the model on Together AI instead, deploy its inference checkpoint.
You trained a model with GRPO, watched it learn, and retrieved the result.
Stop the model resource
The run stops the session and the model resource when it exits, so billing has already ended. To confirm, list the model resources you created most recently:stopped. If the run was interrupted, its resource can still be active. A ready resource is billing, and a pending or creating one starts billing once it’s ready. Stop the resource with its ID from the list:
force=True also stops any session that’s still attached. Without it, the API refuses to stop a resource while a session is active.
You can also stop the resource from the Model resources tab under Post-Training in the console. Manage model resources covers stopping, idle timers, and what a resource costs while it runs.
Next steps
Reinforcement learning with GRPO
Learn how GRPO runs and how to design a reward.
GRPO recipes
Bring your own dataset and reward to the GSM8K recipe.
Concepts
Learn how work splits between your code and Together AI GPUs, and what model resources, sessions, and checkpoints do.
Deploy a checkpoint
Deploy the trained model on dedicated model inference.