> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Train multiple adapters on one base model

> Share one model resource across several adapter sessions to run a hyperparameter sweep for the cost of one base model.

![One model resource, made of GPUs and the frozen base weights, hosting three sessions that train adapters A, B, and C. All sessions share the base weights, which are never updated. A resource holds up to five sessions, and each session has its own low-rank adapter (LoRA), data, hyperparameters, and checkpoints.](https://mintcdn.com/togetherai-52386018/Ta7hn_4Cp_DKcb3y/images/post-training/sessions-share-a-resource.png?fit=max\&auto=format\&n=Ta7hn_4Cp_DKcb3y\&q=85\&s=4247982d4af0b4f8c4f3e65359b347b2)

A model resource can host several low-rank adapter (LoRA) training sessions at once. Each session trains its own adapter over the resource's frozen base weights, with its own data, hyperparameters, and checkpoints.

Post-training bills mainly for the model resource, i.e. the set of GPUs with the base model loaded onto them. Sessions add nothing to the bill, so training several adapters on one resource costs the GPU time of one base model, not one per adapter.

Use a shared resource to run a *hyperparameter sweep*, a set of training runs that are identical except for one or two settings, such as the learning rate or LoRA rank. Each run in a sweep is called an *arm*. You compare the arms' adapters to find the values that work best.

If you write your own training loop, [start one session per arm with the SDK](#train-adapters-with-the-sdk). If you train with the cookbook's GRPO or SFT recipes, [point each arm's recipe at the shared resource](#train-adapters-with-the-cookbook).

> [!NOTE]
>
> This page covers training several adapters on one model resource. You can then [serve the trained adapters from one inference deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md) and compare them.

## Plan a sweep

A resource created with `lora_enabled=True` accepts up to five concurrent sessions by default, and fewer for some models, so check the model's row on [Supported models](https://docs.together.ai/docs/post-training/supported-models.md) before you choose how many arms to run. A full-weight resource (`lora_enabled=False`) accepts one session. If you create a session beyond the cap, the API returns HTTP `429` with an "at capacity" message, and the SDK raises `RateLimitError` after a few retries.

Sessions on a resource share its base model, GPU type, generator replicas, and optimizer, so a sweep can vary only what each session sets for itself: its LoRA config, data, loss function, sampling settings, learning rate and other per-step optimizer parameters, and checkpoints.

The training method can vary by session too. A resource has no method of its own, and each session chooses its loss function per training step, so supervised fine-tuning (SFT), direct preference optimization (DPO), and group relative policy optimization (GRPO) sessions can run side by side. GRPO sessions sample completions, so they need a resource with at least one [generator replica](https://docs.together.ai/docs/post-training/model-resources.md#generator-replicas).

## Train adapters with the SDK

Create one resource, then start a session on it for each adapter. To run the sessions concurrently, call the [`_async` variants](https://docs.together.ai/docs/post-training/training-loop#call-operations-asynchronously) and await them together. This example trains three SFT arms, one per learning rate, on a shared trainer-only resource:

```python theme={null}
import asyncio

from together.post_training import (
    AdamParams,
    ComputeConfig,
    LossConfig,
    ModelResourcesClient,
)

MODEL = "Qwen/Qwen3.8-27B"
LEARNING_RATES = (5e-5, 1e-4, 2e-4)
samples = load_samples()  # your cross-entropy samples


async def train_adapter(resources, lr):
    # Each session trains its own adapter over the shared base weights
    async with await resources.create_session_async(
        display_name=f"lr{lr:.0e}"
    ) as session:
        for step in range(20):
            await session.trainer.forward_backward_async(
                samples=samples,
                loss=LossConfig(type="LOSS_TYPE_CROSS_ENTROPY"),
            )
            await session.trainer.optim_step_async(
                adam_params=AdamParams(learning_rate=lr),
            )
        checkpoint = await session.create_inference_checkpoint_async()
        return checkpoint.id


async def main():
    # Provision the base model once for every arm
    resources = await ModelResourcesClient.create_async(
        base_model=MODEL,
        compute_config=ComputeConfig(
            gpu_type="B200-SXM", num_generator_replicas=0
        ),
    )
    try:
        # Train the arms side by side, one session each
        checkpoint_ids = await asyncio.gather(
            *(train_adapter(resources, lr) for lr in LEARNING_RATES)
        )
        print(dict(zip(LEARNING_RATES, checkpoint_ids)))
    finally:
        # Stop the resource along with any session still attached
        await resources.stop_async(force=True)


asyncio.run(main())
```

Replace `load_samples()` with your own code that [builds cross-entropy samples](https://docs.together.ai/docs/post-training/samples.md).

The `finally` block stops the resource even if an arm raises. It passes `force=True` because the API refuses a plain `stop` while another arm's session is still active.

Each arm saves an [inference checkpoint](https://docs.together.ai/docs/post-training/checkpoints.md#inference-checkpoints), so you can [compare the three adapters on one deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md).

To add sessions to a resource that's already running, such as one a teammate provisioned, [attach to it](https://docs.together.ai/docs/post-training/model-resources.md#attach-to-an-existing-resource) instead of creating one.

## Train adapters with the cookbook

The cookbook's GRPO and SFT recipes accept a `model_resources_id`, so several recipe runs can share one resource, whether it's one you keep running or one that a sweep script provisions.

### Attach to a running resource

A recipe's config names exactly one model source:

* **`model`** provisions a fresh resource that the run owns. When the run exits, it stops both its session and the resource.
* **`model_resources_id`** attaches a session to an existing resource as a tenant. When the run exits, it stops only its own session and leaves the resource running.

Use `model_resources_id` to train on a resource that a teammate provisioned, or one you keep running across several experiments. To find a resource's ID, [list the ready resources](https://docs.together.ai/docs/post-training/model-resources.md#inspect-and-list-resources) or open the **Model resources** tab under [Post-Training](https://api.together.ai/post-training/resources) in the Together AI console. Then set `model_resources_id` and leave `model` unset:

```python theme={null}
import os

from post_training_cookbook.grpo.config import GRPOConfig
from post_training_cookbook.grpo.train import train_single_turn

config = GRPOConfig(
    model_resources_id=os.environ["MODEL_RESOURCES_ID"],
    dataset="gsm8k",
    max_model_length=1536,
    max_prompt_length=768,
    max_sample_tokens=768,
)
train_single_turn(config)  # leaves the resource running
```

When a run attaches, the cookbook checks the resource first, so a mismatch fails before training instead of mid-run. The resource must:

* Be `READY`.
* Be LoRA-enabled when `use_lora=True`.
* Have a generator when the run samples.

For a complete example of a run that attaches, see the cookbook's `recipes/grpo/deepmath_shared.py`.

### Run a sweep

To run a sweep with recipes, provision the resource once, run each arm's recipe concurrently in its own session, and stop the resource once. The cookbook's DeepMath sweep (`recipes/grpo/deepmath_sweep.py`) trains four GRPO arms this way, each with its own learning rate:

```python theme={null}
from concurrent.futures import ThreadPoolExecutor

from post_training_cookbook.grpo.config import GRPOConfig
from post_training_cookbook.grpo.train import train_single_turn
from together.post_training import ModelResourcesClient

MODEL = "Qwen/Qwen3.6-35B-A3B"
LEARNING_RATES = (5e-5, 1e-4, 2e-4, 4e-4)


def build_config(model_resources_id: str, lr: float) -> GRPOConfig:
    return GRPOConfig(
        model_resources_id=model_resources_id,
        learning_rate=lr,
        wandb_run_name=f"lr{lr:.0e}",
        dataset="deepmath",
        num_examples=2000,
        num_val_examples=128,
        batch_size=16,
        group_size=8,
        num_steps=20,
        max_model_length=16384,
        max_prompt_length=1024,
        max_sample_tokens=15360,
        max_sampling_attempts=4,
        validation_interval=5,
        wandb_project="grpo-deepmath",
        wandb_group="deepmath-sweep",
    )


if __name__ == "__main__":
    # Provision the base model once; every arm attaches its own session.
    with ModelResourcesClient.create(
        base_model=MODEL,
        lora_enabled=True,
        compute_config={"gpu_type": "B200-SXM", "num_generator_replicas": 1},
    ) as resource:
        configs = [
            build_config(resource.model_resources_id, lr)
            for lr in LEARNING_RATES
        ]
        with ThreadPoolExecutor(max_workers=len(configs)) as pool:
            arms = {
                cfg.wandb_run_name: pool.submit(train_single_turn, cfg)
                for cfg in configs
            }
        failed = {
            name: exc
            for name, future in arms.items()
            if (exc := future.exception()) is not None
        }

    for name, exc in failed.items():
        print(f"[{name}] arm failed: {exc!r}")
```

How the pieces fit:

* **Concurrency:** `train_single_turn` blocks, so each arm runs in its own thread. The SDK tracks each session's work separately, so one arm's teardown doesn't cancel another arm's work.
* **Isolation:** An arm that raises doesn't stop the others. Collect failures with `future.exception()` after the pool drains.
* **Model selection:** Put the arms in one `wandb_group` and compare their `val/accuracy` curves side by side. Each arm already runs validation, so picking the best adapter takes no extra evaluation code.
* **Teardown:** The `ModelResourcesClient` context manager stops the resource once on exit, even if an arm raised. If you don't use the context manager, call `resource.stop()` yourself.

Each arm logs to Weights & Biases (W\&B), so before you run the sweep, [set up W\&B or turn it off](https://docs.together.ai/docs/post-training/monitoring.md). Then run `uv run python recipes/grpo/deepmath_sweep.py`.

## Keep a resource alive

A resource stops on its own when it goes longer than its [idle timeout](https://docs.together.ai/docs/post-training/model-resources.md#idle-timeout) (one hour by default) without an operation from any of its sessions. An attached session that sends no operations doesn't keep the resource alive. When the resource stops, every session on it is marked `EXPIRED`.

To keep a shared resource alive between runs, start the next run within the idle timeout. For a longer gap, call `extend_expiry` on the resource's handle.

A `READY` resource bills whether or not anything is training on it, so when your team is done, [stop it](https://docs.together.ai/docs/post-training/model-resources.md#stop-a-resource) with `force=True`, which also stops any tenant sessions still attached.

## Next steps

> [card] [Multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md)
>
> Compare the trained adapters on one deployment and pick the winner.

> [card] [Supported models](https://docs.together.ai/docs/post-training/supported-models.md)
>
> Check each model's session cap and GPU types.

> [card] [Manage model resources](https://docs.together.ai/docs/post-training/model-resources.md)
>
> Create, inspect, and stop the resource your sessions share.

> [card] [Pricing](https://docs.together.ai/docs/post-training/pricing.md)
>
> See what a shared resource costs and when it stops.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.