> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Deploy a fine-tuned model

> Serve your fine-tuned model on a dedicated endpoint or download it for local inference.

Once a fine-tuning job completes, your model is available for inference in two ways: hosted on a dedicated endpoint at Together AI, or downloaded as a standalone checkpoint.

## Requirements

* A completed fine-tuning job. See the [quickstart](/docs/fine-tuning/quickstart) for the full lifecycle.
* One of the job's two registry identifiers for the trained weights. Look for them in the `tg fine-tuning retrieve` output on a completed job:
  * `model_object_id` (an `ml_...` value): The model-registry ID. The CLI takes it directly, and the SDK and API take it inside a `projects/{project_id}/models/{model_id}` path.
  * `model_object_name` (a `<project_slug>/<model_name>` value): The qualified registry name. The CLI accepts it in place of the ID. The SDK and API don't.

## Deploy on a dedicated endpoint

Together AI serves fine-tuned models through [dedicated model inference](/docs/dedicated-endpoints/overview) (DMI). A completed fine-tuning job is already a private model in your project, so you deploy it by its Model Object ID (or, from the CLI, by its qualified registry name) with no separate upload step. Deployment and lifecycle operations use the `tg beta` CLI.

<Warning>
  Dedicated model inference bills per minute per running replica, even when idle. Scale the deployment to zero or delete the endpoint when you're done to stop charges.
</Warning>

<Note>
  The dedicated model inference commands require Together CLI version `2.24.0` or later. Install or upgrade with `uv tool install "together[cli]"` (or `uv tool upgrade "together[cli]"`), and check your version with `tg --version`.
</Note>

<Tabs>
  <Tab title="CLI">
    The CLI's `deploy` command creates the endpoint, attaches a deployment, and routes all traffic to it in one step. Pass either the fine-tune's `model_object_name` or its `model_object_id`. The CLI resolves the name to the same model when exactly one model in your project matches it:

    ```bash CLI theme={null}
    # Retrieve the job to copy its identifiers (model_object_name and model_object_id)
    tg fine-tuning retrieve "<JOB_ID>"

    # Deploy the fine-tuned model to a new endpoint
    # (pass <MODEL_OBJECT_ID> instead to deploy by ID)
    tg beta endpoints deploy "<MODEL_OBJECT_NAME>" \
      --endpoint my-finetuned-endpoint
    ```

    Pass the name in full, including the project slug (`acme-corp/my-model-abc123`). The bare model name doesn't resolve, and neither does `model_output_name`.

    Poll until the deployment reaches `DEPLOYMENT_STATE_READY`:

    ```bash CLI theme={null}
    tg beta endpoints get "<ENDPOINT_ID>"
    ```

    Once the deployment is `READY`, the endpoint serves at its **endpoint string** (`your-project-slug/my-finetuned-endpoint`, printed by the deploy output). Pass it as the `model` parameter and point the base URL at `https://api-inference.together.ai/v1`:

    ```bash cURL theme={null}
    curl -s https://api-inference.together.ai/v1/chat/completions \
      -H "Authorization: Bearer $TOGETHER_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "your-project-slug/my-finetuned-endpoint",
        "messages": [{"role": "user", "content": "Hello!"}]
      }'
    ```

    When you're done, delete the endpoint and its deployment to stop billing. The CLI's `rm --force` removes both in one step:

    ```bash CLI theme={null}
    tg beta endpoints rm "<ENDPOINT_ID>" --force
    ```

    To pause billing without deleting anything (for example, if you plan to use the endpoint again later), scale the deployment to zero instead. See [Delete resources](/docs/dedicated-endpoints/manage#delete-resources) for the full teardown order.

    <Note>
      If `deploy` reports that the model has more than one deployment profile, re-run it with `--config <cr_...>`. List a model's profiles with `tg beta models configs "<MODEL_OBJECT_ID>"`. See [Choose a deployment profile](/docs/dedicated-endpoints/configs).
    </Note>
  </Tab>

  <Tab title="SDK">
    The SDK has no single-call equivalent of the CLI's `deploy`, so it runs the same steps individually: create the endpoint, bind the model and a config to a deployment, then route traffic to it. Reference the fine-tune by its **Model Object ID** (`ml_...`, the job's `model_object_id`). Resolving a qualified name is a CLI convenience, so `model_object_name` doesn't work here: the API takes the model as a `projects/{project_id}/models/{model_id}` resource path.

    <CodeGroup>
      ```python Python theme={null}
      from together import Together

      client = Together()
      project_id = client.whoami().project_id

      # Reference the fine-tune by its Model Object ID (ml_...) and a config
      # for the base model. List configs: tg beta models configs <model>.
      model = f"projects/{project_id}/models/<MODEL_OBJECT_ID>"
      config = f"projects/{project_id}/configs/<CONFIG_ID>"

      # 1. Create the endpoint.
      endpoint = client.beta.endpoints.create(
          project_id=project_id,
          name="my-finetuned-endpoint",
      )

      # 2. Bind the model and config to a deployment.
      deployment = client.beta.endpoints.deployments.create(
          endpoint.id,
          project_id=project_id,
          name="prod",
          model=model,
          config=config,
          autoscaling={"min_replicas": 1, "max_replicas": 1},
      )

      # 3. Route 100% of traffic to the deployment.
      client.beta.endpoints.update(
          endpoint.id,
          project_id=project_id,
          traffic_split=[{"deployment_id": deployment.id, "weight": 1}],
      )
      print(endpoint.name)
      ```

      ```typescript TypeScript theme={null}
      import Together from "together-ai";

      const client = new Together();
      const { project_id: projectId } = await client.whoami();

      // Reference the fine-tune by its Model Object ID (ml_...) and a config
      // for the base model. List configs: tg beta models configs <model>.
      const model = `projects/${projectId}/models/<MODEL_OBJECT_ID>`;
      const config = `projects/${projectId}/configs/<CONFIG_ID>`;

      // 1. Create the endpoint.
      const endpoint = await client.beta.endpoints.create({
        projectId,
        name: "my-finetuned-endpoint",
      });

      // 2. Bind the model and config to a deployment.
      const deployment = await client.beta.endpoints.deployments.create(
        endpoint.id,
        {
          projectId,
          name: "prod",
          model,
          config,
          autoscaling: { minReplicas: 1, maxReplicas: 1 },
        },
      );

      // 3. Route 100% of traffic to the deployment.
      await client.beta.endpoints.update(endpoint.id, {
        projectId,
        trafficSplit: [{ deploymentId: deployment.id, weight: 1 }],
      });
      console.log(endpoint.name);
      ```
    </CodeGroup>

    Poll until the deployment reaches `DEPLOYMENT_STATE_READY`. Retrieve the deployment and read `status.state`:

    <CodeGroup>
      ```python Python theme={null}
      deployment = client.beta.endpoints.deployments.retrieve(
          deployment.id,
          project_id=project_id,
          endpoint_id=endpoint.id,
      )
      print(deployment.status.state)
      ```

      ```typescript TypeScript theme={null}
      const current = await client.beta.endpoints.deployments.retrieve(
        deployment.id,
        { projectId, endpointId: endpoint.id },
      );
      console.log(current.status.state);
      ```
    </CodeGroup>

    Once the deployment is `READY`, the endpoint serves at its **endpoint string** (`your-project-slug/my-finetuned-endpoint`, returned as `endpoint.name`). Pass it as the `model` parameter and point the base URL at `https://api-inference.together.ai/v1`:

    <CodeGroup>
      ```python Python theme={null}
      from together import Together

      # A dedicated client for inference: this base URL serves only
      # inference, not the fine-tuning or files APIs.
      inference_client = Together(base_url="https://api-inference.together.ai/v1")

      response = inference_client.chat.completions.create(
          model="your-project-slug/my-finetuned-endpoint",
          messages=[{"role": "user", "content": "Hello!"}],
      )
      print(response.choices[0].message.content)
      ```

      ```typescript TypeScript theme={null}
      import Together from "together-ai";

      // A dedicated client for inference: this base URL serves only
      // inference, not the fine-tuning or files APIs.
      const inferenceClient = new Together({
        baseURL: "https://api-inference.together.ai/v1",
      });

      const response = await inferenceClient.chat.completions.create({
        model: "your-project-slug/my-finetuned-endpoint",
        messages: [{ role: "user", content: "Hello!" }],
      });
      console.log(response.choices[0].message.content);
      ```
    </CodeGroup>

    When you're done, delete the endpoint and its deployment to stop billing. The SDK has no smart-delete equivalent of the CLI's `rm --force`, so stop billing by scaling the deployment to zero, then delete the resources once it reaches `DEPLOYMENT_STATE_STOPPED`:

    <CodeGroup>
      ```python Python theme={null}
      # Scale to zero to stop billing.
      client.beta.endpoints.deployments.update(
          deployment.id,
          project_id=project_id,
          endpoint_id=endpoint.id,
          autoscaling={"min_replicas": 0, "max_replicas": 0},
      )

      # Once it's DEPLOYMENT_STATE_STOPPED, drop its traffic weight, then
      # delete the deployment and the endpoint.
      client.beta.endpoints.update(
          endpoint.id,
          project_id=project_id,
          traffic_split=[{"deployment_id": deployment.id, "weight": 0}],
      )
      client.beta.endpoints.deployments.delete(
          deployment.id, project_id=project_id, endpoint_id=endpoint.id
      )
      client.beta.endpoints.delete(endpoint.id, project_id=project_id)
      ```

      ```typescript TypeScript theme={null}
      // Scale to zero to stop billing.
      await client.beta.endpoints.deployments.update(deployment.id, {
        projectId,
        endpointId: endpoint.id,
        autoscaling: { minReplicas: 0, maxReplicas: 0 },
      });

      // Once it's DEPLOYMENT_STATE_STOPPED, drop its traffic weight, then
      // delete the deployment and the endpoint.
      await client.beta.endpoints.update(endpoint.id, {
        projectId,
        trafficSplit: [{ deploymentId: deployment.id, weight: 0 }],
      });
      await client.beta.endpoints.deployments.delete(deployment.id, {
        projectId,
        endpointId: endpoint.id,
      });
      await client.beta.endpoints.delete(endpoint.id, { projectId });
      ```
    </CodeGroup>

    To pause billing without deleting anything (for example, if you plan to use the endpoint again later), scale the deployment to zero and stop there. See [Delete resources](/docs/dedicated-endpoints/manage#delete-resources) for the full teardown order.
  </Tab>

  <Tab title="UI">
    <Steps>
      <Step title="Open your models">
        Go to [My models](https://api.together.ai/models?category=my-models). Your completed fine-tuned models appear here once training finishes.
      </Step>

      <Step title="Create the endpoint">
        Select your fine-tuned model, then select **Create dedicated endpoint**.
      </Step>

      <Step title="Configure the deployment">
        Choose a deployment configuration (hardware and GPU count) and set the replica bounds.
      </Step>

      <Step title="Deploy">
        Select **Deploy**. Provisioning takes up to 10 minutes. You can navigate away while it runs.
      </Step>

      <Step title="Stop when done">
        Open the endpoint from the dashboard and stop or delete it when you're not using it to halt per-minute billing.
      </Step>
    </Steps>
  </Tab>
</Tabs>

## Run locally

To run your model outside Together, download the checkpoint by job ID:

<CodeGroup>
  ```bash CLI theme={null}
  tg fine-tuning download "<JOB_ID>"
  ```

  ```python Python theme={null}
  from together import Together

  client = Together()

  response = client.fine_tuning.content(ft_id="<JOB_ID>", checkpoint="merged")
  response.write_to_file("my-model/model.tar.zst")
  ```

  ```typescript TypeScript theme={null}
  import Together from "together-ai";
  import fs from "node:fs";

  const client = new Together();

  const response = await client.fineTuning.content({ ft_id: "<JOB_ID>", checkpoint: "merged" });
  const buffer = Buffer.from(await response.arrayBuffer());
  fs.writeFileSync("my-model/model.tar.zst", buffer);
  ```
</CodeGroup>

### Choose a checkpoint type

The `checkpoint` parameter selects what to download. If you omit it, the API picks a default automatically: `merged` for LoRA jobs and `model_output_path` for full fine-tunes. You can also pass `checkpoint_step`, which downloads a specific intermediate step and overrides `checkpoint`. Valid values depend on how the job was trained.

| Job type | Valid `checkpoint` values | What you get |
| - | - | - |
| LoRA fine-tune | `merged`, `adapter`, or `model_output_path` | `merged` combines the base model and adapter into self-contained weights, the usual choice for running the model locally or uploading it elsewhere. `adapter` returns only the LoRA adapter weights, so you can load them on top of the base model yourself (for example, with PEFT or vLLM). |
| Full fine-tune | `model_output_path` only | The full set of trained model weights. `adapter` returns an error for full fine-tunes. |

`model_output_path` returns the raw training output directory before any merging. It works for both job types but is mainly useful for advanced workflows that need the unmodified artifacts: for LoRA jobs, prefer `merged` or `adapter`; for full fine-tunes, it's the only option.

The output is a `.tar.zst` archive that uses [ZStandard](https://github.com/facebook/zstd) compression. On macOS, install `zstd` with Homebrew and decompress:

```bash theme={null}
brew install zstd
cd my-model
zstd -d model.tar.zst
tar -xvf model.tar
```

You should see:

```text theme={null}
tokenizer_config.json
special_tokens_map.json
pytorch_model.bin
generation_config.json
tokenizer.json
config.json
```

Load the model with Hugging Face Transformers:

```python Python theme={null}
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained("./my-model")
model = AutoModelForCausalLM.from_pretrained(
    "./my-model",
    trust_remote_code=True,
).to(device)

input_ids = tokenizer.encode("Space robots are", return_tensors="pt").to(
    device
)
output = model.generate(input_ids, max_length=128, temperature=0.7)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

To download a specific checkpoint instead of the final one, pass `--checkpoint-step <STEP_NUMBER>` to `tg fine-tuning download` (or `checkpoint_step=<STEP_NUMBER>` to `client.fine_tuning.content()`). List checkpoints with `tg fine-tuning list-checkpoints <JOB_ID>`.

## Model registry object IDs

When a job uploads its output artifacts to the Together model registry alongside the primary storage write, the job and checkpoint responses include the registry object and revision IDs, plus qualified object names in `<project_slug>/<model_name>` form. Use the IDs to reference the uploaded weights in downstream workflows, such as binding a fine-tuned adapter to a dedicated endpoint. The CLI's [`deploy`](#deploy-on-a-dedicated-endpoint) command also accepts the qualified name.

On a completed job in the [fine-tuning jobs dashboard](https://api.together.ai/jobs), the **Output model** field shows `model_object_name` and links to that model in the registry by `model_object_id`.

The object ID fields are omitted when the registry upload did not run or did not succeed. A name can still be present for a job whose upload never ran, but the CLI can't resolve it, so treat the matching `*_object_id` as the signal that the artifact reached the registry. Object names are resolved on the fly on [`GET /fine-tunes/{id}`](/reference/get-fine-tunes-id) and [`GET /fine-tunes/{id}/checkpoints`](/reference/get-fine-tunes-id-checkpoint) only (not on list jobs). If the project slug cannot be resolved, the name fields fall back to the corresponding object ID.

### On the job

[`GET /fine-tunes/{id}`](/reference/get-fine-tunes-id) includes the final artifact IDs once the job reaches `completed`:

| Field | Description |
| - | - |
| `model_object_id` | Registry object ID for the final model weights (for example, `ml_…`). Present when the job produced a model artifact and the upload succeeded. |
| `model_object_revision_id` | Registry revision ID for the final model weights (for example, `rv_…`). |
| `model_object_name` | Qualified registry name for the final model weights (for example, `acme-corp/my-model-abc123`). Pass it to `tg beta endpoints deploy` in place of `model_object_id`. The `<model_name>` part comes from `model_output_name` (the segment after `/` when namespaced, or the whole string otherwise). |
| `adapter_object_id` | Registry object ID for the final adapter weights on LoRA jobs. |
| `adapter_object_revision_id` | Registry revision ID for the final adapter weights on LoRA jobs. |
| `adapter_object_name` | Qualified adapter name on LoRA jobs (for example, `acme-corp/my-model-abc123-adapter`). |

<CodeGroup>
  ```bash CLI theme={null}
  tg fine-tuning retrieve "<JOB_ID>"
  ```

  ```python Python theme={null}
  job = client.fine_tuning.retrieve(id="<JOB_ID>")
  if job.api_model_object_id:
      print(job.api_model_object_id, job.api_model_object_revision_id)
      print(job.api_model_object_name)
  if job.adapter_object_id:
      print(job.adapter_object_id, job.adapter_object_revision_id)
      print(job.adapter_object_name)
  ```

  ```typescript TypeScript theme={null}
  const job = await client.fineTuning.retrieve("<JOB_ID>");
  if (job.model_object_id) {
    console.log(job.model_object_id, job.model_object_revision_id);
    console.log(job.model_object_name);
  }
  if (job.adapter_object_id) {
    console.log(job.adapter_object_id, job.adapter_object_revision_id);
    console.log(job.adapter_object_name);
  }
  ```
</CodeGroup>

### On checkpoints

[`GET /fine-tunes/{id}/checkpoints`](/reference/get-fine-tunes-id-checkpoint) returns the same ID pair on each checkpoint entry:

| Field | Description |
| - | - |
| `object_id` | Registry object ID for that checkpoint's artifact (for example, `ml_…`). |
| `object_revision_id` | Registry revision ID for that checkpoint's artifact (for example, `rv_…`). |
| `object_name` | Qualified registry name for that checkpoint (for example, `acme-corp/my-model-abc123-100` for an intermediate model checkpoint, or `acme-corp/my-model-abc123-adapter` for the final adapter). |

Intermediate checkpoints carry the IDs from the upload at that step. Final model and adapter checkpoints in the list reuse the job-level IDs from the table above.

<CodeGroup>
  ```bash CLI theme={null}
  tg fine-tuning list-checkpoints "<JOB_ID>"
  ```

  ```python Python theme={null}
  checkpoints = client.fine_tuning.list_checkpoints("<JOB_ID>")
  for cp in checkpoints.data:
      if cp.object_id:
          print(cp.step, cp.object_id, cp.object_revision_id, cp.object_name)
  ```

  ```typescript TypeScript theme={null}
  const checkpoints = await client.fineTuning.listCheckpoints("<JOB_ID>");
  for (const cp of checkpoints.data) {
    if (cp.object_id) {
      console.log(cp.step, cp.object_id, cp.object_revision_id, cp.object_name);
    }
  }
  ```
</CodeGroup>

## Troubleshooting

* **No Model Object ID yet:** The job hasn't reached `completed`, so `model_object_id` isn't populated. Poll status with `client.fine_tuning.retrieve(id=...)` until it's done. See [Monitor a fine-tuning job](/docs/fine-tuning/monitoring#poll-until-the-job-is-done) for the polling pattern.
* **`endpoints_v1_create_access_disabled` (HTTP 403):** You're calling the retired v1 endpoints API (`client.endpoints.create(...)` or `tg endpoints create`). Deploy on dedicated model inference with `tg beta endpoints deploy` instead. See [Migrate from v1](/docs/dedicated-endpoints/migrate-from-v1).
* **`deploy` reports multiple deployment profiles:** Re-run with `--config <cr_...>`. List a model's profiles with `tg beta models configs "<MODEL_OBJECT_ID>"`.
* **`Model <MODEL_OBJECT_NAME> not found` when deploying by name:** Pass `model_object_name` in full, including the project slug, and check that the job reports a `model_object_id`. Without one, the weights never reached the registry, so neither identifier resolves. If the name still fails, deploy by `model_object_id`.
* **`Multiple models found for "<MODEL_OBJECT_NAME>"`:** More than one model visible in your project has the same final name segment. Deploy by `model_object_id`, which is unambiguous.
* **Either identifier fails with a key from another project:** The model belongs to the project the job ran in, and both identifiers resolve only within your key's project. Use a key scoped to the project that owns the model.
* **Deploy fails because the base isn't supported:** Not every base model can be hosted for dedicated inference. Confirm the base appears in the [supported models](/docs/dedicated-endpoints/models) list before training (`-Reference` models often can't be deployed).
* **404 on inference:** Point the base URL at `https://api-inference.together.ai/v1` and pass the endpoint string (`your-project-slug/<endpoint_name>`, printed by the deploy output) as the `model` parameter, not the Model Object ID.

## Next steps

<CardGroup cols={2}>
  <Card title="Upload a custom model" icon="upload" href="/docs/dedicated-endpoints/custom-models">
    Upload your own model weights from outside the Together catalog.
  </Card>

  <Card title="Manage endpoints" icon="settings" href="/docs/dedicated-endpoints/manage">
    Inspect, start, stop, update, and delete dedicated endpoints.
  </Card>

  <Card title="Configure autoscaling" icon="adjustments-horizontal" href="/docs/dedicated-endpoints/scaling">
    Tune replica bounds and autoscale a deployment on the metric that fits your workload.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.