> ## Documentation Index
> Fetch the complete documentation index at: https://docs.together.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Prepare for FP4 inference

> Merge a trained LoRA adapter into its base model and quantize the result to FP4 for dedicated model inference.

For some large base models, dedicated model inference serves merged models only in FP4, a 4-bit weight format that fits a model on fewer GPUs and runs it faster than full precision. Training doesn't run on those FP4 weights, so a low-rank adapter (LoRA) you train doesn't match the FP4 model you deploy. An FP4 preparation job gives you a trained model in the format production serving expects:

* **Serve at FP4 speed and cost:** An adapter served on top of its base model adds work to every request. A merged model has the adapter's changes built into its weights, so it serves like the FP4 base model, on the same hardware.
* **Keep the adapter's quality:** Merging an adapter straight into FP4 weights would apply its changes to weights that already carry quantization error. The job merges into the full-precision (BF16) weights first, then calibrates and quantizes the merged model to FP4, so the adapter's changes survive quantization. MiniMax-M3.1-Flash already ships in the format it serves in, so its job only merges.

Each job produces one model and can run for hours, depending on the base model, so start one after you've chosen a winning adapter from your [post-training](https://docs.together.ai/docs/post-training/overview.md) sessions. If the base model supports multi-LoRA deployment, compare several candidates first by [serving them on one deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md).

## Supported models

The adapter must be a LoRA adapter from a post-training session, not a full-weight model. Its base model must also support FP4 preparation. [Supported models](https://docs.together.ai/docs/post-training/supported-models.md#checkpoints-and-deployment) lists which base models do, how long each job typically takes, and which base model serves the prepared model.

Together sets the merge and quantization parameters for each model, and you can't change them.

## Find the adapter ID

A job takes the adapter's model registry ID (`ml_...`) and, optionally, a revision ID (`rv_...`). If you omit the revision, the job uses the adapter's latest revision. Read both from `registration.adapter` on the [session's inference checkpoint](https://docs.together.ai/docs/post-training/deployment.md), or from `inference_registration.adapter` [after the session ends](https://docs.together.ai/docs/post-training/checkpoints.md#find-a-checkpoint).

Only a checkpoint that registers an adapter can be prepared. A checkpoint that registers a merged model [deploys directly](https://docs.together.ai/docs/post-training/deployment.md#deploy-a-merged-model).

In an API request, pass the ID as `model_object_id` and the revision as `model_revision_id`. The older names, `adapter_object_id` and `adapter_revision_id`, still work but are deprecated. Use one pair, not both. A request that mixes the two pairs fails with a `400` error, even when the values match.

## Estimate the job

To check a job's price and running time without creating anything, estimate it with the adapter ID:

```bash CLI theme={null}
tg training fp4 estimate ml_abc123 --model-revision-id rv_abc123
```

```python Python theme={null}
from together import Together

client = Together()

estimate = client.post_training.prepare_for_fp4_inference.estimate_cost(
    inputs={
        "model_object_id": "ml_abc123",
        "model_revision_id": "rv_abc123",
    },
)
print(estimate.price_usd, estimate.time_hours, estimate.allowed_to_proceed)
```

```bash cURL theme={null}
curl -X POST "https://api.together.ai/v1/shaping/prepare-for-fp4-inference/estimate" \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "model_object_id": "ml_abc123",
      "model_revision_id": "rv_abc123"
    }
  }'
```

The CLI prints the price and the expected running time:

```text theme={null}
The estimated price of this job is $43.14 and it is expected to run for about 6.0h (excluding queue time).
```

The running time is an estimate, not a time limit, so a job that runs longer than estimated keeps running. If your credit can't cover the price, the CLI adds a warning, and in Python `allowed_to_proceed` is `False`.

A successful estimate also confirms that the adapter is eligible, because estimating and creating a job accept the same adapters.

### How a job is billed

Each job runs on one B200 GPU. The charge is applied when the job ends, and it's based on how long the job actually ran, not on the estimate:

* **Billed time:** Billing covers the time from when the job enters `running` until it ends, rounded up to the next whole minute. Time in `pending` or `queued` isn't billed.
* **Failed jobs:** A job that ends in `error` or `user_error` isn't billed.
* **Cancelled jobs:** A job you cancel is billed only if it had reached `running`.

## Calibrate on your own data

By default, the job calibrates the merged model on a general text corpus. To calibrate on data that matches your workload, pass a conversation dataset with `--calibration-file` (`-c`). The job then fills half of the calibration corpus with your dataset and the other half with general text. A job on a base model that skips calibration, such as MiniMax-M3.1-Flash, ignores the file.

The calibration file must:

* Be a JSONL file of conversations. Each row has `messages` and, optionally, `tools`, in the same chat format as [SFT data](https://docs.together.ai/docs/post-training/sft-recipes.md#use-your-own-dataset).
* Be at most 64 MiB.
* Contain at least 598,016 tokens once the model's chat template is applied, enough to fill its half of the calibration corpus.
* Belong to the same project as the adapter.

Pass the file when you create the job:

```bash CLI theme={null}
# A local file: the CLI uploads it with purpose "calibration"
tg training fp4 create ml_abc123 --calibration-file ./calibration.jsonl

# A file you uploaded earlier
tg training fp4 create ml_abc123 --calibration-file file-abc123
```

```python Python theme={null}
from together import Together

client = Together()

calibration = client.files.upload(
    file="calibration.jsonl", purpose="calibration"
)

job = client.post_training.prepare_for_fp4_inference.create(
    inputs={
        "model_object_id": "ml_abc123",
        "calibration_file_id": calibration.id,
    },
)
```

To check the file before you start a job, pass it to `estimate` as well. The estimate confirms that the file exists and passed validation. It doesn't count tokens, so a file below the token minimum passes the estimate but fails the job with a `user_error`. Add more conversations and retry.

## Start the job

Create the job with the same adapter ID:

```bash CLI theme={null}
tg training fp4 create ml_abc123 --model-revision-id rv_abc123
```

```python Python theme={null}
from together import Together

client = Together()

job = client.post_training.prepare_for_fp4_inference.create(
    inputs={
        "model_object_id": "ml_abc123",
        "model_revision_id": "rv_abc123",
    },
)
print(job.id, job.status)
```

```bash cURL theme={null}
curl -X POST "https://api.together.ai/v1/shaping/prepare-for-fp4-inference" \
  -H "Authorization: Bearer $TOGETHER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": {
      "model_object_id": "ml_abc123",
      "model_revision_id": "rv_abc123"
    }
  }'
```

| Parameter | CLI | API (`inputs`) | Description |
| - | - | - | - |
| Adapter ID | `MODEL-OBJECT-ID` (positional) | `model_object_id`, or the deprecated `adapter_object_id` | The adapter's registry ID (`ml_...`). See [Find the adapter ID](#find-the-adapter-id). |
| Adapter revision | `--model-revision-id`, `-r` | `model_revision_id`, or the deprecated `adapter_revision_id` | Optional. The adapter's revision ID (`rv_...`). Defaults to the adapter's latest revision. |
| Calibration file | `--calibration-file`, `-c` | `calibration_file_id` | Optional. A conversation dataset to calibrate on. The CLI also accepts a local path and uploads it. See [Calibrate on your own data](#calibrate-on-your-own-data). |
| Skip confirmation | `--confirm`, `-y` | Not applicable | Starts the job without the confirmation prompt. |
| Wait for the job | `--watch`, `-w` | Not applicable | Waits in the same command until the job reaches a terminal status. See [Monitor the job](#monitor-the-job). |

In the CLI, `create` shows the estimate and asks you to confirm before it creates the job. `estimate` takes the same adapter ID, `--model-revision-id`, and `--calibration-file`. For every flag, see the [`tg training` reference](https://docs.together.ai/reference/cli/training).

> [!WARNING]
>
> The CLI skips the confirmation prompt when it isn't running in an interactive terminal, such as in CI, with redirected input or output, or when it detects an AI agent. `create` then starts the job at once, so run `estimate` first.

The new job has an ID in the form `shp-quant-...` and starts in `pending`. Save the ID to monitor or cancel the job.

Each request starts a new job, even for an adapter you've already prepared. A repeat job uploads its result to the same model as the earlier job, so reuse that model instead of starting another.

## Monitor the job

A job moves from `pending` to `queued` to `running` to `completed`. Watch it until it reaches a terminal status:

```bash CLI theme={null}
tg training fp4 get shp-quant-1a2b3c4d-5e6f --watch
```

```python Python theme={null}
import time

from together import Together

client = Together()

TERMINAL = {"completed", "cancelled", "error", "user_error"}

job = client.post_training.prepare_for_fp4_inference.retrieve(
    "shp-quant-1a2b3c4d-5e6f"
)
while job.status not in TERMINAL:
    time.sleep(300)
    job = client.post_training.prepare_for_fp4_inference.retrieve(job.id)

print(job.status)
if job.status == "completed":
    print(job.results.api_model_object_id, job.results.api_model_revision_id)
```

`--watch` prints each status change and new event until the job reaches a terminal status. The command exits with a non-zero code if the job failed. Stopping the command doesn't stop the job, so run it again to resume watching.

| Status | Meaning |
| - | - |
| `pending`, `queued` | Accepted and waiting for a GPU. |
| `running` | Merging, quantizing, or uploading the model. |
| `completed` | Done. `results` holds the new model's IDs. |
| `cancel_requested`, `cancelled` | You [cancelled the job](#cancel-a-job). |
| `user_error` | The job failed because of its input. The job's events say what to fix. |
| `error` | The job failed on Together's side. Retry, or [contact support](https://portal.usepylon.com/together-ai/forms/support-request) if it fails again. |

When the job reports `completed`, the prepared model is already validated for deployment. The CLI prints the model's ID (`ml_...`) and revision ID (`rv_...`). In Python, read them from `job.results.api_model_object_id` and `job.results.api_model_revision_id`.

A job's `params` describe the adapter it prepares, not the prepared model. `params.inputs` and `params.configuration` both name the adapter under the `model_*` names and the deprecated `adapter_*` names, with the same values. `params.configuration.model_revision_id` is the revision the job prepares, which is the adapter's latest when the request didn't name one.

`--watch` prints the job's events as they arrive. To read a job's events later, such as the reason it failed, list them:

```bash CLI theme={null}
tg training fp4 list-events shp-quant-1a2b3c4d-5e6f
```

```python Python theme={null}
from together import Together

client = Together()

events = client.post_training.prepare_for_fp4_inference.list_events(
    "shp-quant-1a2b3c4d-5e6f"
)
for event in events.data:
    print(event.created_at, event.type, event.message)
```

A failed job ends with either a `SHAPING_USER_ERROR` event or a `SHAPING_ERROR` event. A `SHAPING_USER_ERROR` event's message says how to fix the input.

To list every FP4 preparation job in your project, run `tg training fp4 list` or call `client.post_training.prepare_for_fp4_inference.list()`.

## Deploy the prepared model

The prepared model is a private model in your project. It's registered against the base model that serves it in FP4, which can differ from the base model the adapter records. [Supported models](https://docs.together.ai/docs/post-training/supported-models.md#checkpoints-and-deployment) names it for each model. List its deployment profiles, then deploy it by its `model_object_id`:

```bash CLI theme={null}
tg models configs ml_def456

tg endpoints deploy ml_def456 \
  --endpoint my-prepared-model \
  --config cr_Cf3yApYdaWMrH6RsyVmh7 \
  --project your_project_id
```

| Parameter | Description |
| - | - |
| `ml_def456` (positional) | The prepared model's ID from the [completed job](#monitor-the-job). |
| `--endpoint` | A name for the new endpoint. |
| `--config` | The config ID of an FP4 profile from the `configs` output, such as `cr_Cf3yApYdaWMrH6RsyVmh7` for GLM-5.3. Optional when the model has only one compatible profile. For help picking one, see [Choose a deployment profile](https://docs.together.ai/docs/dedicated-endpoints/configs.md). |
| `--project` | Your project ID, from `tg whoami`. Pass it, or set `TOGETHER_PROJECT_ID`, even for your default project. Without either, the CLI resolves a private model ID to its base model and deploys that instead. |

The `deploy` command creates the endpoint, attaches a deployment, and routes all traffic to it.

## Cancel a job

To stop a job before it completes, cancel it:

```bash CLI theme={null}
tg training fp4 cancel shp-quant-1a2b3c4d-5e6f
```

```python Python theme={null}
from together import Together

client = Together()

job = client.post_training.prepare_for_fp4_inference.cancel(
    "shp-quant-1a2b3c4d-5e6f"
)
print(job.status)
```

The CLI asks you to confirm. Pass `--force` (`-y`) to skip the prompt. The job moves to `cancel_requested` right away and reaches `cancelled` once the run stops, so watch it to confirm. The CLI won't cancel a job that's already in `cancel_requested`. A job in a terminal status can't be cancelled, and the CLI prints that the job isn't cancellable.

## Troubleshooting

* **"Preparing an adapter on base model ... is not supported":** The adapter's base model isn't one of the [supported models](#supported-models).
* **"model ... is not an adapter" or "does not record a fine-tuning relation":** Pass the adapter's ID (`registration.adapter.id`, or `inference_registration.adapter.id` from the checkpoints API), not a merged model's ID. Full-weight models aren't accepted.
* **"model ... does not exist in this project":** The adapter belongs to a different project. Use an API key scoped to the project that trained the adapter.
* **"Calibration file ... not found":** The file ID is wrong, or the file belongs to another project. Upload the file in the adapter's project.
* **The estimate warns that the price exceeds your credit, or create fails with a `402` error:** Your credit balance can't cover the estimated price. Add credits and retry.
* **Create fails with a `403` error:** Your project is blocked from starting jobs. [Contact support](https://portal.usepylon.com/together-ai/forms/support-request).
* **"Shaping job ... not found":** The job doesn't exist, or it belongs to another project.
* **Job ends in `user_error`:** List the job's events. The `SHAPING_USER_ERROR` event's message describes the problem with the input.

## Next steps

> [card] [Manage checkpoints](https://docs.together.ai/docs/post-training/checkpoints.md)
>
> Save inference checkpoints from a post-training session.

> [card] [Multi-LoRA deployment](https://docs.together.ai/docs/post-training/multi-lora-deployment.md)
>
> Compare candidate adapters before you prepare the winner.

> [card] [Choose a deployment profile](https://docs.together.ai/docs/dedicated-endpoints/configs.md)
>
> Pick the hardware and quantization a deployment runs on.

> [card] [Manage deployments](https://docs.together.ai/docs/dedicated-endpoints/manage.md)
>
> Scale, stop, and delete endpoints and deployments.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.