Find your model’s ID
An inference checkpoint registers its weights in your project’s model registry, so you deploy it by its registry ID. Read the ID from the checkpoint’sregistration:
registration holds:
registration.adapter: A LoRA session registers only the adapter. Deploy the adapter on its own deployment, or attach it to a multi-LoRA deployment.registration.model: A full-weight session, or a LoRA session on custom base weights, registers a merged model. Deploy its ID directly.- No
registration: The checkpoint wasn’t uploaded to the model registry, so it can’t be deployed. Checkpoints on models marked Not registered in Supported models aren’t uploaded. Download the weights instead.
tg training ls-checkpoints --type inference --session <session_id>. The Registry artifact column holds each inference checkpoint’s registry ID and revision ID as <ID>@<REVISION_ID>. tg training get <checkpoint_id> prints the same IDs for one checkpoint. You can also look up a checkpoint’s IDs without the session.
Deploy a LoRA adapter in merged mode
To serve a LoRA adapter for production traffic, deploy it on its own deployment with--merge. Each replica merges the adapter into its base model’s weights when it starts, then serves the merged model at the base model’s speed and cost. There’s no merge job or upload.
Adapters trained on models served only in FP4 can’t deploy with --merge. Prepare them for FP4 inference instead.
Run tg endpoints deploy with the adapter’s ID:
Merging adds to each replica’s startup time, including replicas that autoscaling adds later. A new deployment can take 20 minutes or more to become ready.
You can also deploy the adapter with the SDK.
Choose a config
To find a profile for--config, pass the adapter ID to tg models configs, which lists the deployment profiles of the adapter’s serving base model:
accelerator_type is nvidia-b200-180gb and whose adapter_mode is disabled, then pass its Config ID to --config. The profile’s accelerator_count sets the number of GPUs each replica uses. If no B200 profile is listed, contact your account team for B200 serving support.
Deploy LoRA adapters for FP4 inference
To serve an adapter on its own deployment when it was trained on a model served only in FP4, first run an FP4 preparation job. The job merges the adapter into the base model’s weights, then calibrates and quantizes the merged model to FP4. A base model that already ships in the format it serves in skips calibration and quantization. The job registers the result as a private model in your project. Estimate the job, then start it:estimate prints the job’s price and running time without creating anything. The running time depends on the base model and excludes any wait for GPU capacity. create --watch starts the paid job and prints the prepared model’s ID when it completes. Then deploy the prepared model by that ID.
For custom calibration data, job monitoring, cancellation, and the SDK, see Prepare for FP4 inference.
Deploy a merged model
Deploy a merged model, such as a full-weight checkpoint or an FP4-prepared model, withtg endpoints deploy:
deploy creates the endpoint and routes all traffic to the new deployment. The dedicated model inference quickstart walks through the rest of the flow, from choosing a config to calling the endpoint. Scale the deployment to zero or delete the endpoint when you’re done.
Serve the adapter on a multi-LoRA deployment
Instead of deploying a LoRA adapter on its own, you can attach it to a running deployment of its base model, if the base model supports multi-LoRA deployment. Attaching takes about a minute, with no extra job or upload. Each request does extra work for the adapter, and a deployment holds a limited number of adapters. Use it to compare candidates and to serve adapters with low or mixed traffic. For production traffic, deploy the adapter on its own deployment.Next steps
Multi-LoRA deployment
Attach candidate adapters to one deployment and compare them per request.
Prepare for FP4 inference
Merge an adapter into its base model and quantize the result to FP4.
Dedicated model inference quickstart
Follow the CLI and SDK deploy flow, step by step.
Manage checkpoints
Save the inference checkpoints you deploy.