Job configuration reference
Fields you set when you create a GPU Container Job or a Managed Inference Job.
You set these fields when you create a job, either in the web interface or with cosmicac jobs create. The job type determines which fields apply. In non-interactive mode, set each field with the flag in the CLI flag column. For the create flow, see Create a GPU Container Job and Create a Managed Inference Job.
Common fields
These fields apply to every job type.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Job type | Yes | --type | The kind of job to create, either GPU Container or Managed Inference. |
| Job name | Yes | --name | A name to identify the job. |
| Tags | Yes | --tags | One or more labels for the job. The CLI accepts a comma-separated list. |
| Location | Yes | --location | Where the job runs, for example IN. The CLI lists the locations your racks report. |
| Notifications | No | --notify | The job lifecycle events this job reports. Takes a comma-separated list of job_failed, job_degraded, job_recovered, and job_restart_storm, or all, or none. Defaults to all four. An event also needs the matching switch on in Settings → Notifications before it reaches your webhook. See What controls delivery. |
GPU configuration
These fields select the job's hardware.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| GPU | Yes | --gpu-type | The GPU to use, for example GH100_H100_SXM5_80GB. The CLI lists the GPU types your racks report. |
| GPU count | Yes | --gpu-count | Number of GPUs for one replica. One of 1, 2, 4, or 8. A vLLM Managed Inference Job also takes 16 for a multi-node replica. |
| CUDA / driver | Yes | --driver | GPU driver version. Your model masters set the available versions, for example CUDA 13.0. |
| CPU cores per GPU | No | --cpu-cores-per-gpu | CPU cores to reserve for each GPU. If you omit this field, CosmicAC uses the value your rack reports for the GPU type. |
| RAM per GPU | No | --mem-gb-per-gpu | System memory in GB to reserve for each GPU. Use a whole number. If you omit this field, CosmicAC uses the value your rack reports for the GPU type. |
Set the GPU type and count in one flag with --gpu TYPE=COUNT, for example --gpu H100=2. This replaces --gpu-type and --gpu-count.
On nodes that hold 8 GPUs, a replica with 1, 2, 4, or 8 GPUs runs on one node. A replica with 16 takes two whole nodes. Only a vLLM Managed Inference Job can run a multi-node replica. See Multi-node replicas.
For a Managed Inference Job, the count must equal the model's per_replica_gpu_count, listed as GPUs per replica in Recommended model parameters. CosmicAC rejects any other value with a 400 status and the message gpu.count must equal per_replica_gpu_count (N).
GPU Container parameters
These fields apply to a GPU Container Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Base OS image | Yes | --base-image | Operating system image for the container. Your model masters set the available images, for example Ubuntu 22.04 + CUDA 13.0. |
| Disk | Yes | --root-disk-size-gb | Root disk size in GB, for example 250. Minimum 100. |
vLLM Managed Inference parameters
These fields apply to a vLLM Managed Inference Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Model | Yes | --model | Hugging Face model ID to serve, for example MiniMaxAI/MiniMax-M2.5. |
| Runtime image (CUDA) | Yes | --runtime-image | Serving runtime image as a Docker image reference, for example vllm/vllm-openai:v0.15.1. The web interface takes this value from the model master. |
| Data type | Yes | --data-type | Numeric precision the model runs at, for example BF16 or Auto. |
| Quantisation | Yes | --quantisation | Quantisation scheme, for example fp8 or awq. |
| Tensor parallel | No | --tensor-parallel | GPUs to split the model across. Must divide the GPU count exactly. Defaults to 1 in the CLI. |
| GPU memory utilization | Yes | --gpu-memory-utilization | Fraction of GPU memory to use, between 0 and 1. |
| Max concurrent sequences | Yes | --max-concurrent-sequences | Maximum requests handled at once. |
| Max model length | Yes | --max-model-length | Maximum model context length. |
| Reasoning parser | Yes | --reasoning-parser | Parser for the model's reasoning output. |
| Video & image input | Yes | --multimodal | Whether the model accepts multimodal input. true or false. |
| Root disk size | Yes | --root-disk-size-gb | VM root disk size in GB. Minimum is the model's disk_gb, listed as Root disk size in Recommended model parameters. |
| Environment variables | No | --env | Environment variables passed to the inference service. See Startup timeout. |
| Endpoint name | Yes | --endpoint-name | Name of the inference endpoint. Must be unique across active inference jobs. |
| Replicas | Yes | --replica | Number of endpoint replicas. One of 1, 2, or 4. |
| Require Authorization header | Yes | --require-auth-header / --no-auth-header | Whether callers must send an authorization header. true or false. |
Each model sets its own minimum, so no single size works everywhere. CosmicAC rejects a smaller root disk with root_disk_size_gb (X) is below model master disk_gb (Y).
The web interface doesn't show Tensor parallel. It sends the model master's value, so jobs you create there are already correct.
The CLI is different:
- Interactive mode:
cosmicac jobs createlists your model masters and pre-selects the value from the one you pick. With no model masters, it pre-selects1. - Flags: the CLI sends
1unless you pass--tensor-parallel.
Set tensor parallel to the model's per_replica_gpu_count, listed as GPUs per replica in Recommended model parameters.
If the value doesn't divide the job's GPU count exactly, CosmicAC rejects the job with tensor_parallel must evenly divide the GPU count per instance and a 400 status.
Startup timeout
CosmicAC gives a model a fixed window to load and start serving. If the model server isn't ready when that window closes, CosmicAC restarts it. You set these two variables in the job's Environment variables field:
DOCKER_STARTUP_TIMEOUT_MS: milliseconds the model has to become ready. The default is 30 minutes, written1800000. A multi-node job uses 60 minutes, written3600000. A value of0disables the timeout, so CosmicAC never restarts a slow load.DOCKER_MAX_RESTARTS: how many times CosmicAC restarts the job before it fails. The default is3.
A restart from the startup timeout counts toward that limit. A job that reaches the limit stops retrying and fails.
Parakeet Managed Inference parameters
These fields apply to a Parakeet Managed Inference Job.
| Field | Required | CLI flag | Description |
|---|---|---|---|
| Model | Yes | --model | Parakeet model to serve, nvidia/parakeet-tdt-0.6b-v3. |
| Endpoint name | Yes | --endpoint-name | Name of the transcription endpoint. |
| Chunk duration | Yes | --chunk-duration | Audio chunk length in seconds, for example 600. Minimum 10. |
| Chunk overlap | Yes | --chunk-overlap | Overlap between chunks in seconds, for example 5. Must be less than the chunk duration. Minimum 5. |
| Max file size | Yes | --max-file-size-mb | Maximum upload size in MB, for example 1024. Minimum 1024. |
| Require Authorization header | Yes | --require-auth-header / --no-auth-header | Whether callers must send an authorization header. true or false. |