Getting Started

Requirements Deploy CosmicAC

Install the CLI Create a GPU Container Job Create a Managed Inference Job (vLLM)Create an API key

Guides

GPU Container Job

Managed Inference Job

Create a Managed Inference Job (web interface)Create an API key (web interface)Connect to a Managed Inference endpoint (vLLM)Create a Managed Inference Job (Parakeet)Transcribe audio with a Parakeet endpoint Check Managed Inference endpoint health Create a model master

Platform management

Concepts

Architecture GPU Container Job Managed Inference Job

Reference

CosmicAC CLI commands

Configuration reference Task deployment commands Job configuration reference Recommended model parameters

Managed Inference Job

Create a model master

Create a model master to set default serving parameters for a model.

Create a model master to define the default serving parameters for a model, so they prefill the Serving configuration when you create a Managed Inference Job. For what a model master is, see Model masters.

This guide is in progress. It will cover creating a model master with a request to the app-node API.

Check Managed Inference endpoint health

Previous Page

Platform management

Next Page