Deploy CosmicAC

Deploy the CosmicAC stack on your host machine with Docker Compose. For how CosmicAC connects to your cluster and runs jobs, see Deployment architecture.

Stack

The stack runs six application services, plus Redis and Caddy:

cosmicac-wrk-ork
cosmicac-app-node (port 3000)
cosmicac-proxy-inference-http
cosmicac-proxy-inference-hrpc
cosmicac-wrk-server-k8s-nvidia
cosmicac-ui
redis
caddy (port 5173)

The CosmicAC application services use private images published under ghcr.io/tetherto/ (ghcr.io/tetherto/<repo>:<tag>). Redis and Caddy use their official public images.

Prerequisites

The recommended host is Ubuntu 22.04 or 24.04 LTS on x86_64. On that host, you need the following:

Docker Engine and Docker Compose v2.
Task.
The jq, node, and kubectl command-line tools.
Access to the private CosmicAC deployment repository, which holds the Compose files and deploy scripts. Request access if you do not have it.
GitHub Container Registry (GHCR) credentials for private images.
- Your GitHub username.
- A GitHub personal access token (classic) with the read:packages scope so it can pull the private ghcr.io/tetherto images. If the tetherto org enforces SSO, authorize the token for the org. See Managing your personal access tokens.
A GPU Kubernetes cluster that already meets the Requirements. CosmicAC connects to this cluster but doesn't set it up.
A valid, readable kubeconfig file for the GPU Kubernetes cluster. Bootstrap stops immediately if it is missing or invalid.

Steps

Verify the prerequisite tools are installed

Enable Docker and verify the tools from the Prerequisites are installed:

sudo systemctl enable --now docker
docker compose version
task --version
node --version
kubectl version --client

Set up the environment

Clone the deployment repository and change into it:

git clone <deployment-repo-url>
cd <deployment-repo>

Create the .env file from the example:

cp .env.example .env

Set the variables in .env. See Environment variables in the configuration reference.

Run the first-time bootstrap

task bootstrap deploys the whole stack with one command, using TAG from .env. For what it runs and the other deployment commands, see Task deployment commands.

If GITHUB_PAT and GITHUB_USER are not set in .env, bootstrap prompts for them so it can log in to GHCR and pull the private images.

Set KUBECONFIG_SRC to the absolute host path of your kubeconfig in .env before you bootstrap. CosmicAC mounts this kubeconfig into the worker so it can connect to your cluster and run jobs.

A missing kubeconfig path stops bootstrap with an error. Verify the kubeconfig source:

test -s "$KUBECONFIG_SRC"
kubectl --kubeconfig "$KUBECONFIG_SRC" config current-context
kubectl --kubeconfig "$KUBECONFIG_SRC" cluster-info

Run the bootstrap:

task bootstrap

For the kubeconfig structure and how bootstrap consumes it, see Kubeconfig requirements in the configuration reference.

Verify the deployment

Run these checks:

task ps                                          # all services Up
curl -s4 -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5173/   # 200
TOKEN=$(curl -s4 -X POST http://127.0.0.1:5173/api/login -H 'content-type: application/json' -d "{\"email\":\"${BOOTSTRAP_ADMIN_EMAIL}\",\"password\":\"${BOOTSTRAP_ADMIN_PASSWORD}\"}" | jq -r .token)
curl -s4 "http://127.0.0.1:5173/api/auth/servers?overwrite_cache=true" -H "ttr-token: $TOKEN"
curl -s4 "http://127.0.0.1:5173/api/auth/pricing?location=IN&type=gpu&hrs=2" -H "ttr-token: $TOKEN"
curl -s4 "http://127.0.0.1:5173/api/auth/jobs?page=1&pageSize=10" -H "ttr-token: $TOKEN"

The deployment is healthy when:

task ps shows all services Up.
servers lists your GPUs.
pricing returns a total_cost instead of an error.
jobs responds.

Reference responses

Healthy results: servers → {"data":[{"location":"IN","gpu":{"available":8}}]}, pricing → {"total_cost":2}.

Next steps

With CosmicAC running, install the CLI and create your first job:

On this page