Set up SSH and Tailscale on a GPU Container Job

Configure SSH and Tailscale inside a GPU Container Job for persistent, secure access over Tailscale.

Configure SSH and Tailscale inside a running GPU Container Job. You can then reach it securely from your local machine over the Tailscale network.

Prerequisites

You need the following before you start:

A running GPU Container Job. See Create a GPU Container Job.
Root access inside the container, with sudo -s or a direct root shell.
Your local SSH public key (~/.ssh/id_ed25519.pub on your laptop).
Tailscale admin access to approve the node, if your access control lists (ACLs) require it.

Steps

Access the GPU Container Job

Open a shell into the running container. See Access a GPU Container Job. Then become the root user, since the next steps run as root:

sudo -s

Install the required packages

Update the system, then install the base tools and the Tailscale client:

sudo apt update -y
sudo apt install -y sudo openssh-server nano
curl -fsSL https://tailscale.com/install.sh | sudo sh

Create the SSH directory:

mkdir -p /root/.ssh
chmod 700 /root/.ssh

Add your local machine's public key to authorized_keys, then lock down the permissions:

nano /root/.ssh/authorized_keys   # paste the contents of ~/.ssh/id_ed25519.pub, then save
chmod 600 /root/.ssh/authorized_keys

Harden SSH

Allow key-only root login and disable password authentication:

sed -i \
  -e 's/^#\?PermitRootLogin.*/PermitRootLogin yes/' \
  -e 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' \
  -e 's/^#\?PubkeyAuthentication.*/PubkeyAuthentication yes/' \
  -e 's/^#\?UsePAM.*/UsePAM no/' \
  /etc/ssh/sshd_config

Confirm the settings:

grep -E 'PermitRootLogin|PasswordAuthentication|PubkeyAuthentication|UsePAM' /etc/ssh/sshd_config

The output should be the following:

PermitRootLogin yes
PubkeyAuthentication yes
PasswordAuthentication no
UsePAM no

Root login is enabled because these containers are accessed as root over Tailscale with key-only authentication. Do not enable password authentication.

Start Tailscale and SSH

Create the runtime directories, start tailscaled in the background, then start sshd:

mkdir -p /var/run/sshd
mkdir -p /var/run/tailscale

nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown

/usr/sbin/sshd

Confirm both processes are running:

ps ax | grep -E 'tailscaled|sshd'

Join Tailscale

Bring the node up with a unique hostname, and any tags your Tailscale ACLs require:

tailscale up --hostname=<hostname> --advertise-tags=tag:<your-tag>

For example:

tailscale up --hostname=gpu-container-05 --advertise-tags=tag:gpu-containers

If the node is not authenticated yet, Tailscale prints a login URL. Open it and approve the device.

Verify

From inside the container, check the processes and the Tailscale status:

ps ax | grep -E 'tailscaled|sshd'
tailscale status
tailscale ip -4

From your local machine, once the node appears in the Tailscale admin console, connect over SSH:

ssh root@gpu-container-05

You connect without a password prompt, using key-based authentication only.

Help and troubleshooting

SSH fails with "Permission denied (publickey)"

Confirm the public key is in /root/.ssh/authorized_keys, with permissions 600 on the file and 700 on .ssh.

SSH connects but asks for a password

Re-run the SSH hardening sed command, then restart sshd.

tailscale up fails

Confirm tailscaled is running with ps ax | grep tailscaled, then check /var/log/tailscaled.log.

Node not visible in Tailscale

Run tailscale login, or approve the device in the admin console.

sshd won't start

Confirm /var/run/sshd exists, then check journalctl or /var/log/auth.log.

Restart SSH or Tailscale

Restart SSH after configuration changes:

pkill sshd
/usr/sbin/sshd

Restart Tailscale:

pkill tailscaled
nohup tailscaled \
  --state=/tmp/tailscale.state \
  --socket=/var/run/tailscale/tailscaled.sock \
  > /var/log/tailscaled.log 2>&1 & disown
tailscale up --hostname=<hostname> --advertise-tags=tag:<your-tag>