Skip to content
LabPod LabPod LabPod Docs

Backups & restore

LabPod backs up its SQLite database - workspaces, users, templates, policies, audit logs, and settings. User files in /home/<user>/work are not included in the database backup; back those up with your host-level backup tool separately.

Daily backup timer

Section titled “Daily backup timer”

The install script registers labpod-backup.timer, which runs once a day with a randomized delay and calls:

Terminal window
labpod admin backup --dir /var/lib/labpod/backups --retention 14

This creates a timestamped snapshot (labpod-YYYYMMDDTHHMMSSZ.db) and keeps the 14 most recent, pruning older ones.

Terminal window
# Inspect the timer
sudo systemctl list-timers labpod-backup.timer


# Check recent backup logs
sudo journalctl -u labpod-backup.service --since today


# List existing backups
ls -lah /var/lib/labpod/backups/
Terminal window
# Run these as root
# Inspect the timer
systemctl list-timers labpod-backup.timer


# Check recent backup logs
journalctl -u labpod-backup.service --since today


# List existing backups
ls -lah /var/lib/labpod/backups/

Force a snapshot now:

Terminal window
sudo systemctl start labpod-backup.service
Terminal window
# Run these as root
systemctl start labpod-backup.service

Tune retention or directory

Section titled “Tune retention or directory”

Use a systemd drop-in to override the defaults:

Terminal window
sudo systemctl edit labpod-backup.service
Terminal window
# Run these as root
systemctl edit labpod-backup.service

In the editor, paste:

[Service]
ExecStart=
ExecStart=/usr/local/bin/labpod admin backup --dir /var/lib/labpod/backups --retention 30
Terminal window
sudo systemctl daemon-reload
Terminal window
# Run these as root
systemctl daemon-reload

Ad-hoc backup

Section titled “Ad-hoc backup”

Write a self-contained snapshot to an explicit path:

Terminal window
sudo labpod admin --db /var/lib/labpod/labpod.db backup --out /tmp/labpod-adhoc.db
Terminal window
# Run these as root
labpod admin --db /var/lib/labpod/labpod.db backup --out /tmp/labpod-adhoc.db

This fails fast if the database file doesn’t exist (no silent empty-DB creation).

Restore

Section titled “Restore”

Restore requires stopping the service because SQLite refuses a hot copy when the WAL sidecar (.db-wal, .db-shm) is in use.

Terminal window
sudo systemctl stop labpod


sudo labpod admin --db /var/lib/labpod/labpod.db restore \
  /var/lib/labpod/backups/labpod-20260517T030000Z.db


sudo systemctl start labpod
Terminal window
# Run these as root
systemctl stop labpod


labpod admin --db /var/lib/labpod/labpod.db restore \
  /var/lib/labpod/backups/labpod-20260517T030000Z.db


systemctl start labpod

restore will prompt for confirmation unless you pass --yes.

What the backup contains

Section titled “What the backup contains”

The backup is a complete snapshot of the LabPod database:

  • Users, roles, and hashed passwords
  • Workspace definitions and GPU allocations
  • Templates and their port/env configuration
  • Resource policies and system settings
  • Session records and audit logs
  • Usage samples and rollups

Not included: workspace container layers, user home directories, uploaded files. The workspace container state (running kernels, training checkpoints) is also not captured - only the workspace metadata.