3-2-1 Backup Strategy for Your Homelab

#storage#homelab

A practical guide to the 3-2-1 backup rule — what it means, how to implement it, and how to actually recover when things go wrong.

I’ve watched data loss happen, at home and at work, and every time it was the same shape: the backup existed, and nobody had ever restored from it. 3-2-1 is the smallest rule that covers most of the ways this goes wrong.

3 copies of the data. 2 different media or systems. 1 of them somewhere else.

For a homelab that’s the live data, a local backup target, and one copy geographically separate. Three files on the same array is one copy.

Note

This is the general shape. What my own chain actually looks like today — which legs run when, and what deliberately isn’t backed up — is in the homelab tour.

The stack

Layer Example What it’s for
Primary Proxmox + Longhorn / local disks Working data
Local backup Synology NAS / Proxmox Backup Server Fast restore, same building
Offsite Cloud VPS, Hetzner StorageBox, S3-compatible storage Survives the building

The tools matter far less than people think. What matters is that the three copies exist, that the offsite one is encrypted, and that you have restored from at least one of them.

The ways it fails anyway

  • Never testing restores. A backup you haven’t restored from is a hope.
  • Backups on the same system as the data. That’s a second copy of a single point of failure.
  • No encryption offsite. You’ve handed someone else your files.
  • No versioning. A live mirror replicates the encryption as fast as ransomware writes it.
  • No offsite copy. One fire, one flood, one theft.
Caution

Ransomware looks for backup files specifically, and a mounted backup share is the easiest target in the house. Without versioning or a copy that’s genuinely offline, there is nothing to recover from.

What actually helps: snapshots with a retention policy, at least one copy that’s offline or air-gapped, MFA on whatever manages the backups, and an alert when a job fails. That last one is cheap and catches the most common failure — backups that quietly stopped months ago.

Proxmox Backup Server, from any Linux host

The backup side. --keyfile is what makes the copy on the far end useless to anyone who takes the disk:

#!/bin/bash


export PBS_PASSWORD='token-secret-from-PBS'
export PBS_USER_STRING='token-id-from-PBS'
export PBS_SERVER='PBS-IP'
export PBS_DATASTORE='DATASTORE_PBS'
export PBS_REPOSITORY="${PBS_USER_STRING}@${PBS_SERVER}:${PBS_DATASTORE}"
export PBS_HOSTNAME="$(hostname -s)"
export PBS_KEYFILE='/root/pbscloud_key.json'


echo "Running PBS backup for $PBS_HOSTNAME ..."


proxmox-backup-client backup \
  srv.pxar:/srv \
  volumes.pxar:/var/lib/docker/volumes \
  etc.pxar:/etc \
  scripts.pxar:/usr/local/bin \
  --keyfile "$PBS_KEYFILE" \
  --skip-lost-and-found \
  --repository "$PBS_REPOSITORY"


proxmox-backup-client list --repository "$PBS_REPOSITORY"
echo "Done."

The restore side

Written now, rather than during the outage:

#!/bin/bash


export PBS_PASSWORD='token-secret-from-PBS'
export PBS_USER_STRING='token-id-from-PBS'
export PBS_SERVER='PBS_IP'
export PBS_DATASTORE='DATASTORE_FROM_PBS'
export PBS_KEYFILE='/root/pbscloud_key.json'
export PBS_REPOSITORY="${PBS_USER_STRING}@${PBS_SERVER}:${PBS_DATASTORE}"


SNAPSHOT_PATH="$1"
ARCHIVE_NAME="$2"
RESTORE_DEST="$3"


14 collapsed lines
if [[ -z "$SNAPSHOT_PATH" || -z "$ARCHIVE_NAME" || -z "$RESTORE_DEST" ]]; then
  echo "Usage: $0 <snapshot_path> <archive_name> <destination>"
  echo "Example: $0 \"host/cloud/2025-01-22T15:19:17Z\" srv.pxar /root/restore-srv"
  exit 1
fi


mkdir -p "$RESTORE_DEST"


echo "=== PBS Restore ==="
echo "Snapshot:    $SNAPSHOT_PATH"
echo "Archive:     $ARCHIVE_NAME"
echo "Destination: $RESTORE_DEST"
echo "==================="


proxmox-backup-client restore \
  "$SNAPSHOT_PATH" \
  "$ARCHIVE_NAME" \
  "$RESTORE_DEST" \
  --repository "$PBS_REPOSITORY" \
  --keyfile "$PBS_KEYFILE"


[[ $? -eq 0 ]] && echo "Restore completed." || echo "Restore failed (exit $?)."
Note

Longhorn PVCs don’t go through PBS. Those are restored from the S3 backup target instead.

When one layer dies

What failed What you still have What to do
Local NAS Backup server + offsite Replace hardware, rebuild the array, restore from whichever copy is closest, then confirm the next scheduled job succeeds
Offsite target Everything local New instance, redeploy the client, restore the encryption key from the password manager, run the backup once by hand, confirm the cron is back
Backup server NAS + offsite Reinstall, restore the datastore config and mount point, check the datastore structure is intact (.chunks, namespaces), then pull from a surviving copy

Each row assumes the other two copies are healthy. The reason 3-2-1 works is that it never asks you to bet on one of them.

What lives in the password manager

Restoring is gated on things that are not themselves in the backup:

  • The encryption keyfile, e.g. pbscloud_key.json — without it the offsite copy is noise
  • Token credentials for the backup system
  • The backup paths for each layer
  • The cron definitions, so the automation can be rebuilt from nothing

Keeping it honest

Monthly, run an integrity check over a random subset and read the logs for jobs that failed quietly. Quarterly, do a real restore of at least one backup and write down how long it took — that number is your actual recovery time, and it’s usually longer than anyone’s estimate.

Alert on failed jobs and on capacity thresholds. Running out of space is the most common way a backup stops working without anyone noticing.