Data Loss is catastrophic whether you’re running a homelab or managing production infrastructure. The 3-2-1 rule is the simplest framework that covers most failure scenarios.
The 3-2-1 Rule
- 3 copies of your data
- 2 different storage media/locations
- 1 offsite copy
In practice for a homelab: your original data + a local backup (NAS, backup server) + one copy somewhere geographically separate (cloud VPS, object storage, StorageBox).
Common Mistakes
The ones that make backups useless when disaster strikes:
- Never testing restores — a backup you haven’t restored from is a backup you don’t actually have
- Storing backups on the same system as the data
- No encryption on backup data
- No versioning — ransomware encrypts your “backup” too if it’s just a live mirror
- No offsite copy — a house fire takes out everything local
Danger
Modern ransomware actively targets and encrypts backup files. Without versioning or an air-gapped copy, recovery is impossible.
Mitigations worth implementing: snapshot protection with retention policies, at least one offline or air-gapped copy, MFA on backup systems, alerting on backup job failures.
A Typical Homelab 3-2-1 Stack
| Layer | Example | Purpose |
|---|---|---|
| Primary | Proxmox + Longhorn / local disks | Working data |
| Local backup | Synology NAS / Proxmox Backup Server | Fast local restore |
| Offsite | Cloud VPS, Hetzner StorageBox, S3-compatible storage | Geographic redundancy |
The specific tools don’t matter much — what matters is that the three copies exist, are tested, and the offsite one is encrypted.
Proxmox Backup Server Scripts
If you’re running PBS, these scripts handle the backup and restore flow from any Linux host.
Backup to PBS
#!/bin/bash
export PBS_PASSWORD='token-secret-from-PBS'export PBS_USER_STRING='token-id-from-PBS'export PBS_SERVER='PBS-IP'export PBS_DATASTORE='DATASTORE_PBS'export PBS_REPOSITORY="${PBS_USER_STRING}@${PBS_SERVER}:${PBS_DATASTORE}"export PBS_HOSTNAME="$(hostname -s)"export PBS_KEYFILE='/root/pbscloud_key.json'
echo "Running PBS backup for $PBS_HOSTNAME ..."
proxmox-backup-client backup \ srv.pxar:/srv \ volumes.pxar:/var/lib/docker/volumes \ etc.pxar:/etc \ scripts.pxar:/usr/local/bin \ --keyfile "$PBS_KEYFILE" \ --skip-lost-and-found \ --repository "$PBS_REPOSITORY"
proxmox-backup-client list --repository "$PBS_REPOSITORY"echo "Done."Restore from PBS
#!/bin/bash
export PBS_PASSWORD='token-secret-from-PBS'export PBS_USER_STRING='token-id-from-PBS'export PBS_SERVER='PBS_IP'export PBS_DATASTORE='DATASTORE_FROM_PBS'export PBS_KEYFILE='/root/pbscloud_key.json'export PBS_REPOSITORY="${PBS_USER_STRING}@${PBS_SERVER}:${PBS_DATASTORE}"
SNAPSHOT_PATH="$1"ARCHIVE_NAME="$2"RESTORE_DEST="$3"
if [[ -z "$SNAPSHOT_PATH" || -z "$ARCHIVE_NAME" || -z "$RESTORE_DEST" ]]; then echo "Usage: $0 <snapshot_path> <archive_name> <destination>" echo "Example: $0 \"host/cloud/2025-01-22T15:19:17Z\" srv.pxar /root/restore-srv" exit 1fi
mkdir -p "$RESTORE_DEST"
echo "=== PBS Restore ==="echo "Snapshot: $SNAPSHOT_PATH"echo "Archive: $ARCHIVE_NAME"echo "Destination: $RESTORE_DEST"echo "==================="
proxmox-backup-client restore \ "$SNAPSHOT_PATH" \ "$ARCHIVE_NAME" \ "$RESTORE_DEST" \ --repository "$PBS_REPOSITORY" \ --keyfile "$PBS_KEYFILE"
[[ $? -eq 0 ]] && echo "Restore completed." || echo "Restore failed (exit $?)."Note
For restoring Longhorn PVC backups specifically, see Restoring from Longhorn Backups.
Disaster Recovery Scenarios
Local NAS failure
Data still exists on your backup server and offsite copy. Replace hardware, reconfigure RAID, restore from whichever copy is accessible, verify the next scheduled job completes.
Offsite target failure
Provision a new instance, redeploy the backup client, restore your encryption key from your password manager, re-run the backup script to verify connectivity, confirm the cron job is active.
Backup server failure
Reinstall the backup server software, restore the datastore config and mount point, confirm the datastore directory structure is intact (.chunks, backup namespaces), then pull data from whichever surviving copy is closest — local NAS or offsite.
What to Keep in Your Password Manager
When disaster strikes you’ll need these immediately:
- Encryption key file (e.g.
pbscloud_key.json) - PBS/backup system token credentials
- Backup directory paths for each layer
- Cron job definitions to rebuild automation from scratch
Maintenance
Monthly: integrity check on a random subset of files, review backup logs for silent failures. Quarterly: full test restore of at least one backup, document how long it took.
Real-time: alert on any backup job failure and on storage capacity thresholds — running out of space is the most common way a backup silently stops working.