Migrating Longhorn Backup from MinIO to Garage

merox merox #storage#docker

How to replace MinIO with Garage as the S3 backend for Longhorn backups — setup, bucket config, Kubernetes secret, and HelmRelease update.

MinIO’s been moving in a direction that doesn’t make much sense for homelab use anymore — Docker images dropped, source-only distribution, licensing drift. The result is most self-hosted installs sitting on outdated versions with known CVEs and no clean upgrade path. I replaced it with Garage, which is lightweight, actively maintained, S3-compatible, and doesn’t have any of that baggage.

Note

For restoring from Longhorn backups after this migration, see Restoring from Longhorn Backups. The process is identical regardless of whether you’re using MinIO or Garage.

Prerequisites: Ubuntu server with Docker and Docker Compose, Kubernetes cluster with Longhorn installed. Check Garage releases for the latest version before deploying.

Setting Up Garage

Directory and Secrets

Terminal window
sudo mkdir -p /srv/docker/garage/{meta,data}
cd /srv/docker/garage
# Generate RPC secret
openssl rand -hex 32
# Generate admin token
openssl rand -hex 32

garage.toml

metadata_dir = "/var/lib/garage/meta"
data_dir = "/var/lib/garage/data"
db_engine = "lmdb"
replication_factor = 1
rpc_bind_addr = "0.0.0.0:3901"
rpc_public_addr = "127.0.0.1:3901"
rpc_secret = "YOUR_RPC_SECRET_HERE"
[s3_api]
s3_region = "us-east-1"
api_bind_addr = "0.0.0.0:3900"
root_domain = ".s3.garage"
[admin]
api_bind_addr = "0.0.0.0:3903"
admin_token = "YOUR_ADMIN_TOKEN_HERE"
Tip

replication_factor = 1 is fine for single-node. Use 3 for multi-node redundancy.

docker-compose.yml

version: "3"
services:
garage:
image: dxflrs/garage:v2.1.0
container_name: garage
network_mode: "host"
restart: unless-stopped
volumes:
- ./garage.toml:/etc/garage.toml
- ./meta:/var/lib/garage/meta
- ./data:/var/lib/garage/data
webui:
image: khairul169/garage-webui:latest
container_name: garage-webui
restart: unless-stopped
volumes:
- ./garage.toml:/etc/garage.toml:ro
environment:
API_BASE_URL: "http://127.0.0.1:3903"
S3_ENDPOINT_URL: "http://127.0.0.1:3900"
network_mode: "host"
Terminal window
docker-compose up -d

Configure Storage

Terminal window
alias garage="docker exec -ti garage /garage"
garage node id
# Assign storage capacity
garage layout assign <node-id> -z default -c 100G
garage layout show
garage layout apply --version 1
# Create bucket and key
garage bucket create longhorn
garage key create longhorn-key
garage bucket allow longhorn --read --write --owner --key longhorn-key
# Note the Key ID and Secret key
garage key info longhorn-key --show-secret

Integrate with Longhorn

Kubernetes Secret

apiVersion: v1
kind: Secret
metadata:
name: minio-secret
namespace: longhorn-system
type: Opaque
stringData:
AWS_ACCESS_KEY_ID: <Key-ID-from-garage>
AWS_SECRET_ACCESS_KEY: <Secret-Key-from-garage>
AWS_ENDPOINTS: http://<GARAGE_SERVER_IP>:3900
AWS_REGION: us-east-1
Terminal window
kubectl apply -f minio-secret.yaml

HelmRelease Update

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
name: longhorn
namespace: longhorn-system
spec:
interval: 1h
url: https://charts.longhorn.io
---
19 collapsed lines
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: longhorn
spec:
interval: 1h
chart:
spec:
chart: longhorn
version: 1.10.0
sourceRef:
kind: HelmRepository
name: longhorn
namespace: longhorn-system
values:
defaultSettings:
backupTarget: "s3://longhorn@us-east-1/"
backupTargetCredentialSecret: "minio-secret"
backupstorePollInterval: "300"
Warning

In Longhorn 1.10.0+, backup settings are under defaultSettings, not a separate defaultBackupStore section.

Verify

Access the Garage WebUI at http://<SERVER_IP>:3909. Then test a backup from Longhorn UI → Volume → Create Backup and confirm it appears in the longhorn bucket.

Terminal window
garage bucket list
garage bucket info longhorn

Troubleshooting

WebUI “Unknown Error” — confirm the [admin] section is in garage.toml and restart:

Terminal window
docker-compose restart
docker logs garage

Port 3903 connection refused:

Terminal window
netstat -tlnp | grep 3903

Longhorn can’t reach Garage:

Terminal window
kubectl run -it --rm debug --image=amazon/aws-cli --restart=Never -- \
s3 ls s3://longhorn --endpoint-url http://<GARAGE_IP>:3900

Resources