388 lines
11 KiB
Markdown
388 lines
11 KiB
Markdown
# Database Backup Container
|
|
|
|
A lightweight Alpine-based Docker container for backing up MariaDB and PostgreSQL databases with support for both local snapshots and Restic backups.
|
|
|
|
## Overview
|
|
|
|
This container automatically detects the database type (MariaDB or PostgreSQL) in a target container and creates backups using either local file storage with compression or Restic repositories. It supports backing up databases from other Docker containers by executing dump commands inside them.
|
|
|
|
## Features
|
|
|
|
- **Multi-database support**: Automatically detects and backs up MariaDB or PostgreSQL databases
|
|
- **Dual backup methods**: Choose between local compressed files or Restic repositories
|
|
- **Local snapshots**: Create compressed (zstd) local backup files with timestamps
|
|
- **Restic integration**: Uses Restic for efficient, encrypted, and deduplicated backups
|
|
- **Docker-in-Docker**: Can access and backup databases from other containers
|
|
- **Notification support**: Optional webhook notifications when backups complete
|
|
- **Lightweight**: Based on Alpine Linux for minimal footprint
|
|
|
|
## Prerequisites
|
|
|
|
- Docker with socket access (`/var/run/docker.sock`)
|
|
- Target container with either MariaDB or PostgreSQL client tools
|
|
- For Restic backups: Restic repository (local, S3, B2, etc.)
|
|
- For local backups: Mounted volume for backup storage
|
|
|
|
## Environment Variables
|
|
|
|
### Required (All Methods)
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `CONTAINER` | Name of the Docker container where the database is running |
|
|
|
|
### For Restic Backups
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `RESTIC_PASSWORD` | Password for the Restic repository |
|
|
| `RESTIC_REPOSITORY` | Restic repository URL (e.g., `s3:s3.amazonaws.com/bucket`, `/data/backups`) |
|
|
|
|
### For Local Backups
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `TARGET_DIR` | Directory where backup files will be stored |
|
|
|
|
### Optional
|
|
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `NOTIFY_URL` | Optional webhook URL to call when backup completes |
|
|
|
|
### Database-specific
|
|
|
|
#### For MariaDB containers:
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `MARIADB_ROOT_PASSWORD` | Root password for MariaDB |
|
|
|
|
#### For PostgreSQL containers:
|
|
| Variable | Description |
|
|
|----------|-------------|
|
|
| `DB_USER` | PostgreSQL username |
|
|
| `PGPASSWORD` | PostgreSQL password |
|
|
| `DB_NAME` | PostgreSQL database name |
|
|
|
|
## Usage
|
|
|
|
### Backup Methods
|
|
|
|
The container supports two backup methods:
|
|
|
|
1. **`restic`** - Backup to Restic repositories (cloud storage, remote servers)
|
|
2. **`local`** - Create compressed local backup files
|
|
|
|
### Restic Backups
|
|
|
|
```bash
|
|
docker run --rm \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-e CONTAINER=my-mariadb-container \
|
|
-e RESTIC_PASSWORD=my-secret-password \
|
|
-e RESTIC_REPOSITORY=s3:s3.amazonaws.com/my-backup-bucket \
|
|
-e MARIADB_ROOT_PASSWORD=db-password \
|
|
gitea.ceperka.net/rosti/db-backup:latest restic
|
|
```
|
|
|
|
### Local Backups
|
|
|
|
```bash
|
|
docker run --rm \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-v /host/backup/path:/backups \
|
|
-e CONTAINER=my-mariadb-container \
|
|
-e TARGET_DIR=/backups \
|
|
-e MARIADB_ROOT_PASSWORD=db-password \
|
|
gitea.ceperka.net/rosti/db-backup:latest local
|
|
```
|
|
|
|
### With Docker Compose
|
|
|
|
#### Restic Backup Setup
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
database:
|
|
image: mariadb:latest
|
|
environment:
|
|
MARIADB_ROOT_PASSWORD: secretpassword
|
|
MARIADB_DATABASE: myapp
|
|
volumes:
|
|
- db_data:/var/lib/mysql
|
|
|
|
backup-restic:
|
|
image: gitea.ceperka.net/rosti/db-backup:latest
|
|
depends_on:
|
|
- database
|
|
environment:
|
|
CONTAINER: database
|
|
RESTIC_PASSWORD: my-backup-password
|
|
RESTIC_REPOSITORY: s3:s3.amazonaws.com/my-backup-bucket
|
|
MARIADB_ROOT_PASSWORD: secretpassword
|
|
NOTIFY_URL: https://hc-ping.com/your-healthcheck-uuid
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
command: ["restic"]
|
|
|
|
volumes:
|
|
db_data:
|
|
```
|
|
|
|
#### Local Backup Setup
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
|
|
services:
|
|
database:
|
|
image: mariadb:latest
|
|
environment:
|
|
MARIADB_ROOT_PASSWORD: secretpassword
|
|
MARIADB_DATABASE: myapp
|
|
volumes:
|
|
- db_data:/var/lib/mysql
|
|
|
|
backup-local:
|
|
image: gitea.ceperka.net/rosti/db-backup:latest
|
|
depends_on:
|
|
- database
|
|
environment:
|
|
CONTAINER: database
|
|
TARGET_DIR: /backups
|
|
MARIADB_ROOT_PASSWORD: secretpassword
|
|
NOTIFY_URL: https://hc-ping.com/your-healthcheck-uuid
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
- ./backups:/backups
|
|
command: ["local"]
|
|
|
|
volumes:
|
|
db_data:
|
|
```
|
|
|
|
### Scheduled Backups with Cron
|
|
|
|
To run backups on a schedule, you can use cron or a container orchestrator:
|
|
|
|
#### Restic Backups
|
|
```bash
|
|
# Add to crontab for daily Restic backups at 2 AM
|
|
0 2 * * * docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -e CONTAINER=my-db -e RESTIC_PASSWORD=pass -e RESTIC_REPOSITORY=s3:s3.amazonaws.com/bucket -e MARIADB_ROOT_PASSWORD=dbpass gitea.ceperka.net/rosti/db-backup:latest restic
|
|
```
|
|
|
|
#### Local Backups
|
|
```bash
|
|
# Add to crontab for daily local backups at 3 AM
|
|
0 3 * * * docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /host/backups:/backups -e CONTAINER=my-db -e TARGET_DIR=/backups -e MARIADB_ROOT_PASSWORD=dbpass gitea.ceperka.net/rosti/db-backup:latest local
|
|
```
|
|
|
|
### Kubernetes CronJob
|
|
|
|
#### Restic Backup CronJob
|
|
|
|
```yaml
|
|
apiVersion: batch/v1
|
|
kind: CronJob
|
|
metadata:
|
|
name: database-backup-restic
|
|
spec:
|
|
schedule: "0 2 * * *" # Daily at 2 AM
|
|
jobTemplate:
|
|
spec:
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: backup
|
|
image: gitea.ceperka.net/rosti/db-backup:latest
|
|
args: ["restic"]
|
|
env:
|
|
- name: CONTAINER
|
|
value: "my-database-pod"
|
|
- name: RESTIC_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: backup-secrets
|
|
key: restic-password
|
|
- name: RESTIC_REPOSITORY
|
|
value: "s3:s3.amazonaws.com/my-backup-bucket"
|
|
- name: MARIADB_ROOT_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: db-secrets
|
|
key: root-password
|
|
volumeMounts:
|
|
- name: docker-sock
|
|
mountPath: /var/run/docker.sock
|
|
volumes:
|
|
- name: docker-sock
|
|
hostPath:
|
|
path: /var/run/docker.sock
|
|
restartPolicy: OnFailure
|
|
```
|
|
|
|
#### Local Backup CronJob
|
|
|
|
```yaml
|
|
apiVersion: batch/v1
|
|
kind: CronJob
|
|
metadata:
|
|
name: database-backup-local
|
|
spec:
|
|
schedule: "0 3 * * *" # Daily at 3 AM
|
|
jobTemplate:
|
|
spec:
|
|
template:
|
|
spec:
|
|
containers:
|
|
- name: backup
|
|
image: gitea.ceperka.net/rosti/db-backup:latest
|
|
args: ["local"]
|
|
env:
|
|
- name: CONTAINER
|
|
value: "my-database-pod"
|
|
- name: TARGET_DIR
|
|
value: "/backups"
|
|
- name: MARIADB_ROOT_PASSWORD
|
|
valueFrom:
|
|
secretKeyRef:
|
|
name: db-secrets
|
|
key: root-password
|
|
volumeMounts:
|
|
- name: docker-sock
|
|
mountPath: /var/run/docker.sock
|
|
- name: backup-storage
|
|
mountPath: /backups
|
|
volumes:
|
|
- name: docker-sock
|
|
hostPath:
|
|
path: /var/run/docker.sock
|
|
- name: backup-storage
|
|
persistentVolumeClaim:
|
|
claimName: backup-pvc
|
|
restartPolicy: OnFailure
|
|
```
|
|
|
|
## Backup File Naming
|
|
|
|
### Restic Backups
|
|
Backups are stored with the following naming convention:
|
|
- MariaDB: `mariadb_[CONTAINER]_[DB_NAME].sql`
|
|
- PostgreSQL: `pgsql_[CONTAINER]_[DB_NAME].sql`
|
|
|
|
### Local Backups
|
|
Local backup files include timestamps and are compressed:
|
|
- MariaDB: `YYYYMMDD_HHMMSS_mariadb_[CONTAINER]_[DB_NAME].sql.zst`
|
|
- PostgreSQL: `YYYYMMDD_HHMMSS_pgsql_[CONTAINER]_[DB_NAME].sql.zst`
|
|
|
|
Local backups use zstd compression for efficient storage and include atomic file operations (temporary files are renamed when complete).
|
|
|
|
## MariaDB Backup Features
|
|
|
|
The container includes comprehensive MariaDB backup options:
|
|
|
|
- **`--add-drop-trigger`** - Add DROP TRIGGER statements
|
|
- **`--add-drop-table`** - Add DROP TABLE statements
|
|
- **`--add-drop-database`** - Add DROP DATABASE statements
|
|
- **`--hex-blob`** - Use hexadecimal notation for binary data
|
|
- **`--compress`** - Compress data in backup
|
|
- **`--events`** - Include events in backup
|
|
- **`--routines`** - Include stored procedures and functions
|
|
- **`--single-transaction`** - Consistent backup for InnoDB tables
|
|
- **`--triggers`** - Include triggers in backup
|
|
|
|
## Notification Support
|
|
|
|
Both backup methods support optional webhook notifications:
|
|
|
|
```bash
|
|
# Set NOTIFY_URL to receive notifications when backups complete
|
|
-e NOTIFY_URL=https://hc-ping.com/your-healthcheck-uuid
|
|
```
|
|
|
|
The container will make a GET request to the URL after successful backup completion.
|
|
|
|
## Supported Restic Repositories
|
|
|
|
This container supports all Restic repository types:
|
|
|
|
- **Local**: `/path/to/backup/dir`
|
|
- **SFTP**: `sftp:user@host:/path/to/repo`
|
|
- **S3**: `s3:s3.amazonaws.com/bucket`
|
|
- **Azure**: `azure:container:/path`
|
|
- **Google Cloud**: `gs:bucket:/path`
|
|
- **Backblaze B2**: `b2:bucket:/path`
|
|
- **REST**: `rest:http://host:8000/repo`
|
|
|
|
## Building
|
|
|
|
```bash
|
|
# Build the image
|
|
task build
|
|
|
|
# Tag as latest
|
|
task tag-latest
|
|
|
|
# Push to registry
|
|
task push
|
|
```
|
|
|
|
Or manually:
|
|
|
|
```bash
|
|
docker build -t gitea.ceperka.net/rosti/db-backup:dev .
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **"Docker is not available"**
|
|
- Ensure Docker socket is mounted: `-v /var/run/docker.sock:/var/run/docker.sock`
|
|
- Check Docker daemon is running
|
|
|
|
2. **"Unsupported database type"**
|
|
- Verify the target container has `mariadb-dump` or `pg_dump` installed
|
|
- Check container name is correct
|
|
|
|
3. **Authentication errors**
|
|
- Verify database credentials are correct
|
|
- Ensure environment variables are properly set
|
|
|
|
4. **"TARGET_DIR does not exist"** (Local backups)
|
|
- Ensure the target directory is mounted as a volume
|
|
- Check directory permissions
|
|
|
|
5. **"Unknown backup method"**
|
|
- Ensure you specify either `local` or `restic` as the command argument
|
|
|
|
### Debug Mode
|
|
|
|
To debug issues, you can run the container interactively:
|
|
|
|
#### Restic Debug
|
|
```bash
|
|
docker run -it --rm \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
--entrypoint /bin/bash \
|
|
gitea.ceperka.net/rosti/db-backup:latest
|
|
```
|
|
|
|
#### Local Debug
|
|
```bash
|
|
docker run -it --rm \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-v /host/backup/path:/backups \
|
|
--entrypoint /bin/bash \
|
|
gitea.ceperka.net/rosti/db-backup:latest
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
- Store sensitive environment variables in secrets (Kubernetes secrets, Docker secrets, etc.)
|
|
- Use least-privilege access for Docker socket when possible
|
|
- Regularly rotate Restic repository passwords
|
|
- Consider using encrypted storage for backup repositories
|
|
|