meezaan/library/db-backup

0 forks

id: 112

11 files 13 commits 1 branches 0 tags

428 Lines

Shell 53%
Markdown 28%
Yaml 18.9%

README.md

Database Backups to S3

This utility can back up your MySQL and PGSQL databases to an S3 bucket of your choice at an object storage provider of your choice.

Optionally, it can upload the same backup to a secondary S3 target for geo-redundancy or cross-account replication — the dump runs once and the compressed archive is uploaded to both targets sequentially.

Build the Container

The docker container is available to use from 1x.ax/meezaan/library/db-backup:latest or you can use the Dockerfile to build it yourself.

Environment Variables

Required

Variable	Description
`DBTYPE`	Database type: `mysql` or `pgsql`
`USER`	Database username
`PASSWORD`	Database password
`DBNAME`	Database name to back up
`HOST`	Database host
`PORT`	Database port (e.g., `3306` for MySQL, `5432` for PostgreSQL)
`S3_ACCESS_KEY`	Primary S3 access key
`S3_SECRET_KEY`	Primary S3 secret key
`S3_HOST`	Primary S3 host (e.g., `s3.us-east-1.amazonaws.com`)
`S3_BUCKET`	Primary S3 bucket name
`S3_REGION`	Primary S3 region (e.g., `us-east-1`)

Optional (secondary S3 target)

When these are set, the backup is uploaded to both the primary and secondary S3 targets. The dump is only performed once — the same compressed archive is uploaded to both destinations.

If S3_SECONDARY_HOST and S3_SECONDARY_BUCKET are not set, the secondary upload is skipped entirely.

Variable	Description
`S3_SECONDARY_ACCESS_KEY`	Secondary S3 access key
`S3_SECONDARY_SECRET_KEY`	Secondary S3 secret key
`S3_SECONDARY_HOST`	Secondary S3 host
`S3_SECONDARY_BUCKET`	Secondary S3 bucket name
`S3_SECONDARY_REGION`	Secondary S3 region

Optional (transport)

Variable	Description
`S3_USE_HTTPS`	`false` only for an S3 endpoint without TLS, such as a local test store. Default `true`.

Failure behaviour

A backup that cannot complete exits non-zero, so a Kubernetes Job (and any alerting on failed Jobs) sees the failure. That covers:

a missing required variable;
an unknown DBTYPE;
a dump that fails, or that is incomplete (it is checked for the trailer mysqldump and pg_dump write last);
an upload that fails to either target.

Both targets are always attempted, so one failure does not cost you the other copy, but the run still fails. The dump itself is never printed to the log.

The MySQL client in the image is MariaDB's mysqldump. It works against MySQL 8.4 servers; MySQL-only options such as --source-data are not used. pg_dump refuses servers newer than itself: the image ships PostgreSQL 16 client tools.

Tests

tests/run.sh

Starts MySQL, PostgreSQL and a SeaweedFS S3 store on a throwaway Docker network. It checks that good backups land with their data, and that every failure above exits non-zero.

Deploy on Kubernetes

The file secret.yml is provided as a sample to create a secret that you can use to provide env vars to a Kubernetes cron. The cron.yml file shows an example of this.

Run on the CLI with Docker

Single S3 target

docker run \
    -e DBTYPE=mysql \
    -e USER=root \
    -e PASSWORD=password \
    -e DBNAME=myapp \
    -e HOST=localhost \
    -e PORT=3306 \
    -e S3_ACCESS_KEY="AKIA..." \
    -e S3_SECRET_KEY="..." \
    -e S3_HOST="s3.us-east-1.amazonaws.com" \
    -e S3_BUCKET="my-backups" \
    -e S3_REGION="us-east-1" \
    1x.ax/meezaan/library/db-backup:latest

Dual S3 targets (primary + secondary)

docker run \
    -e DBTYPE=mysql \
    -e USER=root \
    -e PASSWORD=password \
    -e DBNAME=myapp \
    -e HOST=localhost \
    -e PORT=3306 \
    -e S3_ACCESS_KEY="AKIA..." \
    -e S3_SECRET_KEY="..." \
    -e S3_HOST="s3.us-east-1.amazonaws.com" \
    -e S3_BUCKET="primary-backups" \
    -e S3_REGION="us-east-1" \
    -e S3_SECONDARY_ACCESS_KEY="AKIA..." \
    -e S3_SECONDARY_SECRET_KEY="..." \
    -e S3_SECONDARY_HOST="s3.eu-west-1.amazonaws.com" \
    -e S3_SECONDARY_BUCKET="dr-backups" \
    -e S3_SECONDARY_REGION="eu-west-1" \
    1x.ax/meezaan/library/db-backup:latest

Backup file location

Backups are stored in S3 at:

MySQL: s3://<bucket>/<dbname>/<dbname>-DD-Mon-YYYY-HH-MM.sql.tar.gz
PostgreSQL: s3://<bucket>/<dbname>/<dbname>-DD-Mon-YYYY.sql.tar.gz