Minimum Viable Personal Backup Solution

A simple backup solution for Linux desktops.

tl;dr:
I have a keypress combination to put my desktop to sleep. I’m replacing that with a script that will run a quick backup and then sleep.

The easiest and most robust strategy I could come up with involves restic, some minimal configuration, and Backblaze B2.

What? Why?

I want backups. And I want to think about them as little as possible.

I’ll use BackBlaze B2 for my offsite replication. This is an object-storage service much like Amazon’s S3. I use Backblaze because they have good prices and offer decent transparency into their operations. Over the years, I’ve appreciated their quarterly reports of hard drive failure rates, which they publish openly.

I’ll use Restic because it’s flexible and I know it’s easy to orchestrate from the command line. I don’t like running services like Duplicati on my computer all the time.

I want something that runs frequently and runs fast, but which won’t tax my upload bandwidth while I’m actively using my computer. I already have a keyboard shortcut in my window manager to suspend to RAM. My thought is to write a script to take a backup and sync to Backblaze, and then simply invoke that automatically every time the computer is about to suspend.

There’s probably a more graceful way to do this, but I’m aiming for the lowest expenditure of effort here. Because it’s a desktop, it doesn’t need to sleep immediately like a laptop does. So, the slightly ugly side of this solution (my computer will be awake for an extra few minutes until the backup completes after I tell it to sleep) is a non-issue.

This will be a basic 3-2-1 backup strategy. I want a local backup for ease of use. In case I nuke something on my computer that I meant to keep, this backup is handy in that circumstance. Then, to get a remote copy, let’s mirror that backup offsite to Backblaze.

I want this backup to cover my home directory with 1-day, 1-week, and 1-month copies.

To implement this scheme in restic, I will have one local repository and one remote repository. The remote will be a simple mirror.

Steps I took:

Install restic:

As everyone who uses Arch will tell you - I use Arch, so for me installation is:

pacman -S restic

Setup a Backup Repository:

I’m going to start out by setting up a restic repo on an external SSD which should be large enough to backup my home folder for quite some time. (I used baobab to check my disk space usage. It’s my favorite tool for this, as well as for clearing up space.)

Assuming the external backup drive is mounted at /mnt/bkup_usb_ext, and you’ve created a backups/home_slash_$USER directory there, you can run this:

restic init --repo /mnt/bkup_usb_ext/backups/home_slash_$USER

You’ll get a password for that repository that you need to keep track of.

I wrote a backup script with some excluded directories. This will take a single snapshot:

#!/bin/bash

# Here, we will take one snapshot.
# You should have already exported the restic password so this can use it.


LOCAL_RESTIC_REPO="/mnt/bkup_usb_ext/backups/home_slash_$USER"

restic -r $LOCAL_RESTIC_REPO backup /home/$USER --one-file-system \
	--exclude /home/$USER/large_files_you_do_not_want_to_back_up \
	--exclude /home/$USER/.cache \
	--verbose

As you can see, multiple exclude flags can be used to trim anything out of your backups. There’s more documentation here.

Setup Remote Repo:

There’s a lot you could unpack here, if you wanted to. Check out restic’s documentation here.

To make things as simple as possible, I copied the params from the local repo. For me, the command looked like this:

restic -r $REMOTE_RESTIC_REPO init --from-repo $LOCAL_RESTIC_REPO --copy-chunker-params

If you create the remote repo without copying the params from the local repo, you may have issues with deduplication. Doing it this way seems to ensure that everything “just works”.

Mirroring:

To mirror, you need to supply both the password of the source repository (local) and the password for the destination repository (remote).

You can use environment variables for this, among other things. Keeping with our mantra of “simplicity”, I’ve used the environment vars.

Then it’s just a restic copy -–from-repo command. (See the whole script below.)

Retention Policy:

With restic, the way to implement a retention policy is to take snapshots as desired and then, later, go back and forget some of them. Then, you can prune the repo to remove the unneeded blocks.

Since I’m taking snapshots at odd times (whenever I put my computer to sleep), I will set up my forget command with --keep-within-... flags. Here’s a dry-run of that, which you can run to see which snapshots would be deleted.

$RESTIC_CMD forget \
	--keep-last 1 \
	--keep-within-daily 7d \
	--keep-within-weekly 1m \
	--keep-within-monthly 6m \
	--keep-within-yearly 2y \
	--dry-run

Checking Structure, Checking Data:

Bitrot is real, and sooner or later a cosmic ray will get the better of every one of us…

$RESTIC_CMD check –-read-data-subset=1%

Simple Alerts in Case of Problems:

I will just throw pop-ups on my screen using zenity, with the presumption that I should eventually encounter these. Ultimately, it would be nice to send myself email but I’m not going to overthink this.

The Script:

If you want to copy and use this, you will have to go line-by-line to fill in your own information.

Another thing you’ll need, which I did not cover here, is a Backblaze bucket and application key. There are guides already that can help you set these up. In the script, you can see some stuff about AWS, but this is just because we’re using Backblaze’s B2 as an S3 clone. They provide an API compatible with S3 clients. Restic’s S3 support is better than its B2 support, and Backblaze’s S3-compatibility is solid, so I think this is the way to go.

Obviously this is provided without warranty, but feel free to use and modify any way you like. If you share it publicly, please cite this page.

#!/bin/bash

#I want this script to abort on the first sign of failure and give me a pop-up in case it does
set -e
trap alert ERR

alert() {
	zenity --warning --text "Check your restic log at /tmp/restic.log" --title "BACKUP ERROR!" &
	zenity_pid=$!

    	# Wait a little for the Zenity window to appear
    	sleep 1
    	#In i3, this will focus the window.  You will want to remove this if you don't use i3.
    	i3-msg '[class="^Zenity$"] focus'

    	# Make sure to wait for the Zenity dialog to be closed before exiting the script,
   	 	# or else the script might exit before anyone ever sees the dialog
    	wait $zenity_pid
}

# Initialize Local Environment
LOCAL_RESTIC_REPO="/mnt/bkup_usb_ext/backups/home_slash_$USER"
export RESTIC_PASSWORD="this would be the password to your REMOTE repo"
export RESTIC_FROM_PASSWORD="this would be the password to your LOCAL repo"

# Initialize Environment for Remote Repository:
S3_COMPAT_ENDPOINT="s3.us-west-000.backblazeb2.com" # This should also be updated!
BUCKET_NAME="YOUR_BUCKET_NAME_HERE"
BUCKET_DIR_PREFIX="restic"
TF_B2_BKUP_APPLICATION_KEY_ID="YOUR_BACKBLAZE_KEY_ID"
TF_B2_BKUP_APPLICATION_KEY="YOUR_BACKBLAZE_KEY"

# These two need to be exported as if they're AWS-related, because we're using the S3 driver of restic
# with BackBlaze's S3 compat layer
export AWS_ACCESS_KEY_ID=$TF_B2_BKUP_APPLICATION_KEY_ID
export AWS_SECRET_ACCESS_KEY=$TF_B2_BKUP_APPLICATION_KEY

RESTIC_CMD="restic -r $LOCAL_RESTIC_REPO"
REMOTE_RESTIC_CMD="restic --repo s3:${S3_COMPAT_ENDPOINT}/${BUCKET_NAME}/${BUCKET_DIR_PREFIX}"

# Clear logfile from last run
mv /tmp/restic.log /tmp/restic.log.1 || true
echo "### We're starting the backup now: $(date) ###" > /tmp/restic.log 2>&1


# Do a snapshot into the local repo
$RESTIC_CMD backup /home/$USER --one-file-system \
	--exclude /home/$USER/large_files_you_do_not_want_to_back_up \
	--exclude /home/$USER/.cache \
	--verbose \
	>> /tmp/restic.log 2>&1


# Local Cleanup.
# This gets rid of the extraneous snapshots we no longer need to store per my rules
echo "### Forget and Prune ###" >> /tmp/restic.log 2>&1
$RESTIC_CMD forget \
	--keep-last 2 \
	--keep-within-daily 7d \
	--keep-within-weekly 1m \
	--keep-within-monthly 6m \
	--keep-within-yearly 2y \
	--prune \
	>> /tmp/restic.log 2>&1


# Check structure + check actual data
# If you do backups less frequently, or are more concerned about integrity, you
# might increase that percentage.  As written, this checks 1% of the backup.
echo "### Check and Read Data ###" 	>> /tmp/restic.log 2>&1
$RESTIC_CMD check --read-data-subset=1% >> /tmp/restic.log 2>&1


###
# Remote Repo Stuff
###

echo "#########" 			>> /tmp/restic.log 2>&1
echo "### BEGIN REMOTE REPO BUSINESS ###" >> /tmp/restic.log 2>&1
echo "#########" 			>> /tmp/restic.log 2>&1

echo "### Mirror local to remote ###" 	>> /tmp/restic.log 2>&1
$REMOTE_RESTIC_CMD copy --from-repo $LOCAL_RESTIC_REPO >> /tmp/restic.log 2>&1

# Clean up extraneous snapshots and free up the space used by removed blocks
echo "### Forget and Prune ###" >> /tmp/restic.log 2>&1
$REMOTE_RESTIC_CMD forget \
--keep-last 2 \
--keep-within-daily 7d \
--keep-within-weekly 1m \
--keep-within-monthly 6m \
--keep-within-yearly 2y \
--prune \
>> /tmp/restic.log 2>&1


# Check structure + check actual data
echo "### Check and Read Data ###" 	>> /tmp/restic.log 2>&1
$REMOTE_RESTIC_CMD check --read-data-subset=1% >> /tmp/restic.log 2>&1