Remote Incremental Backup Using RSync

I have used the following backup script to backup servers to an external backup server via ssh and rsync.

#!/bin/ksh

#
# A simple script that do incremental backup for hosts locally or remotely over SSH
#
# Install on new host:
# 1. Install dependencies; ksh and rsync (on both local and backup host)
# 2. Setup remote ssh access to backup host with no password (required for remote backup)
# 3. Configure dump databases script (optional)
# 4. Adapt "Host specific parameters"
# 
# Backup structure:
# Structure - <backup root>/<host>/<host>-<date>-<label>
#
# /home/backup/sauron/sauron-201005130900-latest (most recent backup)
#                    /sauron-201005130900-partial (backup folder during rsync)
#                    /sauron-201005120900 (previous backup's)
# /tmp/trash/ (default trash folder, must be cleaned regulary)
# /var/log/incremental-backup.log (default log file)
#
# Backup process: 
# 1. Create root folders and empty latest folders (only first backup)
# 2. Run db dump script (if available)
# 3. Create a new incremental backup to "partial" linked from "latest"
# 4. Remove "latest" label from previous backup and rename "partial" to "latest"  
# 5. Move oldest backup to trash, but keep monthly and daily last week
#
# Author: Peter Ljung (ljung.peter@gmail.com)
#

## Host specific configuration - You need to adapt these to your configuration!
SCRIPT_ROOT=/root/bin/backup
INCLUDE_FOLDERS="/etc /root/bin /var/www"   # Local folders to backup
EXCLUDE_FILE=$SCRIPT_ROOT/excludes.txt      # Exclude patterns (one per row)
LOCAL_HOST=`hostname`                       # "<host>" to override
BACKUP_HOST=library                         # "" if backup is done locally
BACKUP_USER=backup
BACKUP_ROOT=/home/backup
DB_DUMP=$SCRIPT_ROOT/dump_databases.sh      # "" if no db dump needed
LOG_FILE=/var/log/incremental-backup.log
TRASH=$BACKUP_ROOT/trash

## Common variables and root folder creation - Normally not touched
BACKUP_FOLDER=$BACKUP_ROOT/$LOCAL_HOST
BACKUP_SSH=$BACKUP_USER@$BACKUP_HOST:       # BACKUP_SSH == "" if BACKUP_HOST empty
BACKUP_SSH=${BACKUP_HOST:+$BACKUP_SSH}
SSH_CMD="ssh $BACKUP_USER@$BACKUP_HOST"     # SSH_CMD == "" if BACKUP_HOST empty
SSH_CMD=${BACKUP_HOST:+$SSH_CMD}
$SSH_CMD mkdir $BACKUP_FOLDER 2>/dev/null   # Create backup folder
                                            # Find latest backup named *-latest
PREV=`$SSH_CMD find $BACKUP_FOLDER -name "*-latest" -type d -maxdepth 1`
if [[ -z $PREV ]] then
  PREV=$BACKUP_FOLDER/empty-latest          # Create empty latest if non existing
  $SSH_CMD mkdir $PREV                      # e.g. .../sauron/empty-latest
fi
PARTIAL=$BACKUP_FOLDER/$LOCAL_HOST-`date +%Y%m%d%H%M`-partial
NEW=$BACKUP_FOLDER/$LOCAL_HOST-`date +%Y%m%d%H%M`-latest
RSYNC_OPTIONS="--archive --one-file-system --hard-links --human-readable --inplace \
  --numeric-ids --delete --delete-excluded"
RSYNC_DEBUG_OPTIONS="--verbose --progress --itemize-changes --dry-run --protect-args"
RSYNC_FORCE_OPTIONS="--force --ignore-errors --cvs-exclude"

## Pre backup script
$DB_DUMP                                    # Dump databases

## Perform backup
$SSH_CMD mkdir $PARTIAL                     # Create partial (TODO: Remove?)
                                            # RSync to partial linking from latest
rsync $RSYNC_OPTIONS --link-dest=$PREV --log-file=$LOG_FILE \
  --exclude-from=$EXCLUDE_FILE $INCLUDE_FOLDERS $BACKUP_SSH$PARTIAL
$SSH_CMD mv $PREV ${PREV%-latest}           # Remove ยด-latest` label from previous
$SSH_CMD mv $PARTIAL $NEW                   # Rename partial to latest

## Purge old backup
$SSH_CMD mkdir $TRASH 2>/dev/null
$SSH_CMD find $BACKUP_FOLDER -type d -maxdepth 1 -mmin +10080 -name "[a-z]*-[0-9]*" \
  ! -name "[a-z]*-[0-9]*-[a-z]*" ! -name "[a-z]*-[0-9][0-9][0-9][0-9][0-9][0-9]01*" \
  -exec "mv {} $TRASH \;"

MacOS X 10.6

The backup script works fine in MacOS X. The easiest way to execute it daily is to create a /etc/daily.local and call the script from there.

#!/bin/sh

# Do backup of /Documents
/Users/peterljung/bin/backup/incremental_backup.sh

Make sure to make the file executable.

sudo chmod +x /etc/daily.local

References