All,
Here's the scenario: you need a backup solution that will make a
complete backup of your server and automatically rotate the backup
snapshots using rsync.
These two scripts will take a snapshot once every four hours (depending
on your crontab entry) and automatically rotate the snapshots
"backward," that is, 'hourly.3' will be deleted, 'hourly.2' will rotate
to 'hourly.3,' 'hourly.1' will rotate to 'hourly.2' and so on.
Moreover, a snapshot of the server will be taken and the three "dailys"
will rotate in a similar fashion to the "hourlys" at midnight each day.
Here is an excellent article for doing this:
http://www-106.ibm.com/developerworks/ibm/library/i-metro17.html
Why rsync? From the article:
* rsync is fast, stable, and mature; it's been around for years and has
served the author well in a multitude of diverse situations.
* rsync was designed to do exactly what we were trying to do -- create
an exact replica of a local data repository on a remote system (or
visa-versa), and keep the two repositories in "sync" with each other.
* rsync is readily available; it ships with most popular Linux
distributions.
While Tom's article is excellent, there's just one catch: this article
talks about performing synchronized backups when an NFS mount is
available. For security reasons NFS was in my case simply not an option.
I adapted the scripts to work via SSH.
Alert: Automating the scripts via cron requires that root on the origin
server be able to log in to the remote server via SSH without a
password. Of course, the origin server must still authenticate itself
with public/private key authentication. The best way to do this securely
is outlined here:
http://www-106.ibm.com/developerworks/linux/library/l-keyc3/
For your convenience, here is the beginning of the SSH article here:
http://www-106.ibm.com/developerworks/library/l-keyc.html
To implement the system, perform the following:
1) Install the Korne Shell (ksh) if you do not already have it
2) Copy both hourly_snapshot.sh and daily_snapshot.sh to /usr/bin
3) Configure the origin server to log in as root via SSH without a
password
4) Create a file called /etc/excludes (example below) and add the
directories and/or files you do not wish to back up
5) Modify the REMOTE_SERVER variable in both scripts to point to the
server that will receive the backups and make them executable by root
(chmod 744 /usr/bin/script_name.sh)
6) Modify the SNAPSHOT_RW variable in both scripts to point to the
directory on the remote server you wish the system to backup to
7) Add the following to your crontab:
0 */4 * * * /usr/bin/hourly_snapshot.sh
0 0 * * * /usr/bin/daily_snapshot.sh
Since the scripts are so small I have attached them. I hope you will
forgive me as sending the information as a small file increases the
likelihood of correctness in terms of line breaks, etc. As I mentioned,
an example /etc/excludes file is also included.
I wrote the script for the Korne shell. Why? <\start flame here\>
because ksh is the Cadillac of all shells. It has air conditioning and
power steering. In short, it makes writing scripts really nice.
However, if any of you would like to modify this for Bash, please let me
know; I would be very interested to have that.
I hope you find these scripts useful.
Very Truly Yours,
--
--------------------------------------------------------------
| Cooper Stevenson | Em: [EMAIL PROTECTED] |
| General Computer | Ph: 541.924.9434 |
| "Open For Business" | Www: http://www.gencom.us |
--------------------------------------------------------------
#!/bin/ksh
# ---------------------------------------------------------------------
# Original script authored by Mike Rubel
# Modifications by Tom Syroid, December 17, 2002
# Adaptation for SSH by Cooper Stevenson, October 20, 2003
# /usr/bin/hourly_snapshot.sh
# ---------------------------------------------------------------------
#
# ------------- system commands used by this script -------------------
ID=/usr/bin/id;
ECHO=/bin/echo;
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;
TOUCH=/bin/touch;
RSYNC=/usr/bin/rsync;
# ----------- directory/ssh commands used by this script -----------------
SNAPSHOT_RW=/directory/name/of/remote/backup/store;
REMOTE_SERVER=hostname_of_your_server;
EXCLUDES=/etc/excludes;
LS_COMMAND="$(ssh root@"$REMOTE_SERVER" ls -l $SNAPSHOT_RW)";
# ------------- the script itself -------------------------------------
# make sure we're running as root
if (( `$ID -u` != 0 )); then { $ECHO "Sorry, must be root. Exiting..."; exit; } fi
# rotating snapshots of /
# step 1: delete the oldest snapshot, if it exists:
if [[ "$(print "$LS_COMMAND" | grep hourly.3 | awk '{ print $9 }')" == hourly.3 ]]
then
$(ssh root@"$REMOTE_SERVER" $RM -rf "$SNAPSHOT_RW"/hourly.3)
fi;
# step 2: shift the middle snapshots(s) back by one, if they exist
if [[ "$(print "$LS_COMMAND" | grep hourly.2 | awk '{ print $9 }')" == hourly.2 ]]
then
$(ssh root@"$REMOTE_SERVER" $MV "$SNAPSHOT_RW"/hourly.2 "$SNAPSHOT_RW"/hourly.3)
fi;
if [[ "$(print "$LS_COMMAND" | grep hourly.1 | awk '{ print $9 }')" == hourly.1 ]]
then
$(ssh root@"$REMOTE_SERVER" $MV "$SNAPSHOT_RW"/hourly.1 "$SNAPSHOT_RW"/hourly.2)
fi;
# step 3: make a hard-link-only (except for dirs) copy of the
# latest snapshot, if that exists
if [[ "$(print "$LS_COMMAND" | grep hourly.0 | awk '{ print $9 }')" == hourly.0 ]]
then
$(ssh root@"$REMOTE_SERVER" $CP -al "$SNAPSHOT_RW"/hourly.0 "$SNAPSHOT_RW"/hourly.1)
fi;
# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first. If it were not so, this would copy over the other
# snapshot(s) too!
$RSYNC -avze ssh --delete --delete-excluded --exclude-from="$EXCLUDES" / root@"$REMOTE_SERVER":$SNAPSHOT_RW/hourly.0 ;
# step 5: update the mtime of hourly.0 to reflect the snapshot time
$(ssh root@"$REMOTE_SERVER" $TOUCH $SNAPSHOT_RW/hourly.0) ;
# script ends
#!/bin/ksh
# ---------------------------------------------------------------------
# Original script authored by Mike Rubel
# Modifications by Tom Syroid, December 17, 2002
# Adaptation for SSH by Cooper Stevenson, October 20, 2003
# /usr/bin/daily_snapshot.sh
# ---------------------------------------------------------------------
# intended to be run daily as a cron job when hourly.3 contains the
# midnight (or whenever you want) snapshot; say, 13:00 for 4-hour
# snapshots.
#
# ------------- system commands used by this script -------------------
ID=/usr/bin/id;
ECHO=/bin/echo;
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;
TOUCH=/bin/touch;
RSYNC=/usr/bin/rsync;
# ----------- directory/ssh commands used by this script -----------------
SNAPSHOT_RW=/directory/name/of/remote/backup/store;
REMOTE_SERVER=hostname_of_your_server;
LS_COMMAND=$(ssh root@"$REMOTE_SERVER" ls -l $SNAPSHOT_RW);
# ------------- file locations ----------------------------------------
EXCLUDES=/etc/excludes;
# ------------- the script itself -------------------------------------
# make sure we're running as root
if (( `$ID -u` != 0 )); then { $ECHO "Sorry, must be root.
Exiting..."; exit; } fi
# step 1: delete the oldest snapshot, if it exists:
if [[ "$(print "$LS_COMMAND" | grep daily.2 | awk '{ print $9 }')" == daily.2 ]]
then
$(ssh root@"$REMOTE_SERVER" $RM -rf "$SNAPSHOT_RW"/daily.2)
fi;
# step 2: shift the middle snapshots(s) back by one, if they exist
if [[ "$(print "$LS_COMMAND" | grep daily.1 | awk '{ print $9 }')" == daily.1 ]]
then
$(ssh root@"$REMOTE_SERVER" $MV "$SNAPSHOT_RW"/daily.1 "$SNAPSHOT_RW"/daily.2)
fi;
if [[ "$(print "$LS_COMMAND" | grep daily.0 | awk '{ print $9 }')" == daily.0 ]]
then
$(ssh root@"$REMOTE_SERVER" $MV "$SNAPSHOT_RW"/daily.0 "$SNAPSHOT_RW"/daily.1)
fi;
# step 3: make a hard-link-only (except for dirs) copy of
# hourly.3, assuming that exists, into daily.0
if [[ "$(print "$LS_COMMAND" | grep hourly.3 | awk '{ print $9 }')" == hourly.3 ]]
then
$(ssh root@"$REMOTE_SERVER" $CP -al "$SNAPSHOT_RW"/hourly.3 "$SNAPSHOT_RW"/daily.0)
fi;
# note: do *not* update the mtime of daily.0; it will reflect
# when hourly.3 was made, which should be correct.
# script ends
# /etc/excludes
# This file tells rsync wich filesystems to exclude
# to invoke, add --exclude-from=/etc/excludes in the argument to 'rsync'
tmp/
mnt/
proc/
/dev/shm/
_______________________________________________
EuG-LUG mailing list
[EMAIL PROTECTED]
http://mailman.efn.org/cgi-bin/listinfo/eug-lug
