Sean Dague wrote:
I also encourage others to post their solutions to the backup space,
as it's always good to learn new techniques people are using.
I don't have a network drive, but I do have an external HD (USB/eSATA)
and a DVD burner. Here are the three scripts I use.
Every hour, I copy any new/changed stuff from ~/class and ~/book* (both
on the internal HD) to a FAT32 partition on the external HD.
Every night, I copy selected directories into a tarball on the "backup"
ext3 partition of my external HD. This script copies the tarball, along
with a list of its contents, to weekly and/or monthly subdirectories as
appropriate, and purges old daily and weekly backups. Also, the FAT32
partition of my external HD contains programs for both Windows and Mac
systems to access the ext3 partition and to extract files from the
tarballs, so if necessary I can just bring my external HD to anyone
else's computer and retrieve files there, to my flash drive or wherever.
Every week, I burn the important stuff to a DVD. (This script only
creates the .iso file, so I do the burning manually, with verify on of
course.) I use three DVD+RWs and rotate them. Also, about once a
month, I burn this onto a DVD+/-R and store it at my parents' house as
my offsite backup. For that reason, the DVD is not a tarball, but the
unaltered files in their original directory structure. If necessary, I
could talk my parents through emailing me any files from this disc.
And the really important part of ANY backup method: I've tested that I
can restore files from these backups and they are identical to the
originals!!!
Anyone is welcome to use these scripts, and especially to improve them!
(I'd appreciate a copy of any improvements or suggestions.) I've tried
to put all the stuff a user might want to (or need to) change at the
beginning of each script. Also, the daily and weekly scripts
intentionally have names starting with 'z' so they're run /after/ all
the other cron jobs have updated their data. I hope these are useful to
someone!
Adam
[adam@eris ~]$ cat /etc/cron.hourly/cpclass
#!/bin/bash
cp -Lru --preserve=timestamps $1 /home/adam/class /mnt/win_e/
cp -Lru --preserve=timestamps $1 /home/adam/book* /mnt/win_e/
[adam@eris ~]$ cat /etc/cron.daily/zwieback.cron
#!/bin/bash
# zwieback.cron -- back up specified directories and files AR 03/05/11
# name chosen so backup is done after all other cron jobs
# list of directories and files for full, weekly, monthly backup
BACKUP_LIST="/home /accounts /root /etc /usr/local \
/var/lib /var/log /var/spool/mail /var/spool/postfix \
/boot/grub/menu.lst \
/mnt/sda7/etc /mnt/sda7/usr/local /mnt/sda7/var/lib
/mnt/sda7/var/log \
/mnt/sda7/var/spool/mail /mnt/sda7/var/spool/postfix \
--exclude=/home/adam/.thumbnails --exclude=.mozilla/*/*/Cache \
--exclude=.thunderbird/*/Cache --exclude=/accounts/adam/mp3 \
--exclude=/accounts/adam/.pan2/article-cache \
--exclude=*.iso --exclude=tmp --exclude=/var/lib/named/proc \
--exclude=/accounts/adam/book1 --exclude=/accounts/adam/class"
WEEKLY_BACKUP_LIST="$BACKUP_LIST"
MONTHLY_BACKUP_LIST="$WEEKLY_BACKUP_LIST"
# extension for backup file, indicates type of compression to be used by
'tar'
B_EXT="tgz" ; GZIP="-6" # gzip default is -6
# device and directory for backups, can't contain spaces
B_DEV=/dev/sdb8
B_DIR=/mnt/backup
# warn if B_DIR more than USED_WARN per cent full
USED_WARN=75
unalias -a
umask 077
# could use either %Y (YYYY) or %y (YY) for next item
THIS_MONTH=$(date +%y%m)
TODAY="${THIS_MONTH}$(date +%d)"
### comment or uncomment 'mount' and 'umount' (near end) as desired ???
mount $B_DEV $B_DIR
RC=$?
# allow 'mount' return code 32 or 64
[ $(( $RC % 32 )) -ne 0 ] && exit $RC
for D in ${B_DIR} ${B_DIR}/weekly ${B_DIR}/monthly ; do
if [ ! -d $D ] || [ ! -w $D ] ; then
echo "$D is not a writeable directory!"
exit 31
fi
done
# DO_FULL DO_WEEKLY DO_MONTHLY each nonzero if that type of backup is needed
DO_FULL=1 # For now, full bkup every time, diff. saves only 30% of
space
DO_WEEKLY=0
DO_MONTHLY=0
if [ ! -e ${B_DIR}/last_full_backup ] ; then
DO_FULL=1
fi
# Do we have a weekly full backup from the past 7 days?
### There has to be a neater way to determine this!
if [ $(find ${B_DIR}/weekly -daystart -mtime -7 -type f \
-regex "${B_DIR}/weekly/[012][0-9]+\.${B_EXT}" \
-print -quit | wc -c) -eq 0 ] ; then
DO_WEEKLY=1
BACKUP_LIST=$WEEKLY_BACKUP_LIST
GZIP="-9"
fi
# Do we have a monthly full backup from this month?
ls ${B_DIR}/monthly/${THIS_MONTH}[0123][0123456789].${B_EXT} >/dev/null 2>&1
if [ $? -ne 0 ] ; then
DO_MONTHLY=1
BACKUP_LIST=$MONTHLY_BACKUP_LIST
GZIP="-9"
fi
export GZIP
REPORT=$(mktemp)
echo "Starting $0 at $(date)" > $REPORT
echo >> $REPORT
if [ $DO_FULL -ne 0 -o $DO_WEEKLY -ne 0 -o $DO_MONTHLY -ne 0 ] ; then
# do full/weekly/monthly backup; overwrite backup with same date
touch ${B_DIR}/starting_backup
tar -acf ${B_DIR}/${TODAY}.${B_EXT} $BACKUP_LIST 2>/dev/null
echo "${B_DIR}/${TODAY}.${B_EXT} created; tar returned $?" >>
$REPORT
mv -f ${B_DIR}/starting_backup ${B_DIR}/last_full_backup
else
# do differential backup
tar -N ${B_DIR}/last_full_backup \
-acf ${B_DIR}/${TODAY}d.${B_EXT} $BACKUP_LIST 2>/dev/null
echo "${B_DIR}/${TODAY}d.${B_EXT} created; tar returned $?" >>
$REPORT
tar -tvf ${B_DIR}/${TODAY}d.${B_EXT} > ${B_DIR}/${TODAY}d.list
fi
if [ $DO_WEEKLY -ne 0 ] ; then
cp -p ${B_DIR}/${TODAY}.${B_EXT} ${B_DIR}/weekly/
echo "${TODAY}.${B_EXT} copied to ${B_DIR}/weekly/" >> $REPORT
tar -tvf ${B_DIR}/weekly/${TODAY}.${B_EXT} > \
${B_DIR}/weekly/${TODAY}.list
fi
if [ $DO_MONTHLY -ne 0 ] ; then
cp -p ${B_DIR}/${TODAY}.${B_EXT} ${B_DIR}/monthly/
echo "${TODAY}.${B_EXT} copied to ${B_DIR}/monthly/" >> $REPORT
tar -tvf ${B_DIR}/monthly/${TODAY}.${B_EXT} > \
${B_DIR}/monthly/${TODAY}.list
fi
find ${B_DIR}/weekly -daystart -regex "${B_DIR}/weekly/[012][0-9]+\..*" \
-mtime +34 -type f -delete
find ${B_DIR} -daystart -regex "${B_DIR}/[012][0-9]+\..*" \
-mtime +7 -type f -delete
echo >> $REPORT
ls -AhlRrt --ignore="lost+found" $B_DIR >> $REPORT
echo >> $REPORT
df -Th $B_DIR >> $REPORT
USED=$(tail -1 $REPORT | tr -s ' ' | cut -d ' ' -f 6 | tr -d '%')
echo >> $REPORT
umount -l $B_DEV && [ $DO_WEEKLY -ne 0 ] && e2fsck -pf $B_DEV >> $REPORT
2>&1
[ $USED -ge $USED_WARN ] && echo "WARNING: $B_DIR is ${USED}% full!" >>
$REPORT
mail -s "Backup${TODAY}" root < $REPORT
rm $REPORT
exit 0
[adam@eris ~]$ cat /etc/cron.weekly/ziso.cron
#!/bin/bash
# /etc/cron.weekly/ziso.cron -- compile ISO for DVD backup AR 03/05/11
PROG="genisoimage" # or "mkisofs" if you prefer
# list of directories and files to backup
INCLUDE="/home /accounts /root /etc /usr/local \
/var/lib /var/log /var/spool/mail /var/spool/postfix \
/boot/grub/menu.lst"
# one humongous extended regexp of what NOT to include
#EXCLUDE="^/home/adam/class|[.]iso$"
EXCLUDE="[.]iso$|^/var/lib/named/proc/"
#EXCLUDE="[.]iso$|^/home/adam/[.]gvfs$"
# output ISO file name
ISONAME="Backup$(date +%Y%m%d)"
# full path of ISO file
ISOFILE="/home/adam/${ISONAME}.iso"
# $PROG gets upset if things change while it's running,
# so retry it this many times in hopes it'll eventually succeed
RETRIES=2
# $EXCLUDE can't be null or 'grep -Ev' will exclude everything
[[ -z "$EXCLUDE" ]] && EXCLUDE="^$"
PATHLIST=$(mktemp)
REPORT=$(mktemp)
echo "$(date) -- $0 started" > $REPORT
RC=-1
while [ $RC -ne 0 ] ; do
find $INCLUDE -type f 2>/dev/null | grep -Ev "$EXCLUDE" | \
sed -e 's:[=\]:\\&:g' -e 's:[/]\(.*\):\1=&:g' > $PATHLIST
$PROG -output "$ISOFILE" -graft-points -path-list "$PATHLIST" \
-volid "$ISONAME" -appid "$0" -disable-deep-relocation \
-rational-rock -joliet -joliet-long -quiet 2>> $REPORT
RC=$?
if [ $RC -eq 0 ] ; then
break
fi
if [ $RETRIES -gt 0 ] ; then
echo "$PROG returned $RC, retrying..." >> $REPORT
let RETRIES-=1
else
echo "$PROG returned $RC, aborting" >> $REPORT
echo "${ISOFILE} could not be created" >> $REPORT
mail -s "${ISONAME}.iso error" root < $REPORT
rm $PATHLIST $REPORT $ISOFILE 2> /dev/null
exit $RC
fi
done
echo >> $REPORT
echo "$ISOFILE successfully created" >> $REPORT
ls -lh "$ISOFILE" >> $REPORT
mail -s "${ISONAME}.iso Created" root < $REPORT
rm $PATHLIST $REPORT
exit 0
[adam@eris ~]$
_______________________________________________
Mid-Hudson Valley Linux Users Group http://mhvlug.org
http://mhvlug.org/cgi-bin/mailman/listinfo/mhvlug
Upcoming Meetings (6pm - 8pm) MHVLS Auditorium
Apr 6 - Introduction to IPv6
May 4 - Inkscape
Jun 1 - Zimbra
