Hello Marcel, I don't know if this is also of interest to you, but you could use DRBD [0] to perform a kind of network RAID1 that would keep both servers mirrored in real time. Using heartbeat [1] you can detect if the other server is down and start the remote vservers in the local server.
See also [2]. [0] http://www.drbd.org [1] http://www.linux-ha.org/ [2] http://linux-vserver.org/Vserver+DRBD Regards, Rúben On Sun, Mar 19, 2006 at 04:42:22PM +0100, Marcel Gsteiger wrote: > Hi all > > I successfully installed two similar boxes running FC4 w/linux-vserver, > implementing a hot-standby of each vserver on the other box. All VServers are > simply installed on both boxes in the same directory structure; half of the > servers are active on box A and standby on box B, the other half is active on > box B and standby on box A. This is what I call a "poor man's warm standby > solution", i.e. without the need for a SAN infrastructure. This setup does > "load balancing" by having the active vserver on the CPU of my choice > statically. On both boxes a cronjob runs once every hour (of course not both > at the same time) that takes a LVM snapshot of the vserver partition and then > uses rsync to mirror the active server(s) to the standby box. This setup > works very well now for several weeks. Depending on the data changing and > overall load, the hourly script needs about 1-5 minutes to sync 8 vservers in > my case, together occupying about 23 GB of disk space (running on a x86_64 > 2.8GHz Dual Xeon box w/software RAID 1 SATA disks). > > For anyone interested, find my scripts below that do the work of syncing the > servers and migrate a vserver from one mirror to the other (switchover).. As > a prerequisite, each of the two boxes should be able to connect to the other > one via SSH without using passwords, i.e. by public key authentication. The > same scripts are installed on both servers, they automatically find out where > they are running and what to do. I also have a .lan domain on my DNS to avoid > having IP addresses in the script, but of course this can be adapted easily. > In my own setup, I have a few additional site-specific scripts that take care > of networking issues (firewalling, VLAN setup, static routes etc), this is > necessary in my case since my vservers have RFC1918 IPs and are accessible > from the outside ony via NAT. > > Syncing the /etc/vserver directory has to be done manually, my script just > rsyncs the /vservers partition. > > Of course, the scripts come without any warranty whatsoever, all bugs have > been added by me ;) > > Regards > --Marcel > > <file rsync_vservers> > #!/bin/bash > # synchronizes all running vservers to mirror and checks the status. > # Does not automatically launch vservers; this has to be done manually. > # stdout can typically be piped into logger from within a cronjob, e.g. > # /etc/rsync_vservers | /usr/bin/logger -t rsync_vservers > # 3-MAR-2006 milprog/mg > # 10-MAR-2006 milprog/mg : adapted to retry snapshot creation and wait 1 > second before mounting > > if [ "$HOSTNAME" = "milprogmail.ch" ] > then > THISHOST=milprogmail.milnet.lan > else > THISHOST=$HOSTNAME > fi > > MIRROR1=lepus.milnet.lan > MIRROR2=milprogmail.milnet.lan > # maximum 1 MB/s bandwidth please during rsync > BWLIMIT=1024 > SNAPSHOTSIZE=4096M > > VSERVERVG=/dev/vg00 > VSERVERLV=lv00_vservers > VSERVERSNAPSHOT=vservers_snapshot > SNAPSHOTMOUNT=/mnt/vservers-snapshot > VSERVERBASE=/vservers > > # no need to change anything below (hopefully) > > VSERVERCMD=/usr/sbin/vserver > RSYNCCMD=/usr/bin/rsync > LVCREATECMD=/usr/sbin/lvcreate > LVREMOVECMD=/usr/sbin/lvremove > MOUNTCMD=/bin/mount > UMOUNTCMD=/bin/umount > SSHCMD=/usr/bin/ssh > LVSCMD=/usr/sbin/lvs > AWKCMD=/bin/awk > BASENAMECMD=/bin/basename > DATECMD=/bin/date > SYNCCMD=/bin/sync > SLEEPCMD=/bin/sleep > > if [ "$THISHOST" = "$MIRROR1" ] > then > MIRROR=$MIRROR2 > fi > if [ "$THISHOST" = "$MIRROR2" ] > then > MIRROR=$MIRROR1 > fi > if [ "$MIRROR" = "" ] > then > echo "ERROR: VServer resync must be installed either on $MIRROR1 or > $MIRROR2, not here ($THISHOST)" > exit 1 > fi > TIMESTART=`$DATECMD +%s` > RETRIES=0 > MAXRETRIES=5 > $SYNCCMD > if $LVSCMD --noheadings ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null 2>&1 > then > echo "ERROR: Snapshot overlap; cannot rsync from $THISHOST to $MIRROR" > else > # echo "creating LV..." > # $LVSCMD > if $LVCREATECMD -L${SNAPSHOTSIZE} -s --chunksize 4k -n > ${VSERVERVG}/${VSERVERSNAPSHOT} ${VSERVERVG}/${VSERVERLV} >/dev/null 2>&1 && > $MOUNTCMD -t ext2 -ni -o noatime,ro ${VSERVERVG}/${VSERVERSNAPSHOT} > ${SNAPSHOTMOUNT} >/dev/null 2>&1 > then > : > else > # echo "retrying to create snapshot..." > while [ $RETRIES -lt "$MAXRETRIES" ] > do > RETRIES=`expr $RETRIES + 1` > $SYNCCMD > $SLEEPCMD 1 > $UMOUNTCMD -ni $SNAPSHOTMOUNT > $SLEEPCMD 1 > $LVREMOVECMD -f ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null 2>&1 > $SLEEPCMD 1 > $SYNCCMD > $LVREMOVECMD -f ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null 2>&1 > $SLEEPCMD 1 > if $LVCREATECMD -L${SNAPSHOTSIZE} -s --chunksize 4k -n > ${VSERVERVG}/${VSERVERSNAPSHOT} ${VSERVERVG}/${VSERVERLV} >/dev/null 2>&1 && > $MOUNTCMD -t ext2 -ni -o noatime,ro ${VSERVERVG}/${VSERVERSNAPSHOT} > ${SNAPSHOTMOUNT} >/dev/null 2>&1 > then > break > fi > done > if [ $RETRIES -eq "$MAXRETRIES"] > then > echo "ERROR: cannot create snapshot after `expr $MAXRETRIES-1` retries" > $LVREMOVECMD -f ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null 2>&1 > exit 2 > fi > fi > NACTIVE=0 > NSTANDBY=0 > NINACTIVE=0 > NERROR=0 > for i in /etc/vservers/*; do > VSERVER=`$BASENAMECMD $i` > if $VSERVERCMD $VSERVER running > then > # echo "vserver $VSERVER running on $THISHOST" > if $SSHCMD $MIRROR $VSERVERCMD $VSERVER running > then > NERROR=`expr $NERROR + 1` > echo "ERROR: vserver $VSERVER running both on $THISHOST and $MIRROR" > else > NACTIVE=`expr $NACTIVE + 1` > # echo "rsyncing vserver $VSERVER from $THISHOST to $MIRROR..." > if $RSYNCCMD -aqzpotSx4 --numeric-ids --bwlimit $BWLIMIT --recursive > --safe-links --delete --force -e ssh ${SNAPSHOTMOUNT}/${VSERVER}/ > $MIRROR:${VSERVERBASE}/${VSERVER}/ > then > # echo "rsync vserver $VSERVER from $THISHOST to $MIRROR done" > : > else > echo "ERROR: cannot rsync $VSERVER from $THISHOST to $MIRROR" > fi > fi > else > if $SSHCMD $MIRROR $VSERVERCMD $VSERVER running > then > # echo "vserver $VSERVER running on $MIRROR" > NSTANDBY=`expr $NSTANDBY + 1` > else > NINACTIVE=`expr $NSTANDBY + 1` > echo "WARNING: vserver $VSERVER inactive both on $THISHOST and $MIRROR" > fi > fi; > done > $SYNCCMD > if $UMOUNTCMD -ni $SNAPSHOTMOUNT > then > # echo "unmount of $SNAPSHOTMOUNT done" > : > else > echo "ERROR: cannot unmount LV snapshot" > fi > ## else > ## echo -n `$DATECMD +"%x %X" ` > ## echo " ERROR: cannot mount LV snapshot" > ## fi > SNAPPERCENT=`$LVSCMD --noheadings ${VSERVERVG}/${VSERVERSNAPSHOT} | $AWKCMD > '{print $6}'` > $SYNCCMD > if $LVREMOVECMD -f ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null 2>&1 > then > TIMEEND=`$DATECMD +%s` > echo "NOTICE: rsync_vservers complete; elapsedsecs=`expr $TIMEEND - > $TIMESTART`; snapshotsize=$SNAPSHOTSIZE; snap%=$SNAPPERCENT; active=$NACTIVE; > standby=$NSTANDBY; inactive=$NINACTIVE; error=$NERROR; > snapshotretries=$RETRIES" > # $LVSCMD > else > echo "ERROR: cannot remove LV snapshot" > fi > fi > </file> > > > > <file migrate_vserver> > #!/bin/bash > # gets 2 parameters: vserver_name source_server > # ensures that vserver_name is running on source_server and standby on the > corresponding mirror > # stops the vserver on source_server, does rsync of the remaining data and > restarts on mirror > # if only the vserver_name is given, the script assumes that the server it's > running on should become the new active server > # > # 19-MAR-2006 milprog/mg > > if [ "$HOSTNAME" = "milprogmail.ch" ] > then > THISHOST=milprogmail.milnet.lan > else > THISHOST=$HOSTNAME > fi > > MIRROR1=lepus.milnet.lan > MIRROR2=milprogmail.milnet.lan > SNAPSHOTSIZE=4096M > VSERVERVG=/dev/vg00 > VSERVERLV=lv00_vservers > VSERVERSNAPSHOT=vservers_snapshot > SNAPSHOTMOUNT=/mnt/vservers-snapshot > # parameters for taking the snapshot: number of retries, interval in seconds > MAXSNAPSHOTRETRIES=3 > RETRYINTERVAL=250 > > # no need to change anything below (hopefully) > > LVCREATECMD=/usr/sbin/lvcreate > LVREMOVECMD=/usr/sbin/lvremove > LVSCMD=/usr/sbin/lvs > VSERVERCMD=/usr/sbin/vserver > VSERVERROOT=/vservers > RSYNCCMD=/usr/bin/rsync > SSHCMD=/usr/bin/ssh > AWKCMD=/bin/awk > BASENAMECMD=/bin/basename > DATECMD=/bin/date > SYNCCMD=/bin/sync > SLEEPCMD=/bin/sleep > > if [ "$THISHOST" = "$MIRROR1" ] > then > MIRROR=$MIRROR2 > fi > if [ "$THISHOST" = "$MIRROR2" ] > then > MIRROR=$MIRROR1 > fi > if [ "$MIRROR" = "" ] > then > echo "ERROR: $0 must be installed either on $MIRROR1 or $MIRROR2, not here > ($THISHOST)" > exit 1 > fi > > VSERVER=$1 > if [ "$VSERVER" == "" ] > then > echo "ERROR: VServer name missing" > exit 2 > fi > MIGRATEFROM=$2 > if [ "$MIGRATEFROM" == "" ] > then > MIGRATEFROM=$MIRROR > MIGRATETO=$THISHOST > else > if [ $MIGRATEFROM == "$THISHOST" ] > then > MIGRATETO=$MIRROR > else > if [ $MIGRATEFROM == "$MIRROR" ] > then > MIGRATETO=$THISHOST > else > echo "ERROR: $MIGRATEFROM is not a valid host for this script (use > $THISHOST or $MIRROR)" > exit 3 > fi > fi > fi > if [ $MIGRATEFROM == "$THISHOST" ] > then > RUNFROM="" > RUNTO="$SSHCMD $MIGRATETO " > else > RUNFROM="$SSHCMD $MIGRATEFROM " > RUNTO="" > fi > > if $RUNFROM $VSERVERCMD $VSERVER running >/dev/null 2>&1 > then > if $RUNTO $VSERVERCMD $VSERVER running >/dev/null 2>&1 > then > echo "ERROR: vserver $VSERVER active on both $MIGRATEFROM and $MIGRATETO" > exit 5 > else > if [ $? -gt 1 ] > then > echo "ERROR: vserver $VSERVER unknown on $MIGRATETO as a standby server" > exit 6 > else > if $RUNFROM $LVCREATECMD -L${SNAPSHOTSIZE} -s --chunksize 4k -n > ${VSERVERVG}/${VSERVERSNAPSHOT} ${VSERVERVG}/${VSERVERLV} >/dev/null 2>&1 > then > TIMESTART=`$DATECMD +%s` > if $RUNFROM $VSERVERCMD $VSERVER stop >/dev/null 2>&1 > then > if $RUNFROM $RSYNCCMD -aqzpotSx4 --numeric-ids --recursive --safe-links > --delete --force -e ssh ${VSERVERROOT}/${VSERVER}/ > $MIGRATETO:${VSERVERROOT}/${VSERVER}/ > then > if $RUNTO $VSERVERCMD $VSERVER start >/dev/null 2>1 > then > TIMEEND=`$DATECMD +%s` > echo "NOTICE: vserver $VSERVER migrated from $MIGRATEFROM to > $MIGRATETO; elapsedsecs=`expr $TIMEEND - $TIMESTART`" > else > if $RUNFROM $VSERVERCMD $VSERVER start >/dev/null 2>&1 > then > echo "ERROR: cannot start $VSERVER after migration from $MIGRATEFROM > to $MIGRATETO; server was restarted again on $MIGRATEFROM" > exit 8 > else > echo "ERROR: cannot start $VSERVER after migration from $MIGRATEFROM > to $MIGRATETO; server could not be restarted again on $MIGRATEFROM and is now > dead on both mirrors" > exit 9 > fi > fi > else > if $RUNFROM $VSERVERCMD $VSERVER start >/dev/null 2>&1 > then > echo "ERROR: cannot rsync $VSERVER for migration from $MIGRATEFROM > to $MIGRATETO; server was restarted again on $MIGRATEFROM" > exit 10 > else > echo "ERROR: cannot rsync $VSERVER for migration from $MIGRATEFROM > to $MIGRATETO; server could not be restarted again on $MIGRATEFROM and is now > dead on both mirrors" > exit 11 > fi > fi > else > echo "ERROR: vserver $VSERVER cannot be stopped on $MIGRATEFROM" > exit 7 > fi > if $RUNFROM $LVREMOVECMD -f ${VSERVERVG}/${VSERVERSNAPSHOT} >/dev/null > 2>&1 > then > : > else > echo "ERROR: snapshot ${VSERVERVG}/${VSERVERSNAPSHOT} cannot be > removed on $MIGRATEFROM after successful migration" > exit 13 > fi > else > echo "ERROR: cannot migrate $VSERVER from $MIGRATEFROM to $MIGRATETO > because snapshot ${VSERVERVG}/${VSERVERSNAPSHOT} cannot be established > (possible overlap with backup/cronjob)" > exit 12 > fi > fi > fi > else > echo "ERROR: vserver $VSERVER not running on $MIGRATEFROM" > exit 4 > fi > </file> > > _______________________________________________ > Vserver mailing list > [email protected] > http://list.linux-vserver.org/mailman/listinfo/vserver -- Rúben Leote Mendes - [EMAIL PROTECTED] _______________________________________________ Vserver mailing list [email protected] http://list.linux-vserver.org/mailman/listinfo/vserver
