I just added something to my backup script which should make it easy to
restore db and files to be consistent with each other (without a lot of orphan
files).  You simply need to keep a list of all the files present at the time
of mysqldumping.  These can then be used in the restore using 
rsync --files-from



#!/bin/bash

# today's date
DATE=`date -Idate`
# location of base2 files
USERFILES=/data/base2/files
# location to put backups - remember to keep off-site backups too
DEST=/somewhere/else

# mkdir $DEST/mysql-dumps
# mkdir $DEST/files


# dump the db
mysqldump --single-transaction --quick --add-drop-table --databases base2 
base2dynamic | gzip -c > $DEST/mysql-dumps/base2-dbs-$DATE.sql.gz

# dump the files (important: do not use --delete)
rsync -arv $USERFILES/ $DEST/files

# dump the file manifest
(cd $USERFILES ; find . -type f) > $DEST/files/manifest-$DATE



then to restore the files from 2008-01-01, use something like this:

mv $USERFILES $USERFILES.old
rsync -a --files-from $DEST/files/manifest-2008-01-01 $DEST/files $USERFILES 

(restoring the mysql db is self-explanatory I hope)



note that I haven't actually done a restore yet...

cheers,
Bob.


Nicklas Nordborg writes:
 > Bob MacCallum wrote:
 > > Thanks for all the ideas on this. 
 > > 
 > > re: userfiles/db sync, would this work..?
 > > 
 > > backup:
 > > 
 > > 1. mysqldump to date-stamp-named files
 > > 2. rsync -arv /data/base2/files/ /backup/base2/files
 > >    (note, no --delete option on purpose) 
 > > 
 > > restore:
 > > 
 > > 1. undump whichever mysqldump is needed
 > > 2. copy over all files that are older than the mysqldump
 > > 
 > 
 > There is nothing in BASE that guarantees that the name of a deleted file 
 > is not reused by another file later on. However, it is very unlikely 
 > that this should happen with the current implementation. I think you 
 > need to upload a file, delete it, upload another within the same second 
 > AND the system must choose the same random filename for both files. Even 
 > if this happens it would only matter if you started a mysqldump after 
 > the upload of the first file, but before the deletion of it.
 > 
 > For all practical reasons, I think you can assume that files will always 
 > have unique pathnames, and if rsync is just adding new files to the 
 > backup directory you should be able to restore the file system to any 
 > point in time.
 > 
 > > The one problem I see is that the restore can bring back files that had 
 > > been
 > > deleted in BASE.  I doubt these would interfere with the running of BASE
 > > however.  A small program could be written to delete any files that are not
 > > known about by the db (unless there is already something to do this).
 > 
 > No there is no such program.
 > 
 > > Alternatively, one can add the --delete option to rsync and live with only
 > > being able to restore the latest dump (which is OK for us for now).
 > 
 > /Nicklas
 > 
 > -------------------------------------------------------------------------
 > This SF.net email is sponsored by: Microsoft
 > Defy all challenges. Microsoft(R) Visual Studio 2008.
 > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
 > _______________________________________________
 > The BASE general discussion mailing list
 > basedb-users@lists.sourceforge.net
 > unsubscribe: send a mail with subject "unsubscribe" to
 > [EMAIL PROTECTED]

-- 
Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups |
Division of Cell and Molecular Biology | Imperial College London |
Phone +442075941945 | Email [EMAIL PROTECTED]

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
The BASE general discussion mailing list
basedb-users@lists.sourceforge.net
unsubscribe: send a mail with subject "unsubscribe" to
[EMAIL PROTECTED]

Reply via email to