I just added something to my backup script which should make it easy to restore db and files to be consistent with each other (without a lot of orphan files). You simply need to keep a list of all the files present at the time of mysqldumping. These can then be used in the restore using rsync --files-from
#!/bin/bash # today's date DATE=`date -Idate` # location of base2 files USERFILES=/data/base2/files # location to put backups - remember to keep off-site backups too DEST=/somewhere/else # mkdir $DEST/mysql-dumps # mkdir $DEST/files # dump the db mysqldump --single-transaction --quick --add-drop-table --databases base2 base2dynamic | gzip -c > $DEST/mysql-dumps/base2-dbs-$DATE.sql.gz # dump the files (important: do not use --delete) rsync -arv $USERFILES/ $DEST/files # dump the file manifest (cd $USERFILES ; find . -type f) > $DEST/files/manifest-$DATE then to restore the files from 2008-01-01, use something like this: mv $USERFILES $USERFILES.old rsync -a --files-from $DEST/files/manifest-2008-01-01 $DEST/files $USERFILES (restoring the mysql db is self-explanatory I hope) note that I haven't actually done a restore yet... cheers, Bob. Nicklas Nordborg writes: > Bob MacCallum wrote: > > Thanks for all the ideas on this. > > > > re: userfiles/db sync, would this work..? > > > > backup: > > > > 1. mysqldump to date-stamp-named files > > 2. rsync -arv /data/base2/files/ /backup/base2/files > > (note, no --delete option on purpose) > > > > restore: > > > > 1. undump whichever mysqldump is needed > > 2. copy over all files that are older than the mysqldump > > > > There is nothing in BASE that guarantees that the name of a deleted file > is not reused by another file later on. However, it is very unlikely > that this should happen with the current implementation. I think you > need to upload a file, delete it, upload another within the same second > AND the system must choose the same random filename for both files. Even > if this happens it would only matter if you started a mysqldump after > the upload of the first file, but before the deletion of it. > > For all practical reasons, I think you can assume that files will always > have unique pathnames, and if rsync is just adding new files to the > backup directory you should be able to restore the file system to any > point in time. > > > The one problem I see is that the restore can bring back files that had > > been > > deleted in BASE. I doubt these would interfere with the running of BASE > > however. A small program could be written to delete any files that are not > > known about by the db (unless there is already something to do this). > > No there is no such program. > > > Alternatively, one can add the --delete option to rsync and live with only > > being able to restore the latest dump (which is OK for us for now). > > /Nicklas > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > The BASE general discussion mailing list > basedb-users@lists.sourceforge.net > unsubscribe: send a mail with subject "unsubscribe" to > [EMAIL PROTECTED] -- Bob MacCallum | VectorBase Developer | Kafatos/Christophides Groups | Division of Cell and Molecular Biology | Imperial College London | Phone +442075941945 | Email [EMAIL PROTECTED] ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace _______________________________________________ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject "unsubscribe" to [EMAIL PROTECTED]