On Mon, Dec 01, 2003 at 07:33:43PM +0100, Andrzej Filip wrote: > Is there any chance for rdiff ? > > I need to frequently synchronize big text file (60MB+) undertaking small > changes and I am interested in differences between the subsequent versions > [DNS RBL data in dnsbl format, 1E6+ lines of text, new version every 20m, > on average 50 new entries (lines) in every synchronization] > > I would like to get (small) diff file as result of rsync session and apply > it to the file myself.
rdiff sounds like a good solution for what you want. However, there might be a better solution if your data has certain patterns to it... for example if the changes are just lines appended, then a simple 'tail -f' type approach might achieve what you want. For text data, things like 'diff oldfile newfile | gzip -9' can be surprisingly effective, and will have a much lower CPU overhead than rdiff. However, one of the cool things about rdiff is it calculates delta's from a signature, so you don't have to keep a whole copy of 'oldfile', just a signature of it. For example; $ rdiff signature bigfile bigfile.sig [wait 20 mins while bigfile gets modified] $ rdiff delta bigfile.sig bigfile bigfile.delta Note however, that rdiff will not be happy (and neither will just about anything except perhaps 'tail -f') if the file is changing underneath it while it operates on them. If these are live files, you will need to either suspend whatever process modifies them while rdiff is working, or use a filsystem 'snapshot' (see LVM). For synchronising live files, you are probably better off looking at some sort of block-device level syncronisation tool (kind of like RAID mirroring with a remote network block device). -- ---------------------------------------------------------------- Donovan Baarda http://minkirri.apana.org.au/~abo/ ---------------------------------------------------------------- -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
