zfs send receive should work perfectly for this.
If the destination has changes you don't mind loosing e.g. just permission
changes you can have the receive roll back to the last known source snapshot
before replaying changes. The result being two identical copies of everything.
If there are other changes you also want you could maybe write a script
combining zfs diff with rsync to only send the known changed files.
Regards
Steve
----- Original Message -----
From: "Tim Gustafson" <t...@soe.ucsc.edu>
To: <rsync@lists.samba.org>
Sent: Friday, November 30, 2012 5:33 PM
Subject: Speeding Up Rsync for Large File Sets
We have a particular file system that we're trying to keep in sync
between two FreeBSD/ZFS servers using Rsync.
The file system has many millions of files, and about 4TB of data
total. Rsync takes HOURS to run, even when there are no files to
transfer. Just the comparison itself takes hours.
Is there any way to speed up the transfer? The command line I'm using is:
/usr/local/bin/rsync --stats --rsh=/usr/bin/rsh --recursive --delete
--one-file-system --xattrs --links --hard-links --times --perms
--owner --group --exclude=.zfs /foo/bar root@remote:/foo/bar
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please
telephone +44 845 868 1337
or return the E.mail to postmas...@multiplay.co.uk.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
