On Mon, 4 Mar 2013, Matthew Ahrens wrote:

Magic rsync options used:

  -a --inplace --no-whole-file --delete-excluded

This causes rsync to overwrite the file blocks in place rather than writing
to a new temporary file first.  As a result, zfs COW produces primitive
"deduplication" of at least the unchanged blocks (by writing nothing) while
writing new COW blocks for the changed blocks.

If I understand your use case correctly (the application overwrites
some blocks with the same exact contents), ZFS will ignore these
"no-op" writes only on recent Open ZFS (illumos / FreeBSD / Linux)
builds with checksum=sha256 and compression!=off.  AFAIK, Solaris ZFS
will COW the blocks even if their content is identical to what's
already there, causing the snapshots to diverge.

With these rsync options, rsync will only overwrite a "block" if the contents of the block has changed. Rsync's notion of a block is different than zfs so there is not a perfect overlap.

Rsync does need to read files on the destination filesystem to see if they have changed. If the system has sufficient RAM (and/or L2ARC) then files may still be cached from the previous day's run. In most cases only a small subset of the total files are updated (at least on my systems) so the caching requirements are small. Files updated on one day are more likely to be the ones updated on subsequent days.

Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
zfs-discuss mailing list

Reply via email to