On Jan 24, 2012, at 7:52 AM, Jim Klimov wrote: > 2012-01-24 13:05, Mickaël CANÉVET wrote: >> Hi, >> >> Unless I misunderstood something, zfs send of a volume that has >> compression activated, uncompress it. So if I do a zfs send|zfs receive >> from a compressed volume to a compressed volume, my data are >> uncompressed and compressed again. Right ?
correct >> >> Is there a more effective way to do it (without decompression and >> recompression) ? > > > While I can not confirm or deny this statement, it was my > impression as well. Rationale being that the two systems > might demand different compression (i.e. "lzjb" or "none" > on the original system and "gzip-9" on the backup one). > Just like you probably have different VDEV layouts, etc. > Or perhaps even different encryption or dedup settings. that "feature" falls out of the implementation. > > Compression, like many other components, lives on the > layer "under" logical storage (userdata blocks), and > gets applied to newly written blocks only (i.e. your > datasets can have a mix of different compression levels > for different files or even blocks within a file, if > you switched the methods during dataset lifetime). > > Actually I would not be surprised if zfs-send userdata > stream is even above the block level (i.e. it would seem > normal to me if many small userdata blocks of original > pool might become one big block on the recipient). > > So while some optimizations are possible, I think they > would violate layering quite much. data in the ARC is uncompressed. compression/decompression occurs in the ZIO pipeline layer below the DSL. > > But, for example, it might make sense for zfs-send to > include the original compression algorithm information > into the sent stream and send the compressed data (less > network traffic or intermediate storage requirement, > to say the least - at zero price of recompression to > something perhaps more efficient), and if the recipient > dataset's algorithm differs - unpack and recompress it > on the receiving side. > > If that's not done already :) the compression parameter value is sent, but as you mentioned above, blocks in a snapshot can be compressed with different algorithms, so you only actually get the last setting at time of snapshot. > > So far my over-the-net zfs sends are piped into gzip > or pigz, ssh and gunzip, and that often speeds up the > overall transfer. Probably can be done with less overhead > by "ssh -C" for implementations that have it. the UNIX philosophy is in play here :-) Sending the data uncompressed to stdout allows you to pipe it into various transport or transform programs. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
