On Fri, March 26, 2010 09:46, David Dyer-Bennet wrote:
> I don't know that it makes sense to. There are lots of existing filter
> packages that do compression; so if you want compression, just put them in
> your pipeline. That way you're not limited by what zfs send has
> implemented, either. When they implement bzip98 with a new compression
> technology breakthrough, you can just use it :-) .
Actually a better example may be using parallel implementations of popular
algorithms:
http://www.zlib.net/pigz/
http://www.google.com/search?q=parallel+bzip
Given the amount of cores we have nowadays (especially the Niagara-based
CPUs), might as well use them. There are also better algorithms out there
(some of which assume parallelism):
http://en.wikipedia.org/wiki/Xz
http://en.wikipedia.org/wiki/7z
If you're using OpenSSH, there are also some third-party patches that may
help in performance:
http://www.psc.edu/networking/projects/hpn-ssh/
However, if the data is already compressed (and/or deduped), there's no
sense in doing it again. If ZFS does have to go to disk, might as well
send the data as-is.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
