On Sep 4, 2009, at 12:23 PM, Len Zaifman wrote:
We have groups generating terabytes a day of image data from lab
instruments and saving them to an X4500.
Wouldn't it be easier to compress at the application, or between the
application and the archiving file system?
We have tried lzbj : compressratio = 1.13 in 11 hours , 1.3 TB ->
1.1 TB
gzip -9 : compress ratio = 1.68 in > 37 hours,
1.3 TB -> .75 TB
The filesystem performance was noticably laggy (ie ls took > 10
seconds) while gzip -9 compression was used
do you have any idea if lossless jpeg compression is being planned
for ZFS? We can envisage of 1.3 TB, > .8 TB will be images and if we
could get better or equivalent compression on jpeg lossless
compression with less impact on the filesystem than gzip -9
compression, that would be worthwhile, if it worked.
I don't know of anyone working on that specific compression scheme,
but I've put together some thoughts on the subject of adding a new
compressor to ZFS. Perhaps others could comment?
http://richardelling.blogspot.com/2009/08/justifying-new-compression-algorithms.html
-- richard
_______________________________________________
zfs-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
