> From: [email protected] [mailto:[email protected]] On Behalf
> Of Edward Ned Harvey
>
> Summary:
> LZO is the fastest.
> 7-zip (p7zip) produces the best compression ratio. (Overall, it's
> the most impressive.) Doesn't seem possible to use it as an in-line
> filter.
> gzip --fast is the most widely available. Its speed is same order of
> magnitude with LZO, and its compression is same order of magnitude as
> 7-zip, but it's unquestionably beat by those others, in their natural
> habitat. So for general purpose, gzip --fast is the most likely to be
> used in general.
> bzip2 is soundly destroyed by 7-zip, if only 7-zip is made available
> as an in-line filter. Presently, in-line filtering is the only reason
> to ever use bzip2 instead of 7-zip.
A few clarifications / corrections:
7-zip uses lzma algorithm. There is another project "lzma" which uses this,
and is in-line filterable, such as gzip & bzip2 etc. I have had no trouble
today, downloading lzma on every platform that I manage in precompiled
binary form. So I can say availability or compatibility is not a problem.
I repeated the bench with lzma, and included the results. It performs
comparably to 7-zip, of course.
Using default settings, 7-zip & lzma are much slower than bzip2. However,
if you specify --fast, then lzma is both 2x faster and 2x stronger than any
level of bzip2, which IMHO obsoletes bzip2.
I noticed my LZO results were screwed up. Updated results are attached.
------------------
no compression (copy with cp) (cache is cold; this warms cache for
everything else)
207M 0m3.144s
after cache warm:
207M 0m0.658s
------------------
Note: In all these tests, I watch "top" to ensure the benchmark is "fair."
No
processes are multithreading or using multiple cores. The default for
p7zip was
to use multiple cores, but I gave it the switch to disable that.
LZO is so light, even cat /dev/zero | lzop > /dev/null cannot make lzop
consume
100% of the cpu. That's pretty amazing. I thought maybe it was because
it's all
zeros, so I also tried (while true ; do cat somefile ; done) | lzop >
/dev/null
and the result was the same. VERY light compression.
------------------
LZO (lzop) compression level 1
78M 0m1.820s
------------------
LZO (lzop) compression level 5
77M 0m1.913s
------------------
LZO (lzop) compression level 9
71M 1m21.792s
------------------
compress (ncompress, Lempel-Ziv)
81M 0m6.486s
------------------
7-zip (p7zip, 7za) compression level 1
17M 0m13.995s
------------------
7-zip (p7zip, 7za) compression level 3
17M 0m19.222s
------------------
7-zip (p7zip, 7za) compression level 5
13M 1m53.736s
------------------
7-zip (p7zip, 7za) compression level 7
9.1M 2m16.036s
------------------
7-zip (p7zip, 7za) compression level 9
8.7M 2m19.534s
------------------
lzma compression level 1
17M 0m19.956s
------------------
lzma compression level 7
16M 3m41.236s
------------------
lzma compression level 9
12M 7m3.147s
------------------
zlib (gzip) compression level 1
42M 0m5.883s
------------------
zlib (gzip) compression level 2
42M 0m5.643s
------------------
zlib (gzip) compression level 3
42M 0m6.733s
------------------
zlib (gzip) compression level 4
41M 0m7.293s
------------------
zlib (gzip) compression level 5
41M 0m9.257s
------------------
zlib (gzip) compression level 6
41M 0m14.189s
------------------
zlib (gzip) compression level 7
41M 0m15.573s
------------------
zlib (gzip) compression level 8
42M 0m21.628s
------------------
zlib (gzip) compression level 9
42M 0m28.298s
------------------
bzip2 compression level 1
29M 0m34.042s
------------------
bzip2 compression level 2
30M 0m34.244s
------------------
bzip2 compression level 3
30M 0m34.588s
------------------
bzip2 compression level 4
31M 0m34.724s
------------------
bzip2 compression level 5
31M 0m34.254s
------------------
bzip2 compression level 6
31M 0m36.000s
------------------
bzip2 compression level 7
31M 0m36.798s
------------------
bzip2 compression level 8
31M 0m39.771s
------------------
bzip2 compression level 9
32M 0m39.351s
_______________________________________________
Tech mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
