On Wed, 27 Jan 2010, Rogério Brito wrote:
I have been using lbzip2 and being very happy with it.
Thanks :)
I have one feature request to make it even more flexible: for
benchmarking, scripting and psychological purposes :-), would it be
possible to have command line option so that we could get something
similar to the output of xz?
Here is what is given in the middle of a compression of a file, for
example:
,----
| /tmp$ xz -9ev debian-testing-powerpc-businesscard.iso
| debian-testing-powerpc-businesscard.iso (1/1)
| ^C 44.1 % 14.5 MiB / 28.7 MiB = 0.507 866 KiB/s
0:33
|
| /tmp$
`----
Please consider using the great cstream or pv tools in a pipeline with
lbzip2. (The Debian packages have the same names as the binaries.) For
example:
tar -c /usr 2>tar.err \
| pv --name 'plaintext ' --cursor \
| lbzip2 \
| pv --name 'compressed' --cursor \
usr.tar.bz2
Sample diagnostics:
----v----
plaintext : 113MB 0:00:12 [4.62MB/s] [ <=> ]
compressed: 34.4MB 0:00:13 [2.42MB/s] [ <=> ]
----^----
Since lbzip2 can parallelize decompression from a pipe, another example
might be:
pv /usr/src/linux-source-2.6.26.tar.bz2 \
| tar --use=lbzip2 -t >tar.out 2>tar.err
Sample diagnostics:
----v----
16.6MB 0:00:02 [4.94MB/s] [========> ] 35% ETA 0:00:03
----^----
If you find this answer unsatisfactory, please tell me. In that case, I'll
try to think about how to implement the feature *cleanly* within lbzip2.
(Please don't take this as a promise, though.)
Finally, I can't help but recommend a multi-threaded LZMA compressor to
you: you may want to check out plzip [0] [1] [2], originally based on the
lzlib-0.7 compression library [3] by Antonio Diaz Diaz, and the compressor
module of lbzip2-0.17 by yours truly. After the initial bare-bones,
featureless implementation by me (modestly called llzip at that time),
plzip continues to be developed / maintained by Antonio Diaz Diaz
exclusively. He just released 0.3 a few days ago [4].
Comparing, on my dual core,
- plzip-0.3 / lzlib-0.8
- xz (XZ Utils) 4.999.9beta / liblzma 4.999.9beta
$ time -p plzip --keep linux-source-2.6.26.tar
real 129.52
user 246.49
sys 1.78
$ time -p xz --keep linux-source-2.6.26.tar
real 198.50
user 197.71
sys 0.66
$ ls -lgo linux-source-2.6.26.tar*
-r-------- 1 284334080 2010-01-28 01:26:40 +0100 linux-source-2.6.26.tar
-r-------- 1 42130234 2010-01-28 01:26:40 +0100 linux-source-2.6.26.tar.lz
-r-------- 1 41430860 2010-01-28 01:26:40 +0100 linux-source-2.6.26.tar.xz
(Needless to say, this was a completely non-tuned "benchmark".)
Cheers,
lacos
[0] http://www.nongnu.org/lzip/plzip.html
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=563929
[2] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=566084
[3] http://www.nongnu.org/lzip/lzlib.html
[4] http://lists.gnu.org/archive/html/lzip-bug/2010-01/msg00021.html
