Tip O'The Day : pigz and pbzip2

2010-02-08 Thread John Carter

Multicores are becoming more and more common. Compression is still
something I need to do regularly.

So some new tools that combine multi-core speed up with compression.

pigz   is a drop in replacement for gzip
pbzip2 is a drop in replacement for bzip2



John Carter Phone : (64)(3) 358 6639
Tait ElectronicsFax   : (64)(3) 359 4632
PO Box 1645 ChristchurchEmail : john.car...@tait.co.nz
New Zealand



Re: Tip O'The Day : pigz and pbzip2

2010-02-08 Thread Jim Cheetham
On Tue, Feb 9, 2010 at 11:10 AM, John Carter john.car...@tait.co.nz wrote:
 pigz   is a drop in replacement for gzip
 pbzip2 is a drop in replacement for bzip2

Nice, thanks!


Re: Tip O'The Day : pigz and pbzip2

2010-02-08 Thread Craig Falconer
Yep - I've not used pigz, but be advised that pbzip2 does not produce 
the same file as bzip2 with the same options.


Functionally they're the same and can be treated as such, but a parallel 
bzipped file is a few bytes larger and has multiple  index/lookup 
tables of one per core/thread.  (probably have the name wrong there but 
you get the idea)


So, md5 will be different.

John Carter wrote, On 09/02/10 11:11:

Multicores are becoming more and more common. Compression is still
something I need to do regularly.

So some new tools that combine multi-core speed up with compression.

pigz   is a drop in replacement for gzip
pbzip2 is a drop in replacement for bzip2



John Carter Phone : (64)(3) 358 6639
Tait ElectronicsFax   : (64)(3) 359 4632
PO Box 1645 ChristchurchEmail : john.car...@tait.co.nz
New Zealand





--
Craig Falconer