package lbzip2
tags 645999 + confirmed
thanks
Hello Mike,
thanks for the very detailed and to-the-point bugreport; it happens
exactly as you describe. I also agree with the severity (important -- "a
bug which has a major effect on the usability of a package, without
rendering it completely unusable to everyone").
As far as I can tell, this bug was the reason why clonezilla removed
lbzip2 as default (de)compressor (I think in Clonezilla live 1.2.5-15). I
believe the use cases are similar -- disk images with large sections of
zero filled area.
See also
http://sourceforge.net/projects/clonezilla/forums/forum/663168/topic/3703144
Especially my comment posted on 2010-05-12 02:31:49 UTC.
As I wrote there, lbzip2-0.23 does not have a memory throttle, and I'm
very unlikely to add one, or to work on lbzip2 anymore for that matter.
(Note however that lbzip2 as a project is not dead at all!, it's only my
person that's getting distanced from it.)
When I first heard about this problem, I tried to retro-design a memory
throttle for lbzip2-0.23. Under that design, workers would have had to be
"ordered" with regard to each other; no worker would have been allowed to
create a "gap" in the workers-to-muxer queue, as that could have lead to a
deadlock on the memory pool. (The worker trying to produce the missing
block in order to resume the w2m queue could block on missing memory, and
the muxer could be blocking on the gap (the missing w2m block) and
therefore wouldn't flush/release buffered-up blocks.) To avoid that, w2m
block number N+1 couldn't have been enqueued until w2m block number N is
produced.
I didn't like this design (introducing a dependency between workers), and
I could not / did not bother to find (let alone implement) a better one.
I apologize if this bug makes lbzip2 unusable for you. To put users' needs
above everything else :), I'll tell you this: you might want to check out
pbzip2; I believe pbzip2 should have a working memory throttle staring
with v1.1.2.
Laszlo
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
