Bug#409664: bsdiff: is extremely slow on some files, sometimes hangs
On Sun, Feb 04, 2007 at 05:30:18PM +0100, A Mennucc wrote:
> Package: bsdiff
> Version: 4.3-3
> Severity: normal
>
> hi
>
> I manage the debdelta project
> http://tonelli.sns.it/pub/mennucc1/debdelta/
>
> debdelta needs a 'delta compressor' as a backend;
> and bsdiff would be one of the best choices, as shown
> by my tests in
> http://tonelli.sns.it/pub/mennucc1/debdelta/tests
> but for a strange problem ... actually, two problems
>
> ---Problem number one:
> sometimes bsdiff hangs forever (or, for more than
> I am willing to wait). I am capable of extracting from debdelta the
> exact "old" and "new" files that hang it; but unfortunately most of
> those are not easily reproducible ... it is as if bsdiff is
> triggering a bug in my CPU or in my memory.
>
> You find all my collected examples of "old" and "new" files in
> http://tonelli.sns.it/pub/mennucc1/debdelta/slowest_bsdiff/
>
> BTW: I run debdeltas in a amd64 Etch install; the CPU is a Athlon64
> 3000 . That host is quite stable, and never shows problems with any
> other service.
>
> ---Problem number two:
> on some short files, where my experience says that
> 'bsdiff' should compute a diff in a matter of minutes, then 'bsdiff'
> would slow down a lot.
>
> The strangest one is gcl_old gcl_new : once (~6 month ago) I ran
> $ bsdiff gcl_old gcl_new /tmp/diff
> and it took 8 hours to complete this is 10 times slower than
> average. (I am currently running that command, it has gone on
> for 5 hours).
>
> I currently prepend 'stopafter 600 TERM' to my 'bsdiff' invocation
> ('stopafter' is a nice utility that was in Debian once; that
> cmd stops bsdiff after 600 seconds, using SIGTERM).
> This "fixes" my problem, but is not really a fix.
We should apply the following 4 patches from chromium - they are small,
and, as part of Chromium, also presumably well tested:
(1)
https://chromium.googlesource.com/chromiumos/third_party/bsdiff/+/e2b85ce4ea246ca804ee2a9e18005c44e193d868
Replacing the custom suffix sorting implementation with -ldivsufsort. This
brings down the time to generate a delta (for the proposed new format) for
libreoffice-core 5.4~rc2-1->5.4-1 from 93 seconds down to 50 seconds.
It also just involves replacing code with a library call, so that's nice
as well.
(2)
https://chromium.googlesource.com/chromiumos/third_party/bsdiff/+/a055996c743add7a9558839276fd1e4994d16bd3
Speeds up a pathological case
(3)
https://chromium.googlesource.com/chromiumos/third_party/bsdiff/+/58146f74abd6b1b69693943195f37f4ac6a6acef
Fixes a hang
(4)
https://chromium.googlesource.com/chromiumos/third_party/bsdiff/+/426e4aa1cbeb3c8a73002047d7a796ca8e5e17d4
Another pathological case where files differ by less than 8 bytes
--
Debian Developer - deb.li/jak | jak-linux.org - free software dev
| Ubuntu Core Developer |
When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to ('inline'). Thank you.
Bug#409664: bsdiff: is extremely slow on some files, sometimes hangs
Hi,
Jari Aalto wrote:
Could you take a look at Debian bug
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=409664
and see if any of the suggested algorithm solutions could be applied.
It should be possible to fix this problem via algorithmic changes, yes.
Unfortunately I haven't had the {time, money} to make the necessary
adjustments.
--
Colin Percival
Security Officer, FreeBSD | freebsd.org | The power to serve
Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#409664: bsdiff: is extremely slow on some files, sometimes hangs
forwarded 409664 Colin Percival cperc...@daemonology.net thanks Hi Colin, Could you take a look at Debian bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=409664 and see if any of the suggested algorithm solutions could be applied. Thanks, Jari -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#409664: bsdiff: is extremely slow on some files, sometimes hangs
Package: bsdiff
Version: 4.3-3
Severity: normal
hi
I manage the debdelta project
http://tonelli.sns.it/pub/mennucc1/debdelta/
debdelta needs a 'delta compressor' as a backend;
and bsdiff would be one of the best choices, as shown
by my tests in
http://tonelli.sns.it/pub/mennucc1/debdelta/tests
but for a strange problem ... actually, two problems
---Problem number one:
sometimes bsdiff hangs forever (or, for more than
I am willing to wait). I am capable of extracting from debdelta the
exact old and new files that hang it; but unfortunately most of
those are not easily reproducible ... it is as if bsdiff is
triggering a bug in my CPU or in my memory.
You find all my collected examples of old and new files in
http://tonelli.sns.it/pub/mennucc1/debdelta/slowest_bsdiff/
BTW: I run debdeltas in a amd64 Etch install; the CPU is a Athlon64
3000 . That host is quite stable, and never shows problems with any
other service.
---Problem number two:
on some short files, where my experience says that
'bsdiff' should compute a diff in a matter of minutes, then 'bsdiff'
would slow down a lot.
The strangest one is gcl_old gcl_new : once (~6 month ago) I ran
$ bsdiff gcl_old gcl_new /tmp/diff
and it took 8 hours to complete this is 10 times slower than
average. (I am currently running that command, it has gone on
for 5 hours).
I currently prepend 'stopafter 600 TERM' to my 'bsdiff' invocation
('stopafter' is a nice utility that was in Debian once; that
cmd stops bsdiff after 600 seconds, using SIGTERM).
This fixes my problem, but is not really a fix.
thanks
a.
--
Andrea Mennucc
E' un mondo difficile. Che vita intensa! (Tonino Carotone)
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
