On 09/14/03 Marius Mauch wrote: > Now the fact that fixpackages is so slow is because to update this > information the .tbz2 needs to be uncompressed, then the information > can be updated and then it needs to be recompressed. This takes some > time for 1 or 2 GB of data. I'm currently trying to optimize the code > a bit, but I think for a dramatic improvement we would have to drop > the current.tbz2 format (so we can save the uncompress-recompress > steps) itself which is unlikely to happen anytime soon.
Ok, answering to myself: looks as if I was wrong about the speed bottleneck in fixpackages. The uncompress/recompress is not the thing that takes most of the time, it seems to be the fact that the actual changing is done in a shellscript and the launch of about 2 dozen processes for each .tbz2/update-entry combination kills the performance. I ported this shellscript to python and integrated it in portage.py (the portage corefile) and it increased the performance about factor 8 (little test with 4 .tbz2 packages). Still have to check if that change has any side-effects, if not I think it will be integrated in one of the next portage versions. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better.
pgp00000.pgp
Description: PGP signature
