On Tue, 14 Apr 2009, Ciaran McCreesh wrote:
> On Tue, 14 Apr 2009 12:31:23 -0400 (EDT)
> "Benjamin R. Haskell" <[email protected]> wrote:
> > What is paludis actually doing during the new "Checking for possible
> > errors......" step?
>
> Checking digests of things that have already been downloaded.
>
> > It adds about a twelve-minute delay to paludis -i world on my netbook.
>
> That doesn't sound right. Just how much stuff are you reinstalling?
I remember seeing a wishlist item at some point for an option to get a
list of URLs that would be fetched... I assume that's still a wishlist
item?
Either way, I cobbled the following fun commandline:
(paludis-fetch is just a simple script to convert 'paludis -pi world'
output into lines of the form:)
cat/pack-version::repo
paludis-fetch --dry < ~/tmp/paludis-world \
| perl -lpwe '$_="=$_"' \
| xargs paludis -qM \
| grep URI: \
| perl -lanwe 'print +(split m{/})[-1] for grep m{^[a-z]+:}, @F' \
| perl -lnwe 'BEGIN { @distdirs = map +(split)[-1], qx/paludis --info|grep
distdir|sort|uniq/; } for my $d (@distdirs) { $f = "$d/$_"; next unless -f $f;
print $f; last; }' \
| sort | uniq \
| tee ~/tmp/paludis-fetch-files \
| sort-with-size | running-total | humanfy | nl | tail
328 622.3M 23.2M /usr/portage/distfiles/mono-2.2.tar.bz2
329 650.3M 28.0M /usr/portage/distfiles/wxPython-src-2.8.9.2.tar.bz2
330 679.7M 29.4M /usr/portage/distfiles/ghc-bin-6.8.2-x86.tbz2
331 712.3M 32.6M /usr/portage/distfiles/xulrunner-1.9.0.8.tar.bz2
332 745.1M 32.8M /usr/portage/distfiles/mysql-5.0.76.tar.gz
333 782.9M 37.8M /usr/portage/distfiles/emacs-22.3.tar.gz
334 839.5M 56.6M /usr/portage/distfiles/gcc-4.3.3.tar.bz2
335 916.0M 76.4M /usr/portage/distfiles/jdk-6u13-dlj-linux-i586.bin
336 1.0G 108.1M
/usr/portage/distfiles/qt-x11-opensource-src-4.5.0.tar.bz2
337 1.1G 148.6M
/usr/portage/distfiles/OOo_3.0.0_LinuxIntel_install_en-US.tar.gz
So, at least ~1.1 GiB of stuff to be hashed, not counting things in
FILESDIR's.
$ time perl -l0pe '' ~/tmp/paludis-fetch-files | xargs -0 openssl dgst -rmd160
...
xargs -0 openssl dgst -rmd160 37.33s user 4.75s system 46% cpu 1:31.09 total
$ time perl -l0pe '' ~/tmp/paludis-fetch-files | xargs -0 openssl dgst -sha1
...
xargs -0 openssl dgst -sha1 23.42s user 4.34s system 39% cpu 1:10.28 total
So, RMD160's take about 1m30s. SHA1's take about 1:10. But, I'd expect
computing both in a row on the same files would have a lower total time,
thanks to disk cache. Are there other hashes that get computed?
> > But why? And can I disable it?
>
> The checks have always been done, and are necessary. We've just moved
> them to take place at pretend-install time, rather than after you've
> left your computer.
Ah. I saw the "Catch fetch errors at pretend time" log, but didn't
associate it. Sounds good.
Best,
Ben
_______________________________________________
paludis-user mailing list
[email protected]
http://lists.pioto.org/mailman/listinfo/paludis-user
