On Fri, Sep 19, 2014 at 09:00:43AM +0000, Thorsten Glaser wrote: > Package: apt > Version: 1.0.9.1 > Severity: minor
Thanks for your bugreport. > Michael Vogt requested I file this as official bugreport: > > The recent revalidation changes lead to a simple apt-get update > on my buildd machine (m68k VM with 198 MHz CPU) with only the > incoming.d-p.o repository modified, but not unstable, taking: > > Fetched 156 kB in 8min 33s (304 B/s) > > The sources.list on that chroot is just: > > deb http://ftp.de.debian.org/debian-ports unstable main > deb http://ftp.de.debian.org/debian-ports unreleased main > deb-src http://ftp.de.debian.org/debian unstable main > # plus the “incoming” repository > > With almost 200 MHz, this is one of the faster “machines”. > Our regular buildds have 25, 33, 50 MHz. [..] This is indeed a pretty big hit on this hardware :/ The following patch should fix most of this: diff --git a/apt-pkg/acquire-item.cc b/apt-pkg/acquire-item.cc index 5df4372..28ede30 100644 --- a/apt-pkg/acquire-item.cc +++ b/apt-pkg/acquire-item.cc @@ -1558,8 +1558,9 @@ void pkgAcqMetaIndex::AuthDone(string Message) /*{{{*/ } - // Download further indexes with verification - QueueIndexes(true); + // Download further indexes with verification if something has changed + if(IMSHit == false) + QueueIndexes(true); // is it a clearsigned MetaIndex file? if (DestFile == SigFile) The downside if of course that if a Packages file is missing (because of a earlier download error for example) then it won't be fetched again, so the patch is probably too simplistic. But I think something like this should be done to avoid the extra verify step if we know that nothing has changed. Cheers, Michael > The cause for this is that even a not-modified now triggers > reverification of all downloaded files. The visible effect > is a process called “copy” taking >90% CPU for minutes, in > top(1). > > Michael and I think that apt should, when the Release and > Release.gpg (or InRelease, but we aren’t using them any > more, IIRC?) files are not modified, just reverify these, > and, if they pass, assume the other data on the disc is > valid/verified (because the files only get moved out of > partial/ once verified AFAIK, and because if an unmodified > Release file passes reverification, it would have passed > initial verification as well, normally). > > Michael also thinks apt could cache more information, such > as the original hashes, to speed up things more. > > > -- Package-specific info: > > -- apt-config dump -- > > APT ""; > APT::Architecture "m68k"; > APT::Build-Essential ""; > APT::Build-Essential:: "build-essential"; > APT::Install-Recommends "0"; > APT::Install-Suggests "0"; > APT::NeverAutoRemove ""; > APT::NeverAutoRemove:: "^firmware-linux.*"; > APT::NeverAutoRemove:: "^linux-firmware$"; > APT::NeverAutoRemove:: "^linux-image-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-image-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^linux-headers-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-headers-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^linux-image-extra-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-image-extra-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^linux-signed-image-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-signed-image-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^kfreebsd-image-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^kfreebsd-image-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^kfreebsd-headers-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^kfreebsd-headers-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^gnumach-image-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^gnumach-image-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^.*-modules-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^.*-modules-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^.*-kernel-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^.*-kernel-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^linux-backports-modules-.*-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-backports-modules-.*-3\.16-1-m68k$"; > APT::NeverAutoRemove:: "^linux-tools-3\.14-2-m68k$"; > APT::NeverAutoRemove:: "^linux-tools-3\.16-1-m68k$"; > APT::VersionedKernelPackages ""; > APT::VersionedKernelPackages:: "linux-image"; > APT::VersionedKernelPackages:: "linux-headers"; > APT::VersionedKernelPackages:: "linux-image-extra"; > APT::VersionedKernelPackages:: "linux-signed-image"; > APT::VersionedKernelPackages:: "kfreebsd-image"; > APT::VersionedKernelPackages:: "kfreebsd-headers"; > APT::VersionedKernelPackages:: "gnumach-image"; > APT::VersionedKernelPackages:: ".*-modules"; > APT::VersionedKernelPackages:: ".*-kernel"; > APT::VersionedKernelPackages:: "linux-backports-modules-.*"; > APT::VersionedKernelPackages:: "linux-tools"; > APT::Never-MarkAuto-Sections ""; > APT::Never-MarkAuto-Sections:: "metapackages"; > APT::Never-MarkAuto-Sections:: "restricted/metapackages"; > APT::Never-MarkAuto-Sections:: "universe/metapackages"; > APT::Never-MarkAuto-Sections:: "multiverse/metapackages"; > APT::Never-MarkAuto-Sections:: "oldlibs"; > APT::Never-MarkAuto-Sections:: "restricted/oldlibs"; > APT::Never-MarkAuto-Sections:: "universe/oldlibs"; > APT::Never-MarkAuto-Sections:: "multiverse/oldlibs"; > APT::Get ""; > APT::Get::AllowUnauthenticated "false"; > APT::Architectures ""; > APT::Architectures:: "m68k"; > APT::Compressor ""; > APT::Compressor::. ""; > APT::Compressor::.::Name "."; > APT::Compressor::.::Extension ""; > APT::Compressor::.::Binary ""; > APT::Compressor::.::Cost "1"; > APT::Compressor::gzip ""; > APT::Compressor::gzip::Name "gzip"; > APT::Compressor::gzip::Extension ".gz"; > APT::Compressor::gzip::Binary "gzip"; > APT::Compressor::gzip::Cost "2"; > APT::Compressor::gzip::CompressArg ""; > APT::Compressor::gzip::CompressArg:: "-9n"; > APT::Compressor::gzip::UncompressArg ""; > APT::Compressor::gzip::UncompressArg:: "-d"; > APT::Compressor::bzip2 ""; > APT::Compressor::bzip2::Name "bzip2"; > APT::Compressor::bzip2::Extension ".bz2"; > APT::Compressor::bzip2::Binary "bzip2"; > APT::Compressor::bzip2::Cost "3"; > APT::Compressor::bzip2::CompressArg ""; > APT::Compressor::bzip2::CompressArg:: "-9"; > APT::Compressor::bzip2::UncompressArg ""; > APT::Compressor::bzip2::UncompressArg:: "-d"; > APT::Compressor::xz ""; > APT::Compressor::xz::Name "xz"; > APT::Compressor::xz::Extension ".xz"; > APT::Compressor::xz::Binary "xz"; > APT::Compressor::xz::Cost "4"; > APT::Compressor::xz::CompressArg ""; > APT::Compressor::xz::CompressArg:: "-6"; > APT::Compressor::xz::UncompressArg ""; > APT::Compressor::xz::UncompressArg:: "-d"; > APT::Compressor::lzma ""; > APT::Compressor::lzma::Name "lzma"; > APT::Compressor::lzma::Extension ".lzma"; > APT::Compressor::lzma::Binary "xz"; > APT::Compressor::lzma::Cost "5"; > APT::Compressor::lzma::CompressArg ""; > APT::Compressor::lzma::CompressArg:: "--format=lzma"; > APT::Compressor::lzma::CompressArg:: "-9"; > APT::Compressor::lzma::UncompressArg ""; > APT::Compressor::lzma::UncompressArg:: "--format=lzma"; > APT::Compressor::lzma::UncompressArg:: "-d"; > Dir "/"; > Dir::State "var/lib/apt/"; > Dir::State::lists "lists/"; > Dir::State::cdroms "cdroms.list"; > Dir::State::mirrors "mirrors/"; > Dir::State::extended_states "extended_states"; > Dir::State::status "/var/lib/dpkg/status"; > Dir::Cache "var/cache/apt/"; > Dir::Cache::archives "/var/cache/pbuilder/aptcache"; > Dir::Cache::srcpkgcache "srcpkgcache.bin"; > Dir::Cache::pkgcache "pkgcache.bin"; > Dir::Etc "etc/apt/"; > Dir::Etc::sourcelist "sources.list"; > Dir::Etc::sourceparts "sources.list.d"; > Dir::Etc::vendorlist "vendors.list"; > Dir::Etc::vendorparts "vendors.list.d"; > Dir::Etc::main "apt.conf"; > Dir::Etc::netrc "auth.conf"; > Dir::Etc::parts "apt.conf.d"; > Dir::Etc::preferences "preferences"; > Dir::Etc::preferencesparts "preferences.d"; > Dir::Etc::trusted "trusted.gpg"; > Dir::Etc::trustedparts "trusted.gpg.d"; > Dir::Bin ""; > Dir::Bin::methods "/usr/lib/apt/methods"; > Dir::Bin::solvers ""; > Dir::Bin::solvers:: "/usr/lib/apt/solvers"; > Dir::Bin::dpkg "/usr/bin/dpkg"; > Dir::Bin::bzip2 "/bin/bzip2"; > Dir::Bin::xz "/usr/bin/xz"; > Dir::Bin::lzma "/usr/bin/xz"; > Dir::Media ""; > Dir::Media::MountPath "/media/apt"; > Dir::Log "var/log/apt"; > Dir::Log::Terminal "term.log"; > Dir::Log::History "history.log"; > Dir::Ignore-Files-Silently ""; > Dir::Ignore-Files-Silently:: "~$"; > Dir::Ignore-Files-Silently:: "\.disabled$"; > Dir::Ignore-Files-Silently:: "\.bak$"; > Dir::Ignore-Files-Silently:: "\.dpkg-[a-z]+$"; > Dir::Ignore-Files-Silently:: "\.save$"; > Dir::Ignore-Files-Silently:: "\.orig$"; > Dir::Ignore-Files-Silently:: "\.distUpgrade$"; > Acquire ""; > Acquire::cdrom ""; > Acquire::cdrom::mount "/media/cdrom/"; > Acquire::PDiffs "false"; > Acquire::CompressionTypes ""; > Acquire::CompressionTypes::Order ""; > Acquire::CompressionTypes::Order:: "gz"; > Acquire::http ""; > Acquire::http::Pipeline-Depth "0"; > Acquire::Languages ""; > Acquire::Languages:: "en"; > Acquire::Languages:: "none"; > DPkg ""; > DPkg::Pre-Install-Pkgs ""; > DPkg::Pre-Install-Pkgs:: "/usr/sbin/dpkg-preconfigure --apt || true"; > debug ""; > debug::pkgproblemresolver "true"; > CommandLine ""; > CommandLine::AsString "apt-config dump"; > > -- /etc/apt/preferences -- > > Package: glib-networking-common > Pin: version 2.36.1-2~m68k.1 > Pin-Priority: 1001 > > > -- /etc/apt/sources.list -- > > # Debian/m68k /etc/apt/sources.list for regular use > #- > > # Debian/m68k binaries come from debian-ports or one of its mirrors > deb http://ftp.de.debian.org/debian-ports unstable main > deb http://ftp.de.debian.org/debian-ports unreleased main > #deb http://ftp.de.debian.org/debian-ports experimental main > > # Debian arch:all binaries can ingeniously come from a normal debian > # mirror, when dpo is not updating timely > #deb [arch=all] http://ftp.de.debian.org/debian unstable main > > # Debian/m68k unstable/experimental source comes from a normal debian mirror > #T#deb-src http://ftp.de.debian.org/debian unstable main > #deb-src http://ftp.de.debian.org/debian experimental main > > # Debian/m68k unreleased source comes from debian-ports mirrors > #T#deb-src http://ftp.de.debian.org/debian-ports unreleased main > > # [ buildd stuff removed in reportbug ] > > # If really needed: Allow etch-m68k to satisfy dependencies > #deb http://archive.debian.org/debian etch-m68k main contrib non-free > #deb-src http://archive.debian.org/debian etch-m68k main contrib non-free > > ### All following repositories need the 'wtf-debian-keyring' package > ### installed (0x405422DD which is signed by 0xE99007E0 on DD keyring) > > # Debian unstable + Debian-Ports unreleased + src:wtf-debian-keyring > # Only official Debian packages (and the keyring), long-lived, rarely > # updated; contains everything needed for debootstrap; edos-debcheck > # is used to keep all packages from this repository installable > deb http://shore.naturalnet.de/~tg/DP sid main > #deb-src deb http://shore.naturalnet.de/~tg/DP sid main > > # Debian unstable + Debian-Ports unreleased (short-lived: just built > # and not on either dpo or the above one yet) > deb http://shore.naturalnet.de/~tg/DP m68k main > #deb-src http://shore.naturalnet.de/~tg/DP m68k main > > # Mostly when we use older (source or binary) packages from proper Debian > deb http://shore.naturalnet.de/~tg/DP hacks clean > #deb-src http://shore.naturalnet.de/~tg/DP hacks clean > > # Anything goes, even hex-edited .deb files and cross-built packages > #deb http://shore.naturalnet.de/~tg/DP hacks dirty > #deb-src http://shore.naturalnet.de/~tg/DP hacks dirty > > # mirabilos' private packages (not official Debian) > deb http://shore.naturalnet.de/~tg/Debs sid wtf > #deb-src http://shore.naturalnet.de/~tg/Debs sid wtf > > -- System Information: > Debian Release: jessie/sid > APT prefers unreleased > APT policy: (500, 'unreleased'), (500, 'unstable') > Architecture: m68k > > Kernel: Linux 3.16-1-m68k > Locale: LANG=C, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) > Shell: /bin/sh linked to /bin/lksh > > Versions of packages apt depends on: > ii debian-archive-keyring 2014.1 > ii gnupg 1.4.18-4 > ii libapt-pkg4.12 1.0.9.1 > ii libc6 2.19-7 > ii libgcc2 4.9.1-5 > ii libstdc++6 4.9.1-5 > > apt recommends no packages. > > Versions of packages apt suggests: > pn apt-doc <none> > pn aptitude | synaptic | wajig <none> > ii dpkg-dev 1.17.13 > pn python-apt <none> > > -- no debconf information > > > -- > To UNSUBSCRIBE, email to [email protected] > with a subject of "unsubscribe". Trouble? Contact [email protected] > Archive: > https://lists.debian.org/[email protected] > -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected]
