Bug#376817: confirmation
On Tue, Aug 08, 2006 at 12:51:42AM +1200, Mark Robinson wrote: I have several times seen it flag MD5SUM errors where the expected MD5SUM and that computed are identical. I would be interested to see verbatim copies of such errors. Surely these errors should *never* be seen and must raise questions about the security of the entire distribution. Not exactly; the situation is such that apt needs a consistent view of the archive in order to authenticate it (several different files must match), and intermediate caches don't always provide this consistency. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#376817: confirmation
Matt Zimmerman wrote: On Tue, Aug 08, 2006 at 12:51:42AM +1200, Mark Robinson wrote: I have several times seen it flag MD5SUM errors where the expected MD5SUM and that computed are identical. I would be interested to see verbatim copies of such errors. piwakawaka:~# ./apt-update Get:1 http://http.us.debian.org sid Release.gpg [189B] 99% [Working]Metaindex acquired, queueing gpg verification (/var/lib/apt/lists/partial/http.us.debian.org_debian_dists_sid_Release.gpg,/var/lib/apt/lists/http.us.debian.org_debian_dists_sid_Release) Hit http://http.us.debian.org sid Release 99% [Release gpgv 38339]inside VerifyGetSigners gpgv path: /usr/bin/gpgv Keyring path: /etc/apt/trusted.gpg Preparing to exec: /usr/bin/gpgv /usr/bin/gpgv --status-fd 3 --keyring /etc/apt/trusted.gpg /var/lib/apt/lists/partial/http.us.debian.org_debian_dists_sid_Release.gpg /var/lib/apt/lists/http.us.debian.org_debian_dists_sid_Release Read: [GNUPG:] SIG_ID SqqC3jK1DH3yiq6RS+TXK96O9+4 2006-07-24 1153773886 Read: [GNUPG:] GOODSIG 010908312D230C5F Debian Archive Automatic Signing Key (2006) [EMAIL PROTECTED] Read: [GNUPG:] VALIDSIG 084750FC01A6D388A643D869010908312D230C5F 2006-07-24 1153773886 0 3 0 17 2 00 084750FC01A6D388A643D869010908312D230C5F Got VALIDSIG, key ID:VALIDSIG 084750FC01A6D388A643D869010908312D230C5F gpgv exited Got Codename: sid Expecting Dist: sid Transformed Dist: sid Signature verification succeeded: /var/lib/apt/lists/http.us.debian.org_debian_dists_sid_Release Queueing: http://http.us.debian.org/debian/dists/sid/main/binary-i386/Packages Expected MD5: be5387a84ce5de8d2f177c865f8d6784 Queueing: http://http.us.debian.org/debian/dists/sid/contrib/binary-i386/Packages Expected MD5: 79d0311df375b8fe66ba0a97c5b1b069 Queueing: http://http.us.debian.org/debian/dists/sid/non-free/binary-i386/Packages Expected MD5: bb4424d42749fca661c99d6596d51124 Queueing: http://http.us.debian.org/debian/dists/sid/main/source/Sources Expected MD5: c5c77469275f8e6211fcaa215edab58b Queueing: http://http.us.debian.org/debian/dists/sid/contrib/source/Sources Expected MD5: ccfbacc8651bdf69f0b1c78c0c232636 Queueing: http://http.us.debian.org/debian/dists/sid/non-free/source/Sources Expected MD5: b985679429fa5af38d79072fab09520d 99% [Working]gpgv succeeded Hit http://http.us.debian.org sid/main Packages/DiffIndex Get:2 http://http.us.debian.org sid/contrib Packages [79.7kB] Hit http://http.us.debian.org sid/non-free Packages/DiffIndex 99% [2 Packages gzip 0] [Waiting for headers] http://http.us.debian.org/debian/dists/sid/contrib/binary-i386/Packages: Computed MD5: 79d0311df375b8fe66ba0a97c5b1b069 Expected MD5: 79d0311df375b8fe66ba0a97c5b1b069 Get:3 http://http.us.debian.org sid/main Sources [1651kB] Hit http://http.us.debian.org sid/contrib Sources/DiffIndex Hit http://http.us.debian.org sid/non-free Sources/DiffIndex 99% [3 Sources gzip 4915200] 30.9kB/s 0s http://http.us.debian.org/debian/dists/sid/main/source/Sources: Computed MD5: e1168cf5f79a1cc839001f8f1d0eb556 Expected MD5: c5c77469275f8e6211fcaa215edab58b Fetched 1731kB in 58s (29.5kB/s) Failed to fetch http://http.us.debian.org/debian/dists/sid/main/source/Sources.gz MD5Sum mismatch Reading package lists... Done E: Some index files failed to download, they have been ignored, or old ones used instead. piwakawaka:~# Surely these errors should *never* be seen and must raise questions about the security of the entire distribution. Not exactly; the situation is such that apt needs a consistent view of the archive in order to authenticate it (several different files must match), and intermediate caches don't always provide this consistency. When it's checking an MD5SUM it's looking at one file, when it's checking the gpg key it's looking at one file. I've noticed that the packages are uploaded to the servers after the index files. This means that you can get the index and then have to wait an hour say until the packages all turn up. Would this situation not be improved by the index files being uploaded, or made available, last ? Can we force the Get to blitz caches ? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#376817: confirmation
On Fri, Aug 11, 2006 at 10:58:20AM +1200, Mark Robinson wrote: Matt Zimmerman wrote: On Tue, Aug 08, 2006 at 12:51:42AM +1200, Mark Robinson wrote: I have several times seen it flag MD5SUM errors where the expected MD5SUM and that computed are identical. I would be interested to see verbatim copies of such errors. Computed MD5: 79d0311df375b8fe66ba0a97c5b1b069 Expected MD5: 79d0311df375b8fe66ba0a97c5b1b069 These don't match. http://http.us.debian.org/debian/dists/sid/main/source/Sources: Computed MD5: e1168cf5f79a1cc839001f8f1d0eb556 Expected MD5: c5c77469275f8e6211fcaa215edab58b Neither do these. Not exactly; the situation is such that apt needs a consistent view of the archive in order to authenticate it (several different files must match), and intermediate caches don't always provide this consistency. When it's checking an MD5SUM it's looking at one file, when it's checking the gpg key it's looking at one file. That's correct, but irrelevant. The way the system works is that the Release file (which is authenticated by a detached signature in Release.gpg) contains md5sums for the Packages files, which are retrieved separately. If the Release file is an older cached version than the Packages files, or vice versa, there will be a mismatch and authentication will fail. I've noticed that the packages are uploaded to the servers after the index files. This means that you can get the index and then have to wait an hour say until the packages all turn up. Would this situation not be improved by the index files being uploaded, or made available, last ? Official mirrors do this correctly, and mirror the indexes last, for this reason. Can we force the Get to blitz caches ? Yes, there's an apt configuration option for it, but a) there seems to still be trouble sometimes even with this option, and b) the packages files are large and we *want* them to be cached, but only if they're verified against the server so that they're guaranteed up-to-date. That's what the cache-control settings used by default should do according to the standard. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#376817: confirmation
On Fri, Aug 11, 2006 at 11:42:31AM +1200, Mark Robinson wrote: Matt Zimmerman wrote: On Fri, Aug 11, 2006 at 10:58:20AM +1200, Mark Robinson wrote: Matt Zimmerman wrote: On Tue, Aug 08, 2006 at 12:51:42AM +1200, Mark Robinson wrote: I have several times seen it flag MD5SUM errors where the expected MD5SUM and that computed are identical. I would be interested to see verbatim copies of such errors. Computed MD5: 79d0311df375b8fe66ba0a97c5b1b069 Expected MD5: 79d0311df375b8fe66ba0a97c5b1b069 These don't match. eh ? Computed MD5: 79d0311df375b8fe66ba0a97c5b1b069 Expected MD5: 79d0311df375b8fe66ba0a97c5b1b069 That one isn't a failure; in verbose mode, it prints the MD5 in every case. -- - mdz -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Bug#376817: confirmation
I can confirm this behaviour. I am seeing occasional MD5SUM errors as well as GPG failures. Strangely it seems to work considerably better when not using the debug options suggested above. I have several times seen it flag MD5SUM errors where the expected MD5SUM and that computed are identical. Surely these errors should *never* be seen and must raise questions about the security of the entire distribution. regards Mark -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
