clone 868685 -1 reassign -1 apt 1.5~alpha1 retitle -1 apt: fails to update if mirror does not publish pdiff files thanks
On Mon, Jul 17, 2017 at 07:00:13PM +0200, Thorsten Glaser wrote: > Package: debmirror > Version: 1:2.27 > Severity: normal > > I can’t figure out how to reliably mirror sid for users. > I’m attaching the config file, I tried diff_mode none and > use but get errors on the clients running apt-get update > either way: It is possible that we have two separate problems here. I'm seeing the second of your problems with my local mirror at the moment, but not the first. > E: Failed to fetch > http://mirror.lan.tarent.de/debian/dists/sid/main/binary-amd64/Packages.gz > Hash Sum mismatch > Hashes of expected file: > - Filesize:10129139 [weak] > - SHA256:d81d8d4ecff84b52fb6f1bc6e58c79570b95c4b8768c19aacd45a507d6699b76 > - MD5Sum:6ce60c48450df70864589829b407ef5a [weak] > Hashes of received file: > - SHA256:efe6a06883ca7e54d663032a404a6b32eb668024dd6ddeb0ae5e94990f2ba1c9 > - MD5Sum:386edf0f6e19d81905e0b0a93a62b0e6 [weak] > - Filesize:10129113 [weak] > Last modification reported: Mon, 17 Jul 2017 09:00:36 +0000 > Release file created at: Mon, 17 Jul 2017 08:29:18 +0000 > E: Failed to fetch > http://mirror.lan.tarent.de/debian/dists/sid/main/binary-i386/Packages.gz > E: Some index files failed to download. They have been ignored, or old ones > used instead. > > (usually for Translation_en though) and I think the likely cause of this is that debmirror doesn't use by-hash to fetch from its upstream mirror, so it's possible to end up with desynchronisation depending on when it fetches its index files. Have you tried looking at the times in your upstream mirror's project/trace and adjusting the times when you run debmirror so that you avoid running while it's updating? (This is obviously poor, and I definitely want to implement by-hash fetching soon for several reasons.) > E: Failed to fetch > http://mirror.lan.tarent.de/debian/dists/sid/main/binary-amd64/PackagesIndex > 404 Not Found [IP: 172.26.0.180 80]: rename failed, No such file or > directory > (/var/lib/apt/lists/partial/mirror.lan.tarent.de_debian_dists_sid_main_binary-amd64_Packages.diff_Index > -> > /var/lib/apt/lists/mirror.lan.tarent.de_debian_dists_sid_main_binary-amd64_Packages.diff_Index). > E: Failed to fetch > http://mirror.lan.tarent.de/debian/dists/sid/main/i18n/Translation-enIndex > 404 Not Found [IP: 172.26.0.180 80]: rename failed, No such file or > directory > (/var/lib/apt/lists/partial/mirror.lan.tarent.de_debian_dists_sid_main_i18n_Translation-en.diff_Index > -> > /var/lib/apt/lists/mirror.lan.tarent.de_debian_dists_sid_main_i18n_Translation-en.diff_Index). > E: Some index files failed to download. They have been ignored, or old ones > used instead. "diff_mode mirror" might help here (although I haven't tested that myself). However, I think this is a regression in apt, so I'm cloning this bug there. With apt 1.4.7, an unstable chroot about half a day old, and a local mirror that does not publish pdiff files: # apt update Get:1 http://mirror.pelham.vpn.ucam.org/debian sid InRelease [229 kB] Get:2 http://httpredir.debian.org/debian sid InRelease [229 kB] Ign:3 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources.diff/Index Ign:4 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages.diff/Index Ign:5 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en.diff/Index Ign:3 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources.diff/Index Ign:4 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages.diff/Index Ign:5 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en.diff/Index Ign:6 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources Ign:7 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages Ign:8 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en Get:6 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources [7536 kB] Get:7 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages [7570 kB] Get:9 http://httpredir.debian.org/debian sid/main Sources.diff/Index [27.9 kB] Get:10 http://httpredir.debian.org/debian sid/main amd64 Packages.diff/Index [27.9 kB] Get:11 http://httpredir.debian.org/debian sid/main Translation-en.diff/Index [27.9 kB] Get:12 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-0219.31.pdiff [4919 B] Get:13 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-1000.44.pdiff [3640 B] Get:14 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-0219.31.pdiff [29.2 kB] Get:15 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-1000.44.pdiff [8090 B] Get:16 http://httpredir.debian.org/debian sid/main Translation-en 2017-07-23-0219.31.pdiff [1172 B] Get:15 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-1000.44.pdiff [8090 B] Get:13 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-1000.44.pdiff [3640 B] Get:16 http://httpredir.debian.org/debian sid/main Translation-en 2017-07-23-0219.31.pdiff [1172 B] Get:8 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en [5750 kB] Fetched 21.4 MB in 5s (3742 kB/s) Reading package lists... Done Building dependency tree Reading state information... Done 18 packages can be upgraded. Run 'apt list --upgradable' to see them. But with apt 1.5~alpha1 and newer: # apt update Get:1 http://mirror.pelham.vpn.ucam.org/debian sid InRelease [229 kB] Ign:2 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources.diff/Index Ign:3 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages.diff/Index Ign:4 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en.diff/Index Ign:2 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources.diff/Index Ign:3 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages.diff/Index Ign:4 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en.diff/Index Ign:5 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources Ign:6 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages Ign:7 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en Get:5 http://mirror.pelham.vpn.ucam.org/debian sid/main Sources [7536 kB] Get:6 http://mirror.pelham.vpn.ucam.org/debian sid/main amd64 Packages [7570 kB] Get:8 http://httpredir.debian.org/debian sid InRelease [229 kB] Get:7 http://mirror.pelham.vpn.ucam.org/debian sid/main Translation-en [5750 kB] Get:9 http://httpredir.debian.org/debian sid/main Sources.diff/Index [27.9 kB] Get:10 http://httpredir.debian.org/debian sid/main amd64 Packages.diff/Index [27.9 kB] Get:11 http://httpredir.debian.org/debian sid/main Translation-en.diff/Index [27.9 kB] Get:12 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-0219.31.pdiff [4919 B] Get:13 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-1000.44.pdiff [3640 B] Get:14 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-0219.31.pdiff [29.2 kB] Get:15 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-1000.44.pdiff [8090 B] Get:16 http://httpredir.debian.org/debian sid/main Translation-en 2017-07-23-0219.31.pdiff [1172 B] Get:15 http://httpredir.debian.org/debian sid/main amd64 Packages 2017-07-23-1000.44.pdiff [8090 B] Get:13 http://httpredir.debian.org/debian sid/main Sources 2017-07-23-1000.44.pdiff [3640 B] Get:16 http://httpredir.debian.org/debian sid/main Translation-en 2017-07-23-0219.31.pdiff [1172 B] Fetched 21.4 MB in 5s (4132 kB/s) Reading package lists... Done E: Failed to fetch http://mirror.pelham.vpn.ucam.org/debian/dists/sid/main/source/SourcesIndex 404 Not Found [IP: 2001:8b0:bff2:eb14:6a05:caff:fe12:71bf 80]: rename failed, No such file or directory (/var/lib/apt/lists/partial/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_source_Sources.diff_Index -> /var/lib/apt/lists/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_source_Sources.diff_Index). E: Failed to fetch http://mirror.pelham.vpn.ucam.org/debian/dists/sid/main/binary-amd64/PackagesIndex 404 Not Found [IP: 2001:8b0:bff2:eb14:6a05:caff:fe12:71bf 80]: rename failed, No such file or directory (/var/lib/apt/lists/partial/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_binary-amd64_Packages.diff_Index -> /var/lib/apt/lists/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_binary-amd64_Packages.diff_Index). E: Failed to fetch http://mirror.pelham.vpn.ucam.org/debian/dists/sid/main/i18n/Translation-enIndex 404 Not Found [IP: 2001:8b0:bff2:eb14:6a05:caff:fe12:71bf 80]: rename failed, No such file or directory (/var/lib/apt/lists/partial/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_i18n_Translation-en.diff_Index -> /var/lib/apt/lists/mirror.pelham.vpn.ucam.org_debian_dists_sid_main_i18n_Translation-en.diff_Index). E: Some index files failed to download. They have been ignored, or old ones used instead. This is effectively a new requirement for mirrors, and I think it's an unwarranted one. Thanks, -- Colin Watson [cjwat...@debian.org]
