[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
This bug was fixed in the package apt - 1.0.1ubuntu2.15 --- apt (1.0.1ubuntu2.15) trusty; urgency=medium * Fixes failure to download the Package index file when using mirror:// URL in sources.list and the archive fails to profile a file. APT would try the next archive in the list for .deb packages but did not retry when the index file failed to download. (LP: #1625667) -- Louis Bouchard Tue, 20 Sep 2016 17:02:03 +0200 ** Changed in: apt (Ubuntu Trusty) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Fix Released Status in apt source package in Trusty: Fix Released Bug description: [SRU justification] This fix is needed to allow correct archive mirroring functionality. [Impact] Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. [Fix] Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. [Test Case] 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). 2) Add an apache webserver serving the reprepro archive over http on each container 3) Add the following mirrors.txt file to one of the webserver (archive1): $ cat /var/www/html/mirrors.txt http://archive1/ubuntu/ http://archive2/ubuntu/ http://archive3/ubuntu/ 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : $ cat /etc/hosts 127.0.0.1 localhost 10.0.4.182 archive1mirrors.ubuntu.com 10.0.4.127 archive2 10.0.4.193 archive3 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : $ cat /etc/apt/sources.list #deb http://archive.ubuntu.com/ubuntu trusty main #deb http://archive.ubuntu.com/ubuntu trusty-updates main #deb http://archive.ubuntu.com/ubuntu trusty universe #deb http://archive.ubuntu.com/ubuntu trusty-updates universe deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted 6) Run $ apt-get -oDebug::Acquire::mirror=true update The log should display for the Packages file : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages With the fix you will see : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz [Regression] None expected. Worse that can happen is a retry when none was done previously. [Original description of the problem] When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mir
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
** Tags removed: verification-needed ** Tags added: verification-done -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Fix Released Status in apt source package in Trusty: Fix Committed Bug description: [SRU justification] This fix is needed to allow correct archive mirroring functionality. [Impact] Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. [Fix] Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. [Test Case] 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). 2) Add an apache webserver serving the reprepro archive over http on each container 3) Add the following mirrors.txt file to one of the webserver (archive1): $ cat /var/www/html/mirrors.txt http://archive1/ubuntu/ http://archive2/ubuntu/ http://archive3/ubuntu/ 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : $ cat /etc/hosts 127.0.0.1 localhost 10.0.4.182 archive1mirrors.ubuntu.com 10.0.4.127 archive2 10.0.4.193 archive3 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : $ cat /etc/apt/sources.list #deb http://archive.ubuntu.com/ubuntu trusty main #deb http://archive.ubuntu.com/ubuntu trusty-updates main #deb http://archive.ubuntu.com/ubuntu trusty universe #deb http://archive.ubuntu.com/ubuntu trusty-updates universe deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted 6) Run $ apt-get -oDebug::Acquire::mirror=true update The log should display for the Packages file : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages With the fix you will see : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz [Regression] None expected. Worse that can happen is a retry when none was done previously. [Original description of the problem] When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to download. They have been ignored, or old ones used instead. The same configuration works correctly on Xenial To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1625667/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
Patch verified through this sequence: Add "deb http://archive.ubuntu.com/ubuntu trusty-proposed multiverse restricted main universe" to /etc/apt/sources.list apt-get update apt-get install apt=1.0.1ubuntu2.15 apt-get upgrade <- Accept all from maintainer After that sequence, the apt-get update works without failure, even when one of the mirrors is partly broken. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Fix Released Status in apt source package in Trusty: Fix Committed Bug description: [SRU justification] This fix is needed to allow correct archive mirroring functionality. [Impact] Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. [Fix] Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. [Test Case] 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). 2) Add an apache webserver serving the reprepro archive over http on each container 3) Add the following mirrors.txt file to one of the webserver (archive1): $ cat /var/www/html/mirrors.txt http://archive1/ubuntu/ http://archive2/ubuntu/ http://archive3/ubuntu/ 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : $ cat /etc/hosts 127.0.0.1 localhost 10.0.4.182 archive1mirrors.ubuntu.com 10.0.4.127 archive2 10.0.4.193 archive3 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : $ cat /etc/apt/sources.list #deb http://archive.ubuntu.com/ubuntu trusty main #deb http://archive.ubuntu.com/ubuntu trusty-updates main #deb http://archive.ubuntu.com/ubuntu trusty universe #deb http://archive.ubuntu.com/ubuntu trusty-updates universe deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted 6) Run $ apt-get -oDebug::Acquire::mirror=true update The log should display for the Packages file : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages With the fix you will see : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz [Regression] None expected. Worse that can happen is a retry when none was done previously. [Original description of the problem] When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to down
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
** No longer affects: apt (Ubuntu Xenial) ** Changed in: apt (Ubuntu) Status: Invalid => Fix Released ** Changed in: apt (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Fix Released Status in apt source package in Trusty: Fix Committed Bug description: [SRU justification] This fix is needed to allow correct archive mirroring functionality. [Impact] Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. [Fix] Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. [Test Case] 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). 2) Add an apache webserver serving the reprepro archive over http on each container 3) Add the following mirrors.txt file to one of the webserver (archive1): $ cat /var/www/html/mirrors.txt http://archive1/ubuntu/ http://archive2/ubuntu/ http://archive3/ubuntu/ 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : $ cat /etc/hosts 127.0.0.1 localhost 10.0.4.182 archive1mirrors.ubuntu.com 10.0.4.127 archive2 10.0.4.193 archive3 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : $ cat /etc/apt/sources.list #deb http://archive.ubuntu.com/ubuntu trusty main #deb http://archive.ubuntu.com/ubuntu trusty-updates main #deb http://archive.ubuntu.com/ubuntu trusty universe #deb http://archive.ubuntu.com/ubuntu trusty-updates universe deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted 6) Run $ apt-get -oDebug::Acquire::mirror=true update The log should display for the Packages file : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages With the fix you will see : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz [Regression] None expected. Worse that can happen is a retry when none was done previously. [Original description of the problem] When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to download. They have been ignored, or old ones used instead. The same configuration works correctly on Xenial To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+sour
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
Hello Louis, or anyone else affected, Accepted apt into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apt/1.0.1ubuntu2.15 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! ** Changed in: apt (Ubuntu Trusty) Status: In Progress => Fix Committed ** Tags added: verification-needed -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Invalid Status in apt source package in Trusty: Fix Committed Status in apt source package in Xenial: Invalid Bug description: [SRU justification] This fix is needed to allow correct archive mirroring functionality. [Impact] Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. [Fix] Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. [Test Case] 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). 2) Add an apache webserver serving the reprepro archive over http on each container 3) Add the following mirrors.txt file to one of the webserver (archive1): $ cat /var/www/html/mirrors.txt http://archive1/ubuntu/ http://archive2/ubuntu/ http://archive3/ubuntu/ 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : $ cat /etc/hosts 127.0.0.1 localhost 10.0.4.182 archive1mirrors.ubuntu.com 10.0.4.127 archive2 10.0.4.193 archive3 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : $ cat /etc/apt/sources.list #deb http://archive.ubuntu.com/ubuntu trusty main #deb http://archive.ubuntu.com/ubuntu trusty-updates main #deb http://archive.ubuntu.com/ubuntu trusty universe #deb http://archive.ubuntu.com/ubuntu trusty-updates universe deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted 6) Run $ apt-get -oDebug::Acquire::mirror=true update The log should display for the Packages file : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages With the fix you will see : MirrorMethod::Fetch() Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz [Regression] None expected. Worse that can happen is a retry when none was done previously. [Original description of the problem] When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restrict
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
** Description changed: - When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on - Trusty, if the download of the index file fails, apt will not retry on - another archive listed in mirrors.txt and will fail. + [SRU justification] + This fix is needed to allow correct archive mirroring functionality. + + [Impact] + Without this fix, apt-get update and other apt commands requiring Package file access may fail when there are remote transient errors due to archive availability. + + [Fix] + Use std::npos instead of 0 to test presence of "Translation" in ShortDesc. + + [Test Case] + + 1) Setup three containers (archive1, archive2, archive3) with a reprepro configuration mirroring the ubuntu restricted archive (to save on space and time to replicate). + 2) Add an apache webserver serving the reprepro archive over http on each container + 3) Add the following mirrors.txt file to one of the webserver (archive1): + $ cat /var/www/html/mirrors.txt + http://archive1/ubuntu/ + http://archive2/ubuntu/ + http://archive3/ubuntu/ + 4) Add a client container. Add the IP adresses of the archive[1-3] containers to /etc/hosts. Alias the archive1 address to archive.ubuntu.com similar to the following : + $ cat /etc/hosts + 127.0.0.1 localhost + 10.0.4.182 archive1mirrors.ubuntu.com + 10.0.4.127 archive2 + 10.0.4.193 archive3 + 5) Add the following line to the /etc/apt/sources.list, commenting all other entries : + $ cat /etc/apt/sources.list + #deb http://archive.ubuntu.com/ubuntu trusty main + #deb http://archive.ubuntu.com/ubuntu trusty-updates main + #deb http://archive.ubuntu.com/ubuntu trusty universe + #deb http://archive.ubuntu.com/ubuntu trusty-updates universe + + deb mirror://mirrors.ubuntu.com/mirrors.txt trusty restricted + 6) Run + $ apt-get -oDebug::Acquire::mirror=true update + + The log should display for the Packages file : + + MirrorMethod::Fetch() + Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages + Err http://10.0.4.193/ubuntu/ trusty/restricted amd64 Packages + + With the fix you will see : + + MirrorMethod::Fetch() + Failure to get http://10.0.4.193/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz + TryNextMirror: http://10.0.4.127/ubuntu//dists/trusty/restricted/binary-amd64/Packages.gz + + [Regression] + None expected. Worse that can happen is a retry when none was done previously. + + [Original description of the problem] + When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : - sudo apt-get update -qq - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] - E: Some index files failed to download. They have been ignored, or old ones used instead. + sudo apt-get update -qq + W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] + W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] + W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/re
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
Julian, first of all, you are right, I did mix up the series, I'll fix that. Now, Michael Vogt (mvo), Debian's APT main developer is currently reviewing the one line fix that solves this issue. I have a reproducer and was able to identify the root cause and prepare a fix. The source of the problem is the following : string msg = "\nIndex-File: true"; // FIXME: this really should use "IndexTarget::IsOptional()" but that //seems to be difficult without breaking ABI if (ShortDesc().find("Translation") != 0) As outlined in the FIXME, IndexTarget::IsOptional() should be used, and that is what started to happen with Wily. The code now reads : if(Target.IsOptional) msg += "\nFail-Ignore: true"; Here is the complete explanation as provided to MVO : "When the Queue responsible for fetching the Package file runs, it builds the Custom600 header with the following method : > // AcqIndex::Custom600Headers - Insert custom request headers > /*{{{*/ > // - > > /* The only header we use is the last-modified header. */ > > string pkgAcqIndex::Custom600Headers() > > { > >string Final = _config->FindDir("Dir::State::lists"); > >Final += URItoFileName(RealURI); > >if (_config->FindB("Acquire::GzipIndexes",false)) > > Final += ".gz"; > > > >string msg = "\nIndex-File: true"; > >// FIXME: this really should use "IndexTarget::IsOptional()" but that > >//seems to be difficult without breaking ABI > >if (ShortDesc().find("Translation") != 0) > > msg += "\nFail-Ignore: true"; > >struct stat Buf; > >if (stat(Final.c_str(),&Buf) == 0) > > msg += "\nLast-Modified: " + TimeRFC1123(Buf.st_mtime); > > > >return msg; > The custom header is set with "Fail-Ignore: true" in the case of a Translation file. This was true until Wily when the code got refactored to use "IndexTarget::IsOptional" as mentionned in the FIXME. As far as I can tell, the issue here is that this is wrong : if (ShortDesc().find("Translation") != 0) when string::find() does not find the string, it returns string::npos so this test is always true. This means that when Packages is downloaded, Fail-Ignore: is also set to true, which means that TryNextMirror() will not retry. In MirrorMethod::Failed() : >if (!Queue->FailIgnore && TryNextMirror()) > > return; > Changing the test in pkgAcqIndex::Custom600Headers() to be : if (ShortDesc().find("Translation") != string::npos) Seems to fix the issue quite nicely. From what I can tell, this is the only place where I found that test on 0. Other places do test with string::npos. " As soon as MVO is ok with this, I will SRU the modification and upload the fix. Kind regards, ...Louis ** Changed in: apt (Ubuntu Trusty) Status: Invalid => In Progress ** Changed in: apt (Ubuntu Xenial) Status: In Progress => Invalid ** Changed in: apt (Ubuntu Trusty) Importance: Undecided => Medium ** Changed in: apt (Ubuntu Xenial) Importance: Medium => Undecided ** Changed in: apt (Ubuntu Trusty) Assignee: (unassigned) => Louis Bouchard (louis-bouchard) ** Changed in: apt (Ubuntu Xenial) Assignee: Louis Bouchard (louis-bouchard) => (unass
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
"Marking dev + xenial as Invalid since it works from Xenial onward" You marked trusty and dev as invalid, and kept xenial open? Unfortunately, there have been huge changes in the acquire system in 1.1, so there is likely no way to fix this in trusty. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Invalid Status in apt source package in Trusty: Invalid Status in apt source package in Xenial: In Progress Bug description: When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to download. They have been ignored, or old ones used instead. The same configuration works correctly on Xenial To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1625667/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
Notably, there has been no related change of the mirror method since the trusty release. So if it started working later, than that is related to the reworked acquire system. As such, I'd like this bug to be closed as invalid, as it's not worth it. The pre 1.1 acquire system cannot really deal with failures gracefully. You could of course try wily, if it works there, you're lucky, and can backport the fix. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Invalid Status in apt source package in Trusty: Invalid Status in apt source package in Xenial: In Progress Bug description: When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to download. They have been ignored, or old ones used instead. The same configuration works correctly on Xenial To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1625667/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp
[Touch-packages] [Bug 1625667] Re: Trusty: apt does not try next mirror if index file download fails with mirror:// source
Marking dev + xenial as Invalid since it works from Xenial onward -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apt in Ubuntu. https://bugs.launchpad.net/bugs/1625667 Title: Trusty: apt does not try next mirror if index file download fails with mirror:// source Status in apt package in Ubuntu: Invalid Status in apt source package in Trusty: Invalid Status in apt source package in Xenial: In Progress Bug description: When using the mirror://mirrors.ubuntu.com/mirrors.txt functionality on Trusty, if the download of the index file fails, apt will not retry on another archive listed in mirrors.txt and will fail. Running the following on Trusty leads to the following result : sudo apt-get update -qq W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-security/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-amd64/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/main/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/restricted/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] W: Failed to fetch mirror://mirrors.ubuntu.com/mirrors.txt/dists/trusty-updates/universe/binary-i386/Packages 404 Not Found [Mirror: http://ftp.availo.se/ubuntu/] E: Some index files failed to download. They have been ignored, or old ones used instead. The same configuration works correctly on Xenial To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/1625667/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp