Public bug reported: Many people run squid (squid-deb-proxy, or maas-proxy) to provide ubuntu archive mirror caching and proxying. MAAS sets this up by default for users with the 'maas-proxy' package.
On or about Friday February 19, this setup began to fail for many people. Users would see 'apt-get update' returning 503 errors. For me, I saw 503 on security.ubuntu.com addresses. The reason for the failure was that the squid proxy began using ipv6 addresses for instead of ipv4. The squid proxy host did not have ipv6 connectivity and thus would fail. The fix/workaround is to add the following to your squid config: # http://www.squid-cache.org/Doc/config/dns_v4_first/ dns_v4_first on The appropriate squid config file depends on what is running squid. maas-proxy: /usr/share/maas/maas-proxy.conf squid-deb-proxy: /etc/init/squid-deb-proxy.conf I'm not sure how this previously worked, nor what change was made. One change that was made in this time frame was a glibc update (2.19-0ubuntu6.6 to 2.19-0ubuntu6.7) for security (CVE-2013-7423 CVE-2014-9402 CVE-2015-1472 CVE-2015-1473). But it doesn't seem to make sense that that would change squid3 to start looking for AAAA records when it did not previously. i can verify that as late as Thu Feb 18 06:36:07 EST 2016 i was seeing entries in my squid logs with 1455713142.896 335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 - but now i get 1455879482.210 1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 - ** Affects: maas Importance: Undecided Status: New ** Affects: squid (Ubuntu) Importance: Undecided Status: Confirmed ** Affects: squid-deb-proxy (Ubuntu) Importance: Undecided Status: Confirmed ** Also affects: squid (Ubuntu) Importance: Undecided Status: New ** Also affects: squid-deb-proxy (Ubuntu) Importance: Undecided Status: New ** Description changed: Many people run squid (squid-deb-proxy, or maas-proxy) to provide ubuntu archive mirror caching and proxying. MAAS sets this up by default for users with the 'maas-proxy' package. On or about Friday February 19, this setup began to fail for many people. Users would see 'apt-get update' returning 503 errors. For me, I saw 503 on security.ubuntu.com addresses. The reason for the failure was that the squid proxy began using ipv6 addresses for instead of ipv4. The squid proxy host did not have ipv6 connectivity and thus would fail. The fix/workaround is to add the following to your squid config: - # http://www.squid-cache.org/Doc/config/dns_v4_first/ - dns_v4_first on + # http://www.squid-cache.org/Doc/config/dns_v4_first/ + dns_v4_first on The appropriate squid config file depends on what is running squid. - maas-proxy: /usr/share/maas/maas-proxy.conf - squid-deb-proxy: /etc/init/squid-deb-proxy.conf + maas-proxy: /usr/share/maas/maas-proxy.conf + squid-deb-proxy: /etc/init/squid-deb-proxy.conf I'm not sure how this previously worked, nor what change was made. One change that was made in this time frame was a glibc update (2.19-0ubuntu6.6 to 2.19-0ubuntu6.7) for security (CVE-2013-7423 CVE-2014-9402 CVE-2015-1472 CVE-2015-1473). But it doesn't seem to make sense that that would change squid3 to start looking for AAAA records when it did not previously. - - - but if you run squid on a host that has ipv6 , but no ipv6 connectivity - --> Jonathanb ([email protected]) has joined #canonical - <-- Grazina has quit (Client exited) - <smoser> it used to work and end up getting ipv4 addresses - <smoser> something changed today or yesterday - <-- bkhan_AWAY has quit (Ping timeout: 121 seconds) - <smoser> and now that is getting ipv6 addresses. - <-- xnox has quit (Quit: ZNC - http://znc.in) - <smoser> meaning if you have no ipv6 route to security.ubuntu.com, you get 503 on 'apt-get update' - <smoser> i can verify that as late as - <smoser> human time: Thu Feb 18 06:36:07 EST 2016 - <-- verterok has quit (Quit: Coyote finally caught me) - --- john-mcaleely|away is now known as john-mcaleely - <smoser> i was seeing entries in my squid logs with - <-- alan_g has quit (Quit: Ex-Chat) - <smoser> human time: Thu Feb 18 06:36:07 EST 2016 - <-- mup has quit (Connection closed) - <smoser> 1455713142.896 335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 - - <smoser> but now i get - <smoser> 1455879482.210 1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 - + i can verify that as late as + Thu Feb 18 06:36:07 EST 2016 + i was seeing entries in my squid logs with + 1455713142.896 335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 - + but now i get + 1455879482.210 1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 - -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1547640 Title: proxy tries ipv6 and gets 503 when no ipv6 routes To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1547640/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
