Launchpad has imported 23 comments from the remote bug at http://bugs.squid-cache.org/show_bug.cgi?id=5154.
If you reply to an imported comment from within Launchpad, your comment will be sent to the remote bug automatically. Read more about Launchpad's inter-bugtracker facilities at https://help.launchpad.net/InterBugTracking. ------------------------------------------------------------------------ On 2021-09-01T22:03:48+00:00 G-bugs wrote: Playing around with squid-5.1 on my internal proxy, I keep noticing that accessing Google search results sometimes yield a blank page. It seems that one of squid's kids will randomly die due to an assertion failure in Ip::Address::getAddrInfo, file ip/Address.cc:663: if (force == AF_INET6 || (force == AF_UNSPEC && Ip::EnableIpv6 && isIPv6()) ) { /* IPv6 Stuff */ } else if ( force == AF_INET || (force == AF_UNSPEC && isIPv4()) ) { /* IPv4 Stuff */ } else { IASSERT("false",false); } Attaching gdb to a running foreground process shows that this function is getting a struct addrinfo where the address is all zero's. I forcefully disabled IPv6 support in my copy for specific reasons (more on that below). As a result, it fails the first set of conditions because Ip::EnableIpv6 will evaluate to false (but isIPv6() will return true), and then it fails the second condition because isIPv4() returns false, thus it hits the assert call. I don't know exactly why this happens on Google. I suspect it may be due to a 0.0.0.0 address coming back from my internal DNS server because I am running StevenBlack's blocklist and returning NXDOMAIN for known advertising domains, but that's just a guess at this point. It's almost impossible to pin down in the cache.log where the core dump is happening, even with ALL,9 enabled. Looking at the conditional logic, I feel it's not terribly robust and handling all of the possible edge cases, of which mine is likely but one. I am not too sure the best way to work around such dodgy logic. Here's some more technical data: Reading symbols from /usr/local/sbin/squid... (gdb) handle SIGPIPE pass nostop noprint Signal Stop Print Pass to program Description SIGPIPE No No Yes Broken pipe (gdb) handle SIGTERM pass nostop noprint Signal Stop Print Pass to program Description SIGTERM No No Yes Terminated (gdb) handle SIGUSR1 pass nostop noprint Signal Stop Print Pass to program Description SIGUSR1 No No Yes User defined signal 1 (gdb) handle SIGHUP pass Signal Stop Print Pass to program Description SIGHUP Yes Yes Yes Hangup (gdb) handle SIGKILL pass Signal Stop Print Pass to program Description SIGKILL Yes Yes Yes Killed (gdb) handle SIGSEGV stop Signal Stop Print Pass to program Description SIGSEGV Yes Yes Yes Segmentation fault (gdb) handle SIGABRT stop Signal Stop Print Pass to program Description SIGABRT Yes Yes Yes Aborted (gdb) run -NC Starting program: /usr/local/sbin/squid -NC 2021/09/01 17:27:44| WARNING: BCP 177 violation. IPv6 transport forced OFF by build parameters. [Detaching after fork from child process 79855] [Detaching after fork from child process 80160] assert "false" at line 663 Ip::Address invalid? with isIPv4()=F, isIPv6()=T ADDRESS: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Assertion failed: (false), function getAddrInfo, file Address.cc, line 663. Program received signal SIGABRT, Aborted. 0x000000080a49c2ea in thr_kill () from /lib/libc.so.7 (gdb) bt #0 0x000000080a49c2ea in thr_kill () from /lib/libc.so.7 #1 0x000000080a411064 in raise () from /lib/libc.so.7 #2 0x000000080a4c5f29 in abort () from /lib/libc.so.7 #3 0x000000080a3f3f81 in __assert () from /lib/libc.so.7 #4 0x00000000009043e8 in Ip::Address::getAddrInfo (this=0x80c06b5c8, dst=@0x7fffff0b5640: 0x8268e81f0, force=0) at Address.cc:663 #5 0x00000000008f807d in comm_openex (sock_type=1, proto=6, addr=..., flags=1, note=0x80c0f1a78 "::") at comm.cc:345 #6 0x000000000096a989 in Comm::ConnOpener::createFd (this=0x80c06a1f8) at ConnOpener.cc:266 #7 0x000000000096a590 in Comm::ConnOpener::start (this=0x80c06a1f8) at ConnOpener.cc:240 #8 0x00000000008e857f in NullaryMemFunT<AsyncJob>::doDial (this=0x82051b598) at ../../src/base/AsyncJobCalls.h:103 #9 0x00000000008e8ab4 in JobDialer<AsyncJob>::dial (this=0x82051b598, call=...) at ../../src/base/AsyncJobCalls.h:175 #10 0x00000000008e8925 in AsyncCallT<NullaryMemFunT<AsyncJob> >::fire (this=0x82051b560) at ../../src/base/AsyncCall.h:145 #11 0x00000000008e4000 in AsyncCall::make (this=0x82051b560) at AsyncCall.cc:44 #12 0x00000000008e5075 in AsyncCallQueue::fireNext (this=0x80c1086b0) at AsyncCallQueue.cc:60 #13 0x00000000008e4c62 in AsyncCallQueue::fire (this=0x80c1086b0) at AsyncCallQueue.cc:43 #14 0x00000000005e4fb9 in EventLoop::dispatchCalls (this=0x7fffff0b6160) at EventLoop.cc:144 #15 0x00000000005e4e98 in EventLoop::runOnce (this=0x7fffff0b6160) at EventLoop.cc:121 #16 0x00000000005e4cf3 in EventLoop::run (this=0x7fffff0b6160) at EventLoop.cc:83 #17 0x0000000000771930 in SquidMain (argc=2, argv=0x7fffff0b6288) at main.cc:1716 #18 0x0000000000770c29 in SquidMainSafe (argc=2, argv=0x7fffff0b6288) at main.cc:1403 #19 0x0000000000770bf2 in main (argc=2, argv=0x7fffff0b6288) at main.cc:1391 (gdb) frame 4 #4 0x00000000009043e8 in Ip::Address::getAddrInfo (this=0x80c06b5c8, dst=@0x7fffff0b5640: 0x8268e81f0, force=0) at Address.cc:663 663 IASSERT("false",false); (gdb) list 658 659 dst->ai_addrlen = sizeof(struct sockaddr_in); 660 661 dst->ai_family = ((struct sockaddr_in*)dst->ai_addr)->sin_family; 662 } else { 663 IASSERT("false",false); 664 } 665 } 666 667 void (gdb) p *dst $1 = {ai_flags = 4, ai_family = 0, ai_socktype = 1, ai_protocol = 6, ai_addrlen = 0, ai_canonname = 0x0, ai_addr = 0x0, ai_next = 0x0} --- As for why I disabled IPv6, my home network is not ready yet to talk to the outside IPv6 world. I have some limited IPv6 going on my internal network, but Squid appears to aggressively favour IPv6 over IPv4, and some websites return AAAA records first, thus Squid then attempts to connect to them, which will naturally fail. The biggest culprit here is actually the Squid wiki, which may be because the SSL settings on the site have a misconfiguration of some sort (returns SQUID_TLS_ERR_CONNECT+TLS_ERR_IO=1) if you force it to IPv4, so it might be attempting to use the IPv6 address. Beats me, but nonetheless, I'll miss the 'dns_v4_first' setting in 5.*. Had to add a whitelist exception to the wiki to get it to work. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/0 ------------------------------------------------------------------------ On 2021-09-02T00:48:31+00:00 Alex Rousskov wrote: Thank you for reporting and detailing this bug. I agree with your assessment of the conditions in question, but have not studied that code enough to suggest a specific fix. Hopefully, somebody else will volunteer to do that. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/1 ------------------------------------------------------------------------ On 2021-09-02T01:05:01+00:00 G-bugs wrote: (In reply to Alex Rousskov from comment #1) > Thank you for reporting and detailing this bug. I agree with your assessment > of the conditions in question, but have not studied that code enough to > suggest a specific fix. Hopefully, somebody else will volunteer to do that. I ended up re-enabling the IPv6 support, because it got annoying having 2.6-2.8GB core files getting constantly written out by just navigating Google. The assertion goes away with IPv6 enabled, which means the first conditional is catching and handling it in that case. The same exact logic exists in squid-4.16, too. Possibly earlier versions as well, but I did not check. Interestingly-enough, I found this comment in comm.cc:621: /* Handle IPv6 over IPv4-only socket case. * this case must presently be handled here since the getAddrInfo asserts on bad mappings. * NP: because commResetFD is private to ConnStateData we have to return an error and * trust its handled properly. */ if (F->sock_family == AF_INET && !address.isIPv4()) { errno = ENETUNREACH; return Comm::ERR_PROTOCOL; } And I do notice in cache.log that there are instances where squid, during the ssl-bump phases, is, or appears to, connect to itself over [::]:443, even if built without IPv6 support. I suspect this is something akin to the IPv6-over-IPv4 socket case, but not getting handled, so it makes it to the getAddrInfo() call and kills the kid. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/2 ------------------------------------------------------------------------ On 2021-09-02T18:12:40+00:00 Amos Jeffries wrote: For anyone wanting to investigate this further please be aware that IPv4-only 0.0.0.0 maps into Ip::Address as ::ffff:0.0.0.0 and the isIPv4() test will match as true. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/3 ------------------------------------------------------------------------ On 2021-11-30T18:29:33+00:00 Jason-spashett wrote: Just to confirm, I have reproduced the same thing either by DNS resolution, which sometimes returns the ipv6, and sometimes the ipv4 (happy eyeballs I presume) Or, more directly by entering into a browser the url: https://[2001:4860:4864:5::6d] The stack traces are the same I believe. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/4 ------------------------------------------------------------------------ On 2021-11-30T19:09:44+00:00 Alex Rousskov wrote: (In reply to Jason from comment #4) > Just to confirm, I have reproduced the same thing Do you have IPv6 support disabled? If yes, then where/how? That information may help whoever volunteers to fix this bug. > Or, more directly by entering into a browser the url: > https://[2001:4860:4864:5::6d] FWIW, that URL results in ERR_CONNECT_FAIL in my v5-based IPv6-not- disabled environment that does not have IPv6 connectivity. No assertions. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/5 ------------------------------------------------------------------------ On 2021-12-01T14:24:21+00:00 Jason-spashett wrote: My configuration that reproduces the problem is as follows, and has the --disable-ipv6 configure option: $ /usr/local/squid5/sbin/squid --version Squid Cache: Version 5.2-VCS Service Name: squid This binary uses OpenSSL 1.1.1 11 Sep 2018. For legal restrictions on distribution see https://www.openssl.org/source/license.html configure options: '--disable-arch-native' '--disable-dependency- tracking' '--disable-eui' '--enable-auth' '--enable-basic-auth- helpers=getpwnam,LDAP,PAM' '--enable-digest-auth-helpers=password' '-- enable-external-acl-helpers=file_userip,LDAP_group,unix_group' '-- enable-follow-x-forwarded-for' '--enable-ssl-crtd' '--enable- translation' '--localstatedir=/var' '--with-logdir=/var/log/squid' '-- with-openssl' '--disable-ipv6' '--enable-linux-netfilter' '-- prefix=/usr/local/squid5' The sysconf disables ipv6 like so: net.ipv6.conf.all.disable_ipv6 = 1 $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 18.04.6 LTS Release: 18.04 Codename: bionic $ uname -a Linux js_dev 5.4.0-89-generic #100~18.04.1-Ubuntu SMP Wed Sep 29 10:59:42 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux All other interfaces are down except for loopback, which doesn't seem remarkable. $ ip addr show eth0 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000 link/ether 00:50:56:95:c0:7b brd ff:ff:ff:ff:ff:ff inet 10.51.9.149/24 brd 10.51.9.255 scope global dynamic eth0 valid_lft 77839sec preferred_lft 77839sec Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/6 ------------------------------------------------------------------------ On 2021-12-01T16:45:48+00:00 Alex Rousskov wrote: Thank you! Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/7 ------------------------------------------------------------------------ On 2021-12-08T23:01:05+00:00 Jeffrey Kintscher wrote: I created a pull request that fixes this bug. https://github.com/squid-cache/squid/pull/947 Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/8 ------------------------------------------------------------------------ On 2021-12-09T04:09:21+00:00 Alex Rousskov wrote: Thank you for opening a pull request. I do not know enough to point you towards the final solution, but I hope I asked the right questions for others to do that during PR review. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/9 ------------------------------------------------------------------------ On 2022-10-03T10:26:56+00:00 Helmut-pedit wrote: Some information (maybe) related to this bug: I've installed the newest version of squid (5.7) and after a while the process died silently. Since I did know about this bug I tried out connecting to an ipv6 address like Jason described in Comment 4 or in my case some "unique local address" (https://[fd00::4:10:4:201:60]/) and the squid process died. It also makes no difference if compiling squid with or without disable- ipv6! This also happens when using squid 5.2 from the distribution of rhel9_appstream which is compiled without "disable-ipv6". Is there some progress in fixing this bug? In the meanwhile I use an ACL which denies an ipv6 Destination ... I don't know if this produces other problems. Regards, Helmut Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/10 ------------------------------------------------------------------------ On 2022-10-04T20:42:59+00:00 Alex Rousskov wrote: (In reply to Helmut Pedit from comment #10) > Is there some progress in fixing this bug? No progress since January 1, 2022 AFAICT. There was good progress prior to that. You can track the current state at https://github.com/squid- cache/squid/pull/947 Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/11 ------------------------------------------------------------------------ On 2022-10-05T08:16:22+00:00 Helmut-pedit wrote: Thank's for your response ... Some updates (informatoions) from myside: I figured out that my server was configured with "ipv6.disable=1" in GRUB_CMDLINE_LINUX which pass to the kernel. Therefore there are no differences when compiling squid with or without "--disable-ipv6" option! In either case squid dies when trying to connect to http(s)://[fd00::4:10:4:201:60] After Removing this kernel option (and rebboting) and compiling squid without "--disable-ipv6" leads to the result as you described in comment #5. If compiling squid with "--disable-ipv6" and trying the above url squid is dying again ("Address.cc:663 getAddrInfo() assertion failed: false"). Regards,Helmut Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/12 ------------------------------------------------------------------------ On 2022-11-30T16:18:55+00:00 Alex Rousskov wrote: *** Bug 5249 has been marked as a duplicate of this bug. *** Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/13 ------------------------------------------------------------------------ On 2023-02-01T12:03:42+00:00 taoky wrote: You can write a library hooking `getaddrinfo()` to return IPv4 addresses only as a temporary workaround: just set hints.ai_family to AF_INET before calling the real `getaddrinfo()` in libc. Not quite sure if it will bring other problems, but it seems that squid v5 doesn't crash any more after LD_PRELOADing my hook. Although IPv6 has been used widespread now, configuring it in Docker container is still somewhat inconvenient and I have to use this hack as I'm using squid in a production env and it may bring more trouble if IPv6 is enabled in Docker. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/14 ------------------------------------------------------------------------ On 2023-02-06T06:58:52+00:00 Amos Jeffries wrote: Created attachment 3845 Restore bug 3735 patch It looks like the fix for bug 3735 was lost in commit fd9c47d1075b4b01e9a283c2fd67c5958b5060c5 Please try this patch (against Squid-6 code). Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/15 ------------------------------------------------------------------------ On 2023-02-06T15:02:30+00:00 Alex Rousskov wrote: (In reply to Amos Jeffries from comment #15) > Created attachment 3845 [details] > Restore bug 3735 patch > It looks like the fix for bug 3735 was lost in commit > fd9c47d1075b4b01e9a283c2fd67c5958b5060c5 AFAICT, that check was removed (on purpose) because of the following fd9c47d change and the corresponding explicit assumption/assertion that IPv6 addresses cannot get into ipcache: " Do not cache IPv6 /etc/hosts addresses when IPv6 support is disabled. This change simplified code, made it more consistent (we did not cache AAAA records), and fixed ipcacheCycleAddr() and ipcacheMarkAllGood() that were clearing supposed-to-be-permanent "bad (IPv6 disabled)" marks. " If the patch in attachment #3845 works, where can that unsupported IPv6 address come from? > Please try this patch (against Squid-6 code). Please note that a long-term fix should not change a Dns::IpReceiver::noteIp() callback. IpCache should prevent IPv6 addresses from getting delivered to all IP receivers (and cached). Also, AFAICT, the workaround patch is not quite compatible with how Dns::CachedIps::forgetMarking() works today. See the above commit description quote about ipcacheMarkAllGood(). Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/16 ------------------------------------------------------------------------ On 2023-06-07T09:10:15+00:00 FireBurn wrote: Hi, are we any closer to getting a fix for this? We're still using this patch in production to prevent segfaults Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/17 ------------------------------------------------------------------------ On 2023-06-07T14:45:00+00:00 Alex Rousskov wrote: (In reply to Mike Lothian from comment #17) > are we any closer to getting a fix for this? FWIW, I am not aware of any recent relevant changes. Neither this bug state nor the state of PR 947 have changed AFAICT. Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/18 ------------------------------------------------------------------------ On 2023-07-13T20:38:00+00:00 Alex Rousskov wrote: A new PR with a potential fix posted at https://github.com/squid-cache/squid/pull/1421 Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/19 ------------------------------------------------------------------------ On 2023-11-21T14:02:37+00:00 Alex Rousskov wrote: I believe this bug was fixed in master/v7 commit 97bbba6 (i.e. PR 1567 at https://github.com/squid-cache/squid/pull/1567). Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/20 ------------------------------------------------------------------------ On 2024-03-07T17:30:54+00:00 Alex Rousskov wrote: *** Bug 5347 has been marked as a duplicate of this bug. *** Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/21 ------------------------------------------------------------------------ On 2024-03-07T17:34:27+00:00 Alex Rousskov wrote: (In reply to Alex Rousskov from comment #20) > I believe this bug was fixed in master/v7 commit 97bbba6 (i.e. PR 1567 at > https://github.com/squid-cache/squid/pull/1567). That fix was backported to * v6 (commit 985d2d5 included in v6.6) and * unsupported v5 (commit a0a9e6d that happened after v5.9). Reply at: https://bugs.launchpad.net/ubuntu/+source/squid/+bug/2081994/comments/22 ** Changed in: squid Status: Unknown => Fix Released ** Changed in: squid Importance: Unknown => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2081994 Title: squid 5.9 crash during ipv6 requests To manage notifications about this bug go to: https://bugs.launchpad.net/squid/+bug/2081994/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
