I have been having problems with wget where it fails dns resolution too often. Then, at another invocation it would work, more or less randomly.
I finally digged it up today. It turns out to be the same bug reported as #459756 in Fedora [1]. In summary, glibc resolver perform A and AAAA petitions in parallel. Since they both use the same source port, that seems to fail with some DNS servers and/or NAT devices. The outcome I get is that sometimes both replies get through, sometimes only the IPv4 reply and sometimes only the IPv6. What made this worse is that I have the wget timeout set to 5 seconds, which is the RES_TIMEOUT value if not overriden in resolv.conf(5) Thus, the slowness problem of bug 459756, meant in this case that when one of the queries doesn't got a reply, getaddrinfo() blocks for 5 seconds, and then wget signal timeout hits earlier so even if it received one packet with the right answer it won't use it. I gave some think to it in case wget could specify its dns timeout directly to the underlying resolver, but didn't arreive at a good solution. The only way to change that seems to be by setting the RES_OPTIONS environment variable, and I'm not sure we want to do it. However, there's another easier thing to do for wget. Ulrich Drepper strongly argues there that getaddrinfo() callers should use the AI_ADDRCONFIG flag. wget is purposefully not using that flag [2] and so even if disabling ipv6 [3] it still performs the two queries (and produces the waiting behavior) whereas adding AI_ADDRCONFIG avoid it. So I propose readding AI_ADDRCONFIG to wget. Issue #1 (getaddrinfo gets confused on systems with IPv6 loopback interfaces, such as modern Linux) seems fixed per the bug comments and my testing. At least on some systems, which is already an advantage since if it failed they would be on the same state as now. Issue #2 (it is not needed) would not apply, as that avoids at least that waiting on some (broken) network setups. Issue #3 and #4 would need fixing, though (avoid setting it in AIX or if missing). There's also a comment [4] saying that the use of that flag would mean no resolving for hosts-only domains in a box with only the loopback interface, which should be also taken into account. 1- https://bugzilla.redhat.com/show_bug.cgi?id=459756 2- http://osdir.com/ml/web.wget.patches/2005-06/msg00030.html 3- sysctl -w net.ipv6.conf.all.disable_ipv6=1 4- https://bugzilla.redhat.com/show_bug.cgi?id=459756#c65
