bad httplib latency due to IPv6 use
Hi! I noticed yesterday that a single HTTP request to localhost takes roughly 1s, regardless of the actually served data, which is way too long. After some digging, I found that the problem lies in socket.create_connection(), which first tries the IPv6 ::1 and only then tries the IPv4 127.0.0.1. The first one times out after 1s, causing the long latency. What I'm wondering is this: 1. The server only serves on IPv4, changing this to IPv6 would probably help. However, I wouldn't consider this a bug, or? 2. I don't even have any IPv6 addresses configured and I'm not using IPv6 in any way, so why does it try those at all? 3. Of course I can optimize the code for IPv4, but then I would be pessimizing IPv6 and vice versa... Any other suggestions? Uli Notes: * Using 127.0.0.1 as host works without the delay. * I'm using Python 2.7 on win7/64bit -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
On 17/10/12 09:55:13, Ulrich Eckhardt wrote: Hi! I noticed yesterday that a single HTTP request to localhost takes roughly 1s, regardless of the actually served data, which is way too long. After some digging, I found that the problem lies in socket.create_connection(), which first tries the IPv6 ::1 and only then tries the IPv4 127.0.0.1. The first one times out after 1s, causing the long latency. What I'm wondering is this: 1. The server only serves on IPv4, changing this to IPv6 would probably help. However, I wouldn't consider this a bug, or? I'd say it's a bug in your TCP/IP stack. An IP shouldn't take that long to figure out that it is not configured to connect to IPv6 addresses. 2. I don't even have any IPv6 addresses configured and I'm not using IPv6 in any way, so why does it try those at all? I have no experience with win7/64, but on earlier versions of Windows, there's a file named hosts, somewhere in a system directory. When looking up an IP address, this file is consulted first. Removing the ::1 from the entry for localhost might be a sensible work-around. 3. Of course I can optimize the code for IPv4, but then I would be pessimizing IPv6 and vice versa... I'd avoid doing that, if at all possible. Any other suggestions? Try ping localhost on the command line. It that has the same problem, than the problem is not in Python, but in your IP stack. It might be in a FAQ list for some windows-specific forum. Notes: * Using 127.0.0.1 as host works without the delay. That make sense: it's the attempt to connect to ::1 that is the problem. * I'm using Python 2.7 on win7/64bit Hope his helps, -- HansM -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
On Wed, Oct 17, 2012 at 8:37 PM, Hans Mulder han...@xs4all.nl wrote: I have no experience with win7/64, but on earlier versions of Windows, there's a file named hosts, somewhere in a system directory. When looking up an IP address, this file is consulted first. Removing the ::1 from the entry for localhost might be a sensible work-around. It ought to be c:\windows\system32\drivers\etc\hosts if your Widows is installed to default location. If localhost isn't there at all, you can add a line for it. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
What I'm wondering is this: 1. The server only serves on IPv4, changing this to IPv6 would probably help. However, I wouldn't consider this a bug, or? I'd say it's a bug in your TCP/IP stack. An IP shouldn't take that long to figure out that it is not configured to connect to IPv6 addresses. It might also be, that he has a firewall installed that is blocking access to ::1. In that case, it takes much more time to figure out that you cannot connect. Because in that case, it is not a connection refused problem, but a trying to connect to a closed/not responding port problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
In article 1s42l9-9al@satorlaser.homedns.org, Ulrich Eckhardt ulrich.eckha...@dominolaser.com wrote: Hi! I noticed yesterday that a single HTTP request to localhost takes roughly 1s, regardless of the actually served data, which is way too long. After some digging, I found that the problem lies in socket.create_connection(), which first tries the IPv6 ::1 and only then tries the IPv4 127.0.0.1. The first one times out after 1s, causing the long latency. What I'm wondering is this: 1. The server only serves on IPv4, changing this to IPv6 would probably help. However, I wouldn't consider this a bug, or? This is most likely a configuration problem on your server. You don't say what kind of system you're using, so I can only guess at the exact answer, but here's some suggestions. Look at your /etc/hosts file. What entries do you have for localhost? For example, on my OSX laptop, I've got: 127.0.0.1 localhost ::1 localhost fe80::1%lo0 localhost which gives three IP addresses (one IPv4, two IPv6) for localhost. This works because the box is properly configured for IPv6 (i.e. all services of interest listen on both protocols). On the other hand, this is on a linux box I have: 127.0.0.1 localhost ::1 ip6-localhost ip6-loopback Now, localhost only gives me the IPv4 address. If I want to connect to localhost with IPv6, I have to explicitly say ip6-localhost. My guess is that you have some variation of the first example, and what you need is some variation on the second. In the alternative, you need to fix up your HTTP server to listen on both protocols. For sure, in the long term, that's the right solution, because Real Soon Now, we're going to run out of IPv4 addresses and everybody is going to switch to IPv6. Of course, that's been true for about the past 15 years, but now it's really, really true. -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
Some updates on the issue: The etc/hosts file contains the following lines: # localhost name resolution is handled within DNS itself. # 127.0.0.1 localhost # ::1 localhost As I understand it, those effectively mean that localhost is not resolved via this hosts file but within DNS itself, whatever that exactly means. Concerning the question whether ping works, the result is that ping localhost works and that it uses the IPv6 (sic!) address. I also tried ping ::1 and ping 127.0.0.1 and both work. Weird, as ipconfig doesn't list any IPv6 addresses. Concerning the question whether a firewall blocks and unnecessarily delays connection attempts to ::1, I haven't determined that yet. I'll ask our admins here to verify whether that is the case. Lastly, I tried the same using Python 3.2.3/64bit (the other was actually the 32-bit version), and the same issues are there. In summary, I guess that it's a problem with the IP configuration not one in Python's or my code. Sorry for the noise... Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
On Wed, Oct 17, 2012 at 11:40 PM, Ulrich Eckhardt ulrich.eckha...@dominolaser.com wrote: Concerning the question whether a firewall blocks and unnecessarily delays connection attempts to ::1, I haven't determined that yet. I'll ask our admins here to verify whether that is the case. It would only be a software firewall on the local machine that could be doing that; anything on any other machine can't. If you have no particular reason to do otherwise, I would recommend removing all firewalling from localhost - you won't lose much security (two programs running on your computer are allowed to communicate freely in plenty of other ways anyway), and it'll make all sorts of things easier/simpler. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
In article eil2l9-nhm@satorlaser.homedns.org, Ulrich Eckhardt ulrich.eckha...@dominolaser.com wrote: Some updates on the issue: The etc/hosts file contains the following lines: # localhost name resolution is handled within DNS itself. # 127.0.0.1 localhost # ::1 localhost As I understand it, those effectively mean that localhost is not resolved via this hosts file but within DNS itself, whatever that exactly means. The path from hostname to IP address is a long and tangled one. It starts with a call to getaddrinfo(). From there, the details depend on a wide variety of decisions made by whatever idiots designed, installed, and configured your operating system. One likely path is to check in /etc/nsswitch.conf to see what data sources the resolver should consult. On the box I'm using at the moment, it says: hosts: files dns which is pretty typical. That means, first look in [some set of static files, which usually, but not always, means /etc/hosts], and if you don't find it there, ask DNS. Other possibilities include NIS, NISPLUS, and maybe some other perverse things. And, finally, you get to DNS, so now you have to look in (probably) /etc/resolv.conf. I see on my linux box, even that has grown more tendrils; there's a whole /etc/resolvconf *directory* full of more config files which describe additional ways somebody could have misconfigured this mess. Personally, if I were you (and assuming you don't have some corporate IT nazis to deal with), I would just put localhost in /etc/hosts and be done with it. -- http://mail.python.org/mailman/listinfo/python-list
Re: bad httplib latency due to IPv6 use
On Thu, Oct 18, 2012 at 11:28 AM, Roy Smith r...@panix.com wrote: One likely path is to check in /etc/nsswitch.conf to see what data sources the resolver should consult. On the box I'm using at the moment, it says: hosts: files dns This is true on Linux, and presumably on various other Unices, but the OP is on Windows. I think it's hard-coded on Windows to just use the hosts file and then DNS. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
