Re: [Python-Dev] socket.create_connection slow
Right, there is no way to try to simultaneously connect using ipv4 and ipv6, apparently. Also, the problem with setting the registry TcpConnectMaxRetries registry entry is that it also affects retries wen no ACK is received. This is probably something one doesn't want to mess with. Okay, so do we want to bug MS about this? Clearly it is a performance problem when implementing dual stack clients. K -Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] Sent: 15. janúar 2009 00:07 To: Kristján Valur Jónsson Cc: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow And Microsoft, realizing their problem , came up with this: http://msdn.microsoft.com/en-us/library/bb513665(VS.85).aspx Dual-stacked sockets are a useful thing to have (so useful that Linux made them the default, despite that the RFC says that the default should be IPV6_V6ONLY). The Python library should make all server sockets dual-stacked if possible. Unfortunately: - the socket option is not available on all systems, in particular, it is not available on Windows XP (you need Vista) - you'll see the 1s delay on the client side if the server is not dual-stacked, so if the server misbehaves, the client has to suffer. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Right, there is no way to try to simultaneously connect using ipv4 and ipv6, apparently. Ah, I see what you meant. No, this cannot work - what if you get positive ACKs on both protocols? Also, the problem with setting the registry TcpConnectMaxRetries registry entry is that it also affects retries wen no ACK is received. This is probably something one doesn't want to mess with. Indeed. They were wrong to overload this setting. Okay, so do we want to bug MS about this? If you think it helps, go ahead! This has been in the system for so long that they are unlikely to change it. Yet, as IPv6 deployment progresses, this case will occur more and more often (until eventually all services are dual-stacked - in which case the only effect will be that you wait 2 seconds if the service is really not available; 1s delay per protocol). If the default retry counter can't be changed, I'd suggest that they provide a socket option. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson kristjan at ccpgames.com writes: On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Is the fix ok? What if the user wanted to connect to an XMLRPC server using IPv6? 2) Have the SocketServer create a listening socket for each address family by default. I don't see how that fixes the root of the problem, not all XMLRPC servers are implemented using SocketServer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
The fix is for the unittest only, where a server is started on a separate thread using Ipv4 on localhost. Now, the problem I was describing in the mail isn't specific to xmlrpc, but for Anyone that wants to use socket.create_connection() to create a stream socket to a host whose name has both an Ipv4 and Ipv6 address. Unless the host is listening on its Ipv6 port, an unsuccessful connection attempt will first be made, taking approximately one second. Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Antoine Pitrou Sent: 14. janúar 2009 11:34 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Kristján Valur Jónsson kristjan at ccpgames.com writes: On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Is the fix ok? What if the user wanted to connect to an XMLRPC server using IPv6? 2) Have the SocketServer create a listening socket for each address family by default. I don't see how that fixes the root of the problem, not all XMLRPC servers are implemented using SocketServer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Victor Stinner wrote: Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. This is something of a bugbear on Vista in general. Doing local web-development with localhost can be really painful until you realise that switching to 127.0.0.1 solves the problem... Michael ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Maybe it's a vista thing? from socket import * socket(AF_INET6).connect((::1, 8080)) takes about one second to report active refusal. But so does an IPv4 connect. Maybe it is some kind of DOS attack throttling? I couldn't find any info. I've already asked the client in the test to use IPV4 by specifying the connection address as an IPv4 tuple (http://127.0.0.1:...;). I see no other way to do it without extensive subclassing because the HTTPConnection() class uses socket.create_connection(). Cheers, Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stinner Sent: 14. janúar 2009 12:46 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
This may be way out on a limb, but could it be a reverse lookup issue? -Original Message- From: python-dev-bounces+billy.earney=gmail@python.org [mailto:python-dev-bounces+billy.earney=gmail@python.org] On Behalf Of Kristján Valur Jónsson Sent: Wednesday, January 14, 2009 8:36 AM To: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? from socket import * socket(AF_INET6).connect((::1, 8080)) takes about one second to report active refusal. But so does an IPv4 connect. Maybe it is some kind of DOS attack throttling? I couldn't find any info. I've already asked the client in the test to use IPV4 by specifying the connection address as an IPv4 tuple (http://127.0.0.1:...;). I see no other way to do it without extensive subclassing because the HTTPConnection() class uses socket.create_connection(). Cheers, Kristján -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Victor Stinner Sent: 14. janúar 2009 12:46 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/billy.earney%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Michael Foord wrote: Victor Stinner wrote: Hi, Le Wednesday 14 January 2009 12:23:46 Kristján Valur Jónsson, vous avez écrit : socket.create_connection() trying to connect to (localhost, port) (...) return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. This is the normal behaviour of dual stack (IPv4+IPv6): IPv6 is tried before IPv4. SocketServer uses AF_INET by default, so the IPv6 port is closed on your host. Why does it take so long to try to connect to the IPv6 port? On Linux, it's immediate: $ time nc6 ::1 8080 nc6: unable to connect to address ::1, service 8080 real0m0.023s user0m0.000s sys 0m0.008s On my host (Ubuntu Gutsy), localhost name has only an IPv4 address. The address ::1 is ip6-localhost or ip6-loopback. You should check why the connect() to IPv6 is so long to raise an error. About the test: since SocketServer address family is constant (IPv4), you can force IPv4 for the client. This is something of a bugbear on Vista in general. Doing local web-development with localhost can be really painful until you realise that switching to 127.0.0.1 solves the problem... It barfs on Macs as well: indeed, it is worse, because the connection just fails there, rather than trying IPv6 and then falling back to IPv4. For instance, tunneling a connection over SSH to a Mac box via '-L :localhost:' will fail to connect at all, unless the server is listening on IPv6. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJbgo9+gerLs4ltQ4RAvK9AKCfWhQx7ntw+sUNK7FCPU+Kb9jp5QCdEqCu 9BXvzTgBKipSCtA3SdydqjI= =tYDj -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Hardly. Successful connects complete in a jiffy, only actively refused ones take a second to do so. I suspect some michief in the vista tcp stack. -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Billy Earney Sent: 14. janúar 2009 15:43 To: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow This may be way out on a limb, but could it be a reverse lookup issue? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
On Vista, it will return an AF_INET6 entry before the AF_INET one and try connection to that. This connect() attemt fails after approximately one second, after which we proceed to do an immediately successful connect() call to the AF_INET address. Can you find out why it takes a second? That should not happen; it should fail immediately (assuming no server is listening). Now, I did fix this in test_xmlrpc.py by just speficying the loopback address, but I wonder if this might not be a problem in general? Yes, but possibly with your operating system or installation. We need to understand what is happening first before trying to fix it. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Anyone that wants to use socket.create_connection() to create a stream socket to a host whose name has both an Ipv4 and Ipv6 address. Unless the host is listening on its Ipv6 port, an unsuccessful connection attempt will first be made, taking approximately one second. Again, it is a bug in the system or the installation takes a second. There should be no timeout, but an immediate error. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
It barfs on Macs as well: indeed, it is worse, because the connection just fails there, rather than trying IPv6 and then falling back to IPv4. That depends on the application. Some applications fall back (as they should, if they added their support for IPv6 correctly), some don't. For instance, tunneling a connection over SSH to a Mac box via '-L :localhost:' will fail to connect at all, unless the server is listening on IPv6. That's actually bug in sshd. It should fall back to connecting through v4, but doesn't. It's not system specific; sshd as included in Debian has the same bug. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Can you run wireshark, to find out whether it's sending out any requests that don't get responses? Could it be that your firewall is discarding the connection request? (rather than sending an ICMP destination unreachable back) Anything added to the event log? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. Kristján -Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] Sent: 14. janúar 2009 20:31 To: Kristján Valur Jónsson Cc: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
And Microsoft, realizing their problem , came up with this: http://msdn.microsoft.com/en-us/library/bb513665(VS.85).aspx K -Original Message- From: python-dev-bounces+kristjan=ccpgames@python.org [mailto:python-dev-bounces+kristjan=ccpgames@python.org] On Behalf Of Kristján Valur Jónsson Sent: 14. janúar 2009 21:40 To: Martin v. Löwis Cc: python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. Kristján -Original Message- From: Martin v. Löwis [mailto:mar...@v.loewis.de] Sent: 14. janúar 2009 20:31 To: Kristján Valur Jónsson Cc: Victor Stinner; python-dev@python.org Subject: Re: [Python-Dev] socket.create_connection slow I have no idea why the connect refusal takes so long. Maybe it's a vista thing? I've looked into this further. It doesn't just happen for localhost, but also for remote hosts, and not just for IPv6, but also for IPv4, and not just for Vista, but also for XP. The problem is this: 1. Vista sends SYN to target machine (say, through v6) 2. Target machine has no port open, and responds with RST,ACK 3. Vista waits 0.5s 4. Vista sends another SYN to target machine 5. Target machine responds with another RST,ACK 6. Vista waits another 0.5s 7. Vista retries a third time, again getting no connection 8. Vista gives up, having spend 1s in trying to establish a connection to a remote port where the remote system has confirmed that the connection is refused already a second ago. I have not found documentation for this feature of the Windows TCP stack, yet. There is the TcpMaxConnectRetransmissions parameter, but this supposed affects requests without responses (and supposed also starts of with 3s) Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson wrote: Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I know it's pointless to express my shock here, but I can't resist. It's truly amazing to me that they'd delay the connect call's failure for a second by default, in hopes that the other end might come back up between SYN's. How often could that possibly happen? Eric. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Eric Smith wrote: Kristján Valur Jónsson wrote: Aha, thanks, since my wireshark wasn't working. I boiled a few pints of water (thanks, Google) and came up with this: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I know it's pointless to express my shock here, but I can't resist. It's truly amazing to me that they'd delay the connect call's failure for a second by default, in hopes that the other end might come back up between SYN's. How often could that possibly happen? When I read it I was tempted to observe they must have been testing Microsoft network services. It is a truly bizarre rationalization of a default that appears to have been taken from DOS-era network client applications. I remember demonstrating the phenomenon on a cli-based Telnet client at least 15 years ago. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Yet another undefined thing affecting us, Martin. I think it's just another case of Microsoft says it's undefined, even though the standards clearly specify what the behavior must be, and Microsoft managed to implement the behavior that not only violates the specification, but also hurts users of their systems. See Scott's analysis for why this is a case of such ignorance; I agree with this analysis, and many other people in the net also agree. Regards, Martin P.S. The sad thing is not that Microsoft makes mistakes; all system vendors do. The said thing is that they refuse to acknowledge their mistakes, and rather chose to persist the buggy implementation than risking backwards incompatibilities: somebody might rely on getting the connection only on the third attempt. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
Kristján Valur Jónsson wrote: http://support.microsoft.com/kb/175523 Here is the summary: Note that with other implementations of TCP, such as those commonly found in many UNIX systems, the connect() fails immediately upon the receipt of the first ACK/RST packet, resulting in the awareness of an error very quickly. However, this behavior is not specified in the RFCs and is left to each implementation to decide. The approach of Microsoft platforms is that the system administrator has the freedom to adjust TCP performance-related settings to their own tastes, namely the maximum retry that defaults to 3. The advantage of this is that the service you're trying to reach may have temporarily shut down and might resurface in between SYN attempts. In this case, it's convenient that the connect() waited long enough to obtain a connection since the service really was there. Yet another undefined thing affecting us, Martin. I think RFC793 is actually pretty clear in stating that: If the receiver [of the RST packet] was in any other state, it aborts the connection and advises the user and goes to the CLOSED state. But alas, Microsoft thinks they know better.. A brief search yields a number of threads on mailing lists for proxies having to deal with this feature. The solution that I see as most viable is temporarily making the sockets nonblocking before the connect() and scheduling our own timeout. The only variation on this is I have seen is to use GetTcpTable() to retrieve the status of the socket to determine the state of the socket (since a timeout would kill connects() that are just slow too..). -- Scott Dial sc...@scottdial.com scod...@cs.indiana.edu ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] socket.create_connection slow
And Microsoft, realizing their problem , came up with this: http://msdn.microsoft.com/en-us/library/bb513665(VS.85).aspx Dual-stacked sockets are a useful thing to have (so useful that Linux made them the default, despite that the RFC says that the default should be IPV6_V6ONLY). The Python library should make all server sockets dual-stacked if possible. Unfortunately: - the socket option is not available on all systems, in particular, it is not available on Windows XP (you need Vista) - you'll see the 1s delay on the client side if the server is not dual-stacked, so if the server misbehaves, the client has to suffer. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
