Re: [Python-Dev] Adding timeout option to httplib...connect()
[EMAIL PROTECTED] wrote: Guido, I looked at urllib2 and quickly gave up. I have no idea how that code works (where is a lower level library's connection object instantiated, for example?). I presume with timeouts in the lower level libraries someone who knows how urllib2 works will be able to graft timeouts onto it without too much work. I have in my TODO list an item that says put timout to urllib2.urlopen. I once digged in urllib2, trying to figure out something, and in a lightning shock I understood it. I took a look at it again a few weeks ago, but this time wasn't able to follow it, :(. Anyway, I'll check it once again, hope to understand it, and try to came up with a patch to be able to call urllib2.urlopen with timeout parameter, for all its protocols. Regards, -- . Facundo . Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Martin You should read your SF bug list more frequently, then. You are Martin currently assigned bug #723312 (and have been since 2003-04-17), Martin which has a patch to change ftplib. Assigned by the author, because I apparently offered him some advice I no longer recall giving? After much chiding here I'll take a look. I don't know if feature requests for Roundup are still being accepted, but I hope one of its features is that it can remind people periodically of the tickets they own. My primary goal in life is not to close Python bugs and patches, so I hope people will understand if I don't think on a Thursday evening, Gee, I think I'll drop by SourceForge and see what work I have to do tonight. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
[EMAIL PROTECTED] schrieb: I don't know if feature requests for Roundup are still being accepted, but I hope one of its features is that it can remind people periodically of the tickets they own. My primary goal in life is not to close Python bugs and patches, so I hope people will understand if I don't think on a Thursday evening, Gee, I think I'll drop by SourceForge and see what work I have to do tonight. Maybe it would be better to remove you from the list of possible assignees, then? You have four more patches and four bugs assigned to you, please consider unassigning them if you don't plan to work on them. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
I don't know if feature requests for Roundup are still being accepted, but I hope one of its features is that it can remind people periodically of the tickets they own. My primary goal in life is not to close Python bugs and patches, so I hope people will understand if I don't think on a Thursday evening, Gee, I think I'll drop by SourceForge and see what work I have to do tonight. Martin Maybe it would be better to remove you from the list of possible Martin assignees, then? I'll take a look. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Martin You should read your SF bug list more frequently, then. You are Martin currently assigned bug #723312 (and have been since 2003-04-17), Martin which has a patch to change ftplib. I updated the patch to be compatible with current svn: http://python.org/sf/723312 All tests pass. Well, the only tests which fail for me seem to have nothing to do with networking. I looked at gopherlib for about five seconds then decided it wasn't worth it. (Shouldn't it be deprecated and moved out to PyPI for external maintenance?) There are still no documentation updates. I don't have the few minutes necessary at the present time to cobble something together (headed out the door). Maybe tomorrow. Guido, I looked at urllib2 and quickly gave up. I have no idea how that code works (where is a lower level library's connection object instantiated, for example?). I presume with timeouts in the lower level libraries someone who knows how urllib2 works will be able to graft timeouts onto it without too much work. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
On 2/10/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Guido, I looked at urllib2 and quickly gave up. I have no idea how that code works (where is a lower level library's connection object instantiated, for example?). I presume with timeouts in the lower level libraries someone who knows how urllib2 works will be able to graft timeouts onto it without too much work. Thanks for looking anyway! I had a quick look myself; I think it relies on the implicit call to connect() in the HTTPConnection class's request() method. This answers my question: the timeout should be passed to the HTTPConnection constructor which stores it in an instance variable just like host and port, and then connect() will get it from there. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
On 2/10/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Martin You should read your SF bug list more frequently, then. You are Martin currently assigned bug #723312 (and have been since 2003-04-17), Martin which has a patch to change ftplib. I updated the patch to be compatible with current svn: http://python.org/sf/723312 All tests pass. Well, the only tests which fail for me seem to have nothing to do with networking. I looked at gopherlib for about five seconds then decided it wasn't worth it. (Shouldn't it be deprecated and moved out to PyPI for external maintenance? PEP 3108 has gopherlib slated for removal. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
On 2/10/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Martin You should read your SF bug list more frequently, then. You are Martin currently assigned bug #723312 (and have been since 2003-04-17), Martin which has a patch to change ftplib. Assigned by the author, because I apparently offered him some advice I no longer recall giving? After much chiding here I'll take a look. I don't know if feature requests for Roundup are still being accepted, but I hope one of its features is that it can remind people periodically of the tickets they own. Good idea. Kind of like a weekly here is your Python todo list in terms of issues email. I think it should be doable. As for taking new ideas, the goal right now is to get the transition done. The biggest thing holding it up now is documentation, especially howto docs. Once we are transitioned we are expecting people to want to change things or add features. But if you want to get the ball rolling on this for possible inclusion afterwards, email tracker-discuss about your idea. My primary goal in life is not to close Python bugs and patches, so I hope people will understand if I don't think on a Thursday evening, Gee, I think I'll drop by SourceForge and see what work I have to do tonight. I understand. =) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido van Rossum schrieb: I recently needed to access an HTTP URL with a timeout. I ended up monkey-patching httplib.HTTPConnection so that the connect() method has an optional second paramer, timeout, defaulting to None; if not None, a call to settimeout() is added right after successful creation of the socket. Does anybody else think this is a good idea? (Personally I think this should've been done years ago. :-) Shall I check it into the head? Please also consider #723312, which is more general, but also more outdated. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido I recently needed to access an HTTP URL with a timeout. I ended Guido up monkey-patching httplib.HTTPConnection so that the connect() Guido method has an optional second paramer, timeout, defaulting to Guido None; if not None, a call to settimeout() is added right after Guido successful creation of the socket. Guido Does anybody else think this is a good idea? In principle it's probably a fine idea. We should consider if it's possible to develop a uniform approach to timeouts for all the libraries that use sockets though. Otherwise you run the risk of doing it in different ways for different libraries and having to make a (small, but annoying) semantic leap when going from, say, httplib to smtpllib or ftplib. Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
On Friday 09 February 2007 08:52, [EMAIL PROTECTED] wrote: In principle it's probably a fine idea. We should consider if it's possible to develop a uniform approach to timeouts for all the libraries that use sockets though. Otherwise you run the risk of doing it in different ways for different libraries and having to make a (small, but annoying) semantic leap when going from, say, httplib to smtpllib or ftplib. Agreed. In the meanwhile, there's socket.setdefaulttimeout(), which has proved quite useful. -Fred -- Fred L. Drake, Jr. fdrake at acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
On 2/9/07, Fred L. Drake, Jr. [EMAIL PROTECTED] wrote: On Friday 09 February 2007 08:52, [EMAIL PROTECTED] wrote: In principle it's probably a fine idea. We should consider if it's possible to develop a uniform approach to timeouts for all the libraries that use sockets though. Otherwise you run the risk of doing it in different ways for different libraries and having to make a (small, but annoying) semantic leap when going from, say, httplib to smtpllib or ftplib. Agreed. In the meanwhile, there's socket.setdefaulttimeout(), which has proved quite useful. Didn't work for me, since my ap is a multi-threaded webserver and I only want one specific type of request to time out. I'm not going to change ftplib.py and all the others. I do think it's relevant to decide whether the timeout should be passed to the constructor or to the connect() method. I think it may be better to pass the timeout to the constructor. -Fred -- Fred L. Drake, Jr. fdrake at acm.org -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido Didn't work for me, since my ap is a multi-threaded webserver and Guido I only want one specific type of request to time out. Understood. Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Most people are only interested in a single protocol (or module) and therefore aren't motivated to change/fix the other protocols which could benefit from timeouts. After all, who really wants to update gopherlib? So there is sits. Guido I do think it's relevant to decide whether the timeout should be Guido passed to the constructor or to the connect() method. I think it Guido may be better to pass the timeout to the constructor. Another thing to consider is how this change might percolate up to urllib/urllib2. (I advocate leaving urllib alone, but urllib2 should probably grow a timeout feature.) Skip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
You're not helping, Skip. Can you spend a minute looking at urllib2 and seeing how the two variants of my proposal (pass to constructor vs. pass to connect()) would impact on it? --Guido On 2/9/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Guido Didn't work for me, since my ap is a multi-threaded webserver and Guido I only want one specific type of request to time out. Understood. Guido I'm not going to change ftplib.py and all the others. Also understood. This has, as far as I know, been the response of everybody who has encountered this problem before. Most people are only interested in a single protocol (or module) and therefore aren't motivated to change/fix the other protocols which could benefit from timeouts. After all, who really wants to update gopherlib? So there is sits. Guido I do think it's relevant to decide whether the timeout should be Guido passed to the constructor or to the connect() method. I think it Guido may be better to pass the timeout to the constructor. Another thing to consider is how this change might percolate up to urllib/urllib2. (I advocate leaving urllib alone, but urllib2 should probably grow a timeout feature.) Skip -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Adding timeout option to httplib...connect()
I recently needed to access an HTTP URL with a timeout. I ended up monkey-patching httplib.HTTPConnection so that the connect() method has an optional second paramer, timeout, defaulting to None; if not None, a call to settimeout() is added right after successful creation of the socket. Does anybody else think this is a good idea? (Personally I think this should've been done years ago. :-) Shall I check it into the head? Index: httplib.py === --- httplib.py (revision 53456) +++ httplib.py (working copy) @@ -656,7 +656,7 @@ def set_debuglevel(self, level): self.debuglevel = level -def connect(self): +def connect(self, timeout=None): Connect to the host and port specified in __init__. msg = getaddrinfo returns an empty list for res in socket.getaddrinfo(self.host, self.port, 0, @@ -664,6 +664,8 @@ af, socktype, proto, canonname, sa = res try: self.sock = socket.socket(af, socktype, proto) +if timeout is not None: +self.sock.settimeout(timeout) if self.debuglevel 0: print connect: (%s, %s) % (self.host, self.port) self.sock.connect(sa) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
[GvR] I recently needed to access an HTTP URL with a timeout. I ended up monkey-patching httplib.HTTPConnection so that the connect() method has an optional second paramer, timeout, defaulting to None; if not None, a call to settimeout() is added right after successful creation of the socket. Does anybody else think this is a good idea? (Personally I think this should've been done years ago. :-) Shall I check it into the head? Yes. This is has been requested more than once, but no one ever got around to adding the timeout.. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding timeout option to httplib...connect()
Guido van Rossum [EMAIL PROTECTED] wrote: I recently needed to access an HTTP URL with a timeout. I ended up monkey-patching httplib.HTTPConnection so that the connect() method has an optional second paramer, timeout, defaulting to None; if not None, a call to settimeout() is added right after successful creation of the socket. Does anybody else think this is a good idea? (Personally I think this should've been done years ago. :-) Shall I check it into the head? I've done similar things to urllib in the past (also handling timeouts and insufficient data transfer rates). Seems like a reasonable change to me. It also makes me think that I should switch to httplib (since all the urls I ever fetch are http). - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
