[issue23576] HTTPS reads can block when content length not available and timeout set.
Antoine Pitrou added the comment: This is now fixed in all branches. Thanks for the report! -- resolution: - fixed stage: - resolved status: open - closed versions: +Python 2.7 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Roundup Robot added the comment: New changeset 371cf371a6a1 by Antoine Pitrou in branch '2.7': Issue #23576: Avoid stalling in SSL reads when EOF has been reached in the SSL layer but the underlying connection hasn't been closed. https://hg.python.org/cpython/rev/371cf371a6a1 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Roundup Robot added the comment: New changeset 01cf9ce75eda by Antoine Pitrou in branch '3.4': Issue #23576: Avoid stalling in SSL reads when EOF has been reached in the SSL layer but the underlying connection hasn't been closed. https://hg.python.org/cpython/rev/01cf9ce75eda New changeset fc0201ccbcd4 by Antoine Pitrou in branch 'default': Issue #23576: Avoid stalling in SSL reads when EOF has been reached in the SSL layer but the underlying connection hasn't been closed. https://hg.python.org/cpython/rev/fc0201ccbcd4 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
New submission from Cory Benfield: Initially reported on the requests bug list at https://github.com/kennethreitz/requests/issues/2467 In cases when a remote web server sends a non-chunked response that does not have a content length, it is possible to get http.client to hang on a read. To hit it requires a specific set of circumstances: - Python 3 (this bug is not seen on Python 2 in my testing) - HTTPS (using plaintext HTTP resolves the problem) - A socket timeout must be set (leaving it unset, i.e. leaving the socket in blocking mode, resolves this problem) - The server must not send a content-length or set Transfer-Coding: chunked. - The reads must not be an even divisor of the content's length. - The timeout must be longer than the time the server is willing to keep the connection open (otherwise an exception is thrown). The following code can be used as a sample to demonstrate the bug: import http.client import time def test(connection_class=http.client.HTTPSConnection, timeout=10, read_size=7): start = time.time() c = connection_class('sleepy-reaches-6892.herokuapp.com') c.connect() if timeout is not None: c.sock.settimeout(timeout) c.request('GET', '/test') r = c.getresponse() while True: data = r.read(read_size) if not data: break print('Finished in {}'.format(time.time() - start)) Below are the results from several different runs: test(): Finished in 4.8294830322265625 test(connection_class=http.client.HTTPConnection): Finished in 0.3060309886932373 test(timeout=None): Finished in 0.6070599555969238 test(read_size=2): Finished in 0.6600658893585205 As you can see, only this particular set of features causes the bug. Note that if you change the URL to one that does return a Content-Length (e.g. http2bin.org/get), this bug also goes away. This is a really weird edge case bug. There's some extensive investigation over at the requests bug report, but I've not been able to convincingly point at the problem yet. -- components: Library (Lib) messages: 237151 nosy: Lukasa priority: normal severity: normal status: open title: HTTPS reads can block when content length not available and timeout set. versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Changes by Zachary Salzbank z...@key.me: -- nosy: +Zachary Salzbank ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Changes by Demian Brecht demianbre...@gmail.com: -- nosy: +demian.brecht ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Changes by Piotr Dobrogost p...@bugs.python.dobrogost.net: -- nosy: +piotr.dobrogost ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Antoine Pitrou added the comment: Reproducing seems a bit irregular. Note that the last bytestring (the empty bytestring) is what takes time to read. Also note that HTTPResponse is a buffered I/O object, so normally you don't need to read up to the empty string. You can stop when you got less bytes than what you asked for. -- nosy: +pitrou versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23576] HTTPS reads can block when content length not available and timeout set.
Antoine Pitrou added the comment: Varying reproduceability may have to do with sleepy-reaches-6892.herokuapp.com resolving to different endpoints (that domain name has a stupidly small TTL, by the way). Anyway, for an unknown reason the following patch seems to fix the issue. -- keywords: +patch Added file: http://bugs.python.org/file38326/ssl_read_fixup.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23576 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com