[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Bruce Merry added the comment: > Will you accept patches to fix this for 3.9? I'm not clear whether the "bug > fixes only" status of 3.9 allows for fixing performance regressions. Never mind, I see your already answered this on bpo-42853 (as a no). Thanks for taking the time to answer my que

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Bruce Merry added the comment: > There is nothing to do here. Will you accept patches to fix this for 3.9? I'm not clear whether the "bug fixes only" status of 3.9 allows for fixing performance regressions. -- ___ Python tracker

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Łukasz Langa added the comment: Inadasan is right, this is still fixed in 3.10 and 3.11. The 3.9 revert is due to 3.9 supporting OpenSSL older than 1.1.1. 3.10+ requires OpenSSL 1.1.1+ per PEP 644. There is nothing to do here. -- resolution: -> fixed stage: needs patch -> resolved s

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Change by Gregory P. Smith : -- resolution: fixed -> stage: resolved -> needs patch ___ Python tracker ___ ___ Python-bugs-list mai

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Inada Naoki added the comment: Note that it was reverted only in 3.9 branch. -- ___ Python tracker ___ ___ Python-bugs-list mailing

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Bruce Merry added the comment: Re-opening because the patch to fix this has just been reverted due to bpo-42853. -- status: closed -> open ___ Python tracker ___

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Łukasz Langa added the comment: New changeset 153365d864c411f6fb523efa752ccb3497d815ca by Inada Naoki in branch '3.9': [3.9] bpo-42853: Fix http.client fails to download >2GiB data over TLS (GH-27405) https://github.com/python/cpython/commit/153365d864c411f6fb523efa752ccb3497d815ca

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Inada Naoki added the comment: New changeset d6bf6f2d0c83f0c64ce86e7b9340278627798090 by Inada Naoki in branch 'master': bpo-36050: optimize HTTPResponse.read() (GH-12698) https://github.com/python/cpython/commit/d6bf6f2d0c83f0c64ce86e7b9340278627798090 -- _

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Change by Inada Naoki : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___ P

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Change by Inada Naoki : -- keywords: +patch pull_requests: +12623 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-lis

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Inada Naoki added the comment: Additionally, _safe_read calls fp.read() multiple times to handle EINTR. But EINTR is handled by socket module now (PEP 475). Now the function can be very simple. -- ___ Python tracker

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Inada Naoki added the comment: issue1296004 is too old (512MB RAM machine!) and I can not confirm it. But I think it was caused by inefficient realloc() in CRT. See https://github.com/python/cpython/blob/6c52d76db8b1cb836c136bd6a1044e85bfe8241e/Lib/socket.py#L298-L303 _fileobject called soc

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Change by Inada Naoki : -- nosy: +inada.naoki ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.py

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

Martin Panter added the comment: The 1 MiB limit was added for Issue 1296004; apparently some platforms were overallocating multiple buffers and running out of memory. I suspect the loop in "_safe_read" was inherited from Python 2, which has different kinds of file objects. The comments sugg

[issue36050] Why does http.client.HTTPResponse._safe_read use MAXAMOUNT

New submission from Bruce Merry : While investigating poor HTTP read performance I discovered that reading all the data from a response with a content-length goes via _safe_read, which in turn reads in chunks of at most MAXAMOUNT (1MB) before stitching them together with b"".join. This can re