New submission from Kristján Valur Jónsson <krist...@ccpgames.com>:
This is related to issue 4336. While that issue finally revolved around fixing the Nagle problem for xmlrpc and other http clients, here I propose to fix another performance bottleneck with xmlrpc, reading the HTTP Headers. HTTPResponse creates a socket.fileobject() with zero buffering which means that the readline() operations used to read the headers become very inefficient since individual socket.recv() calls are used for each character. The reason for this is to support users who subsequently do socket.recv() on the underlying socket, and so we don't want to read too much of the headers. Note that there is no example of this in the standard library. In the provided patch, I have removed some vestigial code from xmrlpclib.py which attempted to use recv() even though it never did (because the connection has been closed when HTTPConnection returns the response). Even so, Guido has expressed his concern that we need to keep the support for this recv() behaviour in place. Therefore, I have added a new optional argument, nobuffer=True, which users that promise not to call recv() can set to false. This will then cause the generated fileobject from HTTPResponse to have default buffering, greatly increasing the performance of the reading of the headers. ---------- components: Library (Lib) files: nobuffer.patch keywords: patch, patch messages: 79406 nosy: krisvale severity: normal status: open title: Allo buffering for HTTPResponse type: performance versions: Python 2.7 Added file: http://bugs.python.org/file12649/nobuffer.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue4879> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com