New submission from Zveinn <sve...@zkynet.io>:
Hey, some time ago I ran into some code in the cpython code I thought might be possible to improve it a little bit. https://github.com/python/cpython/blob/master/Lib/http/client.py#L903 This code specifically. Notice how the self.send() method is used multiple time to construct the CONNECT request. When the network load is high, these different parts actually get split into separate network frames. This causes additional meta data to be sent, effectively costing more bandwidth and causing the request to be split into multiple network frames. This has some interesting behavior on the receiving end as well. If you send everything as a single network frame, then the receiver can read the entire thing in a single read call. If you send multiple frames, the main reader pipe now needs a temporary buffer to encapsulate the multiple calls. Because of this, sending requests as many network frames actually causes a rise in processing complexity on the receiving end. Here is a github issue I made about this problem some time ago: https://github.com/psf/requests/issues/5384 In this issue you will find detailed information and screenshots. My recommendation would be to construct the query as a whole before using a single self.send() to send the whole payload in one network frame. Even if we ignore the added complexity on the receivers end, the gain in network performance is worth it. ---------- components: C API messages: 387742 nosy: zveinn priority: normal severity: normal status: open title: def _tunnel(self): - uses multiple network writes, possibly causing unnecessary implementation complexity on the receiving end type: performance versions: Python 3.10 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue43332> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
