On May 13, 9:35 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > > In a pure world, the only writing that is done would be within the > > handle_send() callbacks within the select loop. Then again, in a > > perfect world, calling readable() and writable() would have no strange > > side affects (as your example below has), and all push*() calls would > > be made within the handle_*() methods. > > It wouldn't have those side-effects if push really just pushed. :-P > > > We do not live in a pure world, Python isn't pure (practicality beats > > purity), and by attempting to send some data each time a .push*() > > method is called, there are measurable increases in transfer rates. > > -- 8< -- > > > Yes, it would be convenient to not have push*() actually send data > > when called in some cases, but in others, the increase in data > > transfer rates and/or reduction in latency is substantial. > > If it increases transfer speed that much, the calling application > almost has to be broken, or at least not designed as it should be - of > course there are such applications, but you know...
It's not a matter of being broken at all, it's a matter of control flow. When we immediately try to send whenever a .push() call is made, the underlying TCP/IP stack will accept a reasonably large amount of data before it actually fills up (the most recent FreeBSD, from what I understand, will accept up to 1 meg, which is how they are able to saturate 10Gbit links), and by tossing the data into the the TCP/IP buffer early, the data gets sent earlier, thus reducing latency. Further, because we are making more actual calls to socket.send(), assuming the underlying TCP/IP buffer isn't filled (which may or may not be a good assumption), and assuming that the link has more capacity than is being used (usually the case on LANs and high-speed internet links), putting more data into the buffer to be handled by the underlying link layers will also increase transfer speeds. When the socket.send() calls are delayed until the next pass through the loop, and we aren't doing an initial send, then we don't get the benefit of the underlying TCP/IP socket layer buffering. In my experience over high-speed connections (LANs, Gbit WAN links, local machine connections), I have found that increasing block sizes to 32k to significantly improve performance for bandwidth constrained applications, as there are far fewer blocks to toss to the underlying layers, less Python code execution (Python 2.5 has a default block size of 512 bytes, or 64x as much Python execution to send the same amount of data, and one of the proposed 2.6 changes is to up this to a more reasonable 4096 bytes), and more effective use of the TCP/IP buffers (which are typically 64k or larger). > Anyway, I went for a subclassing way of dealing with it, and it works > fine. > > Thanks for the reply though, hadn't considered possibly "flawed" > applications where the asyncore loop isn't revisited as often as it > should. :-> > Ludvig Again, it's not about the application being flawed, it's a matter of control flow. ;) Also, it's not a matter of any timeouts in the select/poll loops (as Giampaolo suggested); if any socket is readable or writable, those calls will return immediately (a few hundred microseconds per call isn't bad). - Josiah -- http://mail.python.org/mailman/listinfo/python-list
