On Nov 14, 5:27 pm, Greg Copeland <[EMAIL PROTECTED]> wrote: > On Nov 14, 9:56 am, "Giampaolo Rodola'" <[EMAIL PROTECTED]> wrote: > > > > > > > Hi, > > I'd like to know if there's a way to determine which is the best > > buffer size to use when you have to send() and recv() some data over > > the network. > > I have an FTP server application which, on data channel, uses 8192 > > bytes as buffer for both incoming and outgoing data. > > Some time ago I received a report from a guy [1] who stated that > > changing the buffers from 8192 to 4096 results in a drastical speed > > improvement. > > I tried to make some tests by using different buffer sizes, from 4 Kb > > to 256 Kb, but I'm not sure which one to use as default in my > > application since I noticed they can vary from different OSes. > > Is there a recommended way to determine the best buffer size to use? > > > Thanks in advance > > > [1]http://groups.google.com/group/pyftpdlib/browse_thread/thread/f13a82b... > > > --- Giampaolohttp://code.google.com/p/pyftpdlib/ > > As you stated, the answer is obviously OS/stack dependant. Regardless, > I believe you'll likely find the best answer is between 16K-64K. Once > you consider the various TCP stack improvements which are now > available and the rapid increase of available bandwidth, you'll likely > want to use the largest buffers which do not impose scalability issues > for your system/application. Unless you have reason to use a smaller > buffer, use 64K buffers and be done with it. This helps minimize the > number of context switches and helps ensure the stack always has data > to keep pumping. > > To look at it another way, using 64k buffers requires 1/8 the number > of system calls and less time actually spent in python code. > > If as you say someone actually observed a performance improvement when > changing from 8k buffers to 4k buffers, it likely has something to do > with python's buffer allocation overhead but even that seems contrary > to my expectation. The referenced article was not available to me so I > was not able to follow and read. > > Another possibility is 4k buffers require less fragmentation and is > likely to perform better on lossy connections. Is it possible he/she > was testing on a high lossy connection? In short, performance wise, > TCP stinks on lossy connections.- Hide quoted text - > > - Show quoted text -
Thanks for the precious advices. The discussion I was talking about is this one (sorry for the broken link, I didn't notice that): http://groups.google.com/group/pyftpdlib/browse_thread/thread/f13a82b2f41764be# --- Giampaolo http://code.google.com/p/pyftpdlib/ -- http://mail.python.org/mailman/listinfo/python-list