On Wed, 30 Sep 2009, Peter Stuge wrote:
Libssh2 is slow with a 1024 byte buffer. If I transfer big files (~10GB)
from Europe to USA, I hardly see more than 100 kbit/sec. Using "sftp" the
transfer-rate is about 2 Mbit/sec.
Yes, there is certainly opportunity for performance improvement in libssh2.
Please feel free to dig into the windowing and buffer code.
The primary area to fix is to make sure it works fine with >1024 bytes
buffers. This is basically "just" bugfixing. Buffer size around 32K or so
should currently be the optimal size.
The other thing to do to really improve SFTP performance is to make libssh2
not use the incoming buffers and map them to a single outgoing sftp packet but
instead send and read multiple packets of some specific "page" size. Or
possibly to introduce a different API more like sendfile() or similar that
knows earlier about more data. OpenSSH's primary benefit compared to libssh2
is in how sends/receives multiple independent packages and thus gets less
punished by the back-and-forth nature of the SFTP protocol. This is not just
about bugfixing some minor parts of the lib.
Fixing windowing will not improve transfer speed as much as it will make us
use less memory, especially when the network speeds are slow.
Should also look into the hpn patch, and the research behind that.
Right, but I don't think we need to do that until we've caught up with
OpenSSH for "normal" SFTP.
--
/ daniel.haxx.se
_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
