Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-20 Thread Daniel Stenberg
On Wed, 20 Jun 2007, Mononen Jussi wrote: > But I guess the packet pipelining is out of the scope of 0.15 release. Yes, I would think that such pipelining support will take some thinking, some internal restructuring and more, so we better save that until after 0.15... -

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-20 Thread James Housley
On Jun 20, 2007, at 2:10 AM, Mononen Jussi wrote: > What about a comparison with openssh's sftp next to our sftp_nonblocking? >>> >>> OpenSSH sftp: 0:01.06, 0:01.04, 0:01.01 >>> >>> libssh2 sftp_nonblock (libssh2 packet size:4K, sftp_nonblock >>> read 64K "static"): 0:01.60, 0:01.60, 0:

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-19 Thread Mononen Jussi
>> > What about a comparison with openssh's sftp next to our >> > sftp_nonblocking? >> >> OpenSSH sftp: 0:01.06, 0:01.04, 0:01.01 >> >> libssh2 sftp_nonblock (libssh2 packet size:4K, sftp_nonblock >> read 64K "static"): 0:01.60, 0:01.60, 0:01.58 >> >> That looks close enough to be called th

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-19 Thread James Housley
On Jun 19, 2007, at 10:09 AM, Daniel Stenberg wrote: > On Tue, 19 Jun 2007, James Housley wrote: > >> Some quick test with the sftp and sftp_nonblocking programs in >> example/simple shows that the amount of data requested in the call >> to libssh2_sftp_read() is much more important the the SS

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-19 Thread Daniel Stenberg
On Tue, 19 Jun 2007, James Housley wrote: > Some quick test with the sftp and sftp_nonblocking programs in > example/simple shows that the amount of data requested in the call to > libssh2_sftp_read() is much more important the the SSH packet size. What about a comparison with openssh's sftp ne

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-19 Thread James Housley
Some quick test with the sftp and sftp_nonblocking programs in example/simple shows that the amount of data requested in the call to libssh2_sftp_read() is much more important the the SSH packet size. All test were between two computers on the same network connected by 100MBs switch and t

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-18 Thread Mononen Jussi
>> * src/packet.c 1.39 - reading bigger blocks from >> network. Significantly improves performance due to lesser syscalls >> (and probably it influences the TCP window changes too). This is a >> temporary measure while libssh2 is mostly synchronous. >> >> I made diff of vers 1.38 and 1.3

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-15 Thread Daniel Stenberg
On Thu, 14 Jun 2007, Gutjahr, Troy wrote: > I'm using libssh2 on Solaris to retrieve files via sftp from a > telecommunications network element that has (adapted) OpenSSH code embedded > in it. As far as I know, I can't use scp with this device. Also, my > requirements explictly force me to use

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-15 Thread Mononen Jussi
>> I'm using libssh2 on Solaris to retrieve files via sftp from a >> telecommunications network element that has (adapted) OpenSSH code >> embedded in it. As far as I know, I can't use scp with this device. >> Also, my requirements explictly force me to use sftp. Also, SCP supports only get/put f

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Gutjahr, Troy
Sent: Thursday, June 14, 2007 3:02 AM To: libssh2-devel@lists.sourceforge.net Cc: Mononen Jussi Subject: Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write On 6/14/07, Mononen Jussi <[EMAIL PROTECTED]> wrote: > > But SCP is not compatible with all SSH implementations

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Mononen Jussi
>> 1 - do updated SCP measurements on current libssh2 CVS code >> and compare to recent openssh tools. >> >> 2 - work on making SCP perform roughly at the same speed as >> openssh - or more. We should do profiling etc to see where >> time is spent and where we should focus our effo

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Daniel Stenberg
On Thu, 14 Jun 2007, Mononen Jussi wrote: > We did some tests with libssh2-0.12 (with our own patch to change the 16 > byte read loop into 32kb read loop) and we came to a conclusion that libssh2 > is roughly 50% slower in file transfer than the command line counterparts. > Eg. we had one test

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Mononen Jussi
>> > But SCP is not compatible with all SSH implementations. We >> are using >> > libssh2 in mixed environments and sometimes OpenSSH jsuts isn't an >> > option. >> >> I am not doubting what you say, I am just curious: what environments >> have ssh and sftp, but not scp? The uncompatibility is

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Chris Nystrom
On 6/14/07, Mononen Jussi <[EMAIL PROTECTED]> wrote: > > But SCP is not compatible with all SSH implementations. We are using > libssh2 in mixed environments and sometimes OpenSSH jsuts isn't an > option. I am not doubting what you say, I am just curious: what environments have ssh and sftp, but n

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-14 Thread Mononen Jussi
>> > The most critical part is file transfer. >> >> You're then referring to SFTP here. But AFAIK, libssh2 is >> also slower on pure >> SCP transfers... Isn't it? Did anyone actually measure >> libssh2 transfers >> recently and compared them to the openssh tools? You are right, it affects sc

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-13 Thread Daniel Stenberg
On Wed, 13 Jun 2007, Mononen Jussi wrote: Thanks for your patience and explanations. > The most critical part is file transfer. You're then referring to SFTP here. But AFAIK, libssh2 is also slower on pure SCP transfers... Isn't it? Did anyone actually measure libssh2 transfers recently and co

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-13 Thread Mononen Jussi
>> Ok thanks, I'm trying to understand this to see what can be >> done about this to >> improve libssh2 performance. >> >> What requests do libssh2 send that it waits for a response >> to that the openssh >> tools don't? I'm looking at the code for a simple SCP >> download case, and I >> do

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-13 Thread Daniel Stenberg
On Wed, 13 Jun 2007, Mononen Jussi wrote: > "Acknowledgment" is a wrong word, sorry. Libssh2 blocks while it waits for > response to a sent request. Command line tools do not have this limitation. > They have their own "stack" implementation where they fire away tens of > requests for data afte

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-13 Thread Mononen Jussi
>> > Also, when comparing to command line tools (like OpenSSH >> sftp), libssh2 is >> > much slower because its approach to sending/receiving is >> synchronous where >> > as the command line tools use asynchronous approach. When >> libssh2 sends >> > onepacket, it can not send another before

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-12 Thread Daniel Stenberg
On Tue, 12 Jun 2007, Mononen Jussi wrote: > Also, when comparing to command line tools (like OpenSSH sftp), libssh2 is > much slower because its approach to sending/receiving is synchronous where > as the command line tools use asynchronous approach. When libssh2 sends > onepacket, it can not s

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-12 Thread Mononen Jussi
>> maximum packet size is just the protocol (SSH2) maximum >> packet size. If packet is larger than this it must be rejected Just to correct myself, it is not a MUST to reject larger packets, but application MUST be able to handle atleast 35000 byte packets. /jUSSi -- perl -e '[EMAIL PROTECTED

Re: [libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-12 Thread Mononen Jussi
>> I am reading/writing to Remote SFTP server using libssh2 >> library. However there is very poor performance compared to >> the sftp utility comes along with UNIX. >> >> I am using libssh2-0.14 version. >> >> I tried to investigate the library and found that for Read >> maximum packet siz

[libssh2] How to increase performance of libssh2 SFTP Read/Write

2007-06-11 Thread Kamal Bhatt
I am reading/writing to Remote SFTP server using libssh2 library. However there is very poor performance compared to the sftp utility comes along with UNIX. I am using *libssh2-0.14 *version. I tried to investigate the library and found that for Read maximum packet size is 4 bytes and for wr