On Mon, Feb 1, 2010 at 2:52 AM, Roman Yakovenko <roman.yakove...@gmail.com> wrote: > Hello. > > I am using SCPClient class from branch( > http://bazaar.launchpad.net/~jbardin/paramiko/paramiko_scp/annotate/500?file_id=scp.py-20081117202350-5q0ozjv6zz9ww66y-1 > ) with paramiko 1.7.6 and Python 2.6 on Ubuntu "Karmic Koala". > > I am testing my code with file size 1 GB. > > The SCPClient upload rate starts with 10 MB/s and than drops to 5.2 > MB/s. The average is 5.2 MB/s. I tried to change buffer size, but this > didn't help > The scp command upload rate starts with 20 MB/s and then drops to 10 > MB/s. The average is 10 MB/s. > To complete the statistics, paramiko built-in SFTPClient average rate > is 2.2 MB. I use "put" method as is, with the default configuration. > > I am not sure where to start to solve the problem. Initially, I > suspected that "local file reading" is a problem, but that > functionality works pretty well.
You would normally start by using a profiler to see where the performance bottleneck is, before you start speculating. You would have seen that most of the time is spent in paramiko.Transport manipulating data, and waiting for pyCrypto. SCPClient adds almost nothing to the overall time. > Right now, I am using work around ( executing scp with subprocess ) > but it is less than optimal solution. > > Any help is appreciated. > Yes, the solution written entirely in c will be significantly faster. Since this is mostly python, cpu is the limiting factor. There may be some places where optimizations could be made in paramiko and pyCrypto, but I haven't looked into it myself. -jim _______________________________________________ paramiko mailing list paramiko@lag.net http://www.lag.net/cgi-bin/mailman/listinfo/paramiko
