Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Les Mikesell
On Thu, Jan 29, 2015 at 7:05 AM, Joseph L. Brunner j...@affirmedsystems.com wrote: our investigation showed the rsync process even with all switches we found has to open the file a bit before it copies it... so rsync sucks for this kind of stuff with 2 MILLION small files - it never gets

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Eero Volotinen
2015-01-23 11:44 GMT+02:00 Götz Reinicke - IT Koordinator goetz.reini...@filmakademie.de: Hi, I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s The options I use are: rsync -aHAXxv --numeric-ids --progress -e ssh -T -c arcfour -o

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Joseph L. Brunner
...@centos.org] On Behalf Of Gordon Messmer Sent: Wednesday, January 28, 2015 06:40 PM To: CentOS mailing list Subject: Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions? On 01/23/2015 01:44 AM, Götz Reinicke - IT Koordinator wrote: I do have two centos 6.6 servers

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Marcelo Ricardo Leitner
On 28-01-2015 21:39, Gordon Messmer wrote: On 01/23/2015 01:44 AM, Götz Reinicke - IT Koordinator wrote: I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s That *is* pretty slow for sustained writes. Does the same rate hold true for individual

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Gordon Messmer
On 01/23/2015 11:10 AM, Valeri Galtsev wrote: I'm not certain what the problem could be. But enabling jumbo packets would be the fist thing I would try Jumbo frames are an excellent option if your system is spending a lot of time processing interrupts, which is why I asked about the hi value

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Gordon Messmer
On 01/23/2015 10:54 AM, Patrick Flaherty wrote: Not an expert in rsync/ssh, but I'm pretty sure it's ssh's tcp window size that is the slowness. Probably not? TCP window size is usually only a problem if the sender is capable of sending more data than the window size before the receiver

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Gordon Messmer
On 01/29/2015 09:09 AM, Les Mikesell wrote: our investigation showed the rsync process even with all switches we found has to open the file a bit before it copies it Rsync is going to read the directory tree first, then walk it on both sides comparing timestamps (for incrementals) and block

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-29 Thread Gordon Messmer
On 01/28/2015 09:47 PM, Charles Polisher wrote: Add this parameter: --bwlimit=0 That's the default, so it's unlikely to change anything. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-28 Thread Gordon Messmer
On 01/23/2015 01:44 AM, Götz Reinicke - IT Koordinator wrote: I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s That *is* pretty slow for sustained writes. Does the same rate hold true for individual large files as it does for lots of small

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-28 Thread Charles Polisher
On Wed, Jan 28, 2015 at 03:39:50PM -0800, Gordon Messmer wrote: On 01/23/2015 01:44 AM, Götz Reinicke - IT Koordinator wrote: I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s Add this parameter: --bwlimit=0

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-23 Thread Patrick Flaherty
I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s The options I use are: rsync -aHAXxv --numeric-ids --progress -e ssh -T -c arcfour -o Compression=no -x If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd (r/w) on the

Re: [CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-23 Thread Valeri Galtsev
On Fri, January 23, 2015 12:54 pm, Patrick Flaherty wrote: I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s The options I use are: rsync -aHAXxv --numeric-ids --progress -e ssh -T -c arcfour -o Compression=no -x If I copy files by smb

[CentOS] network copy performance is poor (rsync) - debugging suggestions?

2015-01-23 Thread Götz Reinicke - IT Koordinator
Hi, I do have two centos 6.6 servers. With a performance optimized rsync I get an speed of 15 - 20 MB/s The options I use are: rsync -aHAXxv --numeric-ids --progress -e ssh -T -c arcfour -o Compression=no -x If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd (r/w) on the