Re: How to tune rsync to speed up?
NFS is slowing things down even more than your bandwidth measurements as it is also forcing --whole-file. On 8/2/23 05:03, Perry Hutchison via rsync wrote: Sebastian G??decke via rsync wrote: We're facing some flapping traffic when rsyncing atm 70T from one server to an DELL Isilon. Both systems are connected with 10G Fiber (not Channel). So we started with one simple "rsync -a /src /dest" to the DELL by using NFS3. ... I always thought (and had observed it so far with rsync) that it makes full use of the network card ... If you are using NFS, it is NFS -- not rsync -- which is managing the network traffic. In such a setup rsync will perform about as well as cp(1). -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Floridak...@sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: How to tune rsync to speed up?
Sebastian G??decke via rsync wrote: > We're facing some flapping traffic when rsyncing atm 70T from > one server to an DELL Isilon. > Both systems are connected with 10G Fiber (not Channel). > So we started with one simple "rsync -a /src /dest" to the DELL > by using NFS3. > ... > I always thought (and had observed it so far with rsync) that it > makes full use of the network card ... If you are using NFS, it is NFS -- not rsync -- which is managing the network traffic. In such a setup rsync will perform about as well as cp(1). -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
How to tune rsync to speed up?
Hi there, we're facing some flapping traffic when rsyncing atm 70T from one server to an DELL Isilon. Both systems are connected with 10G Fiber (not Channel). So we started with one simple "rsync -a /src /dest" to the DELL by using NFS3. So it runs with around 3Gbit for some seconds, than it were 30Kbit for also some seconds and again "some" Gig were transferred and than again it decreases to 50kbit. This was all the time for about 24 hours. Than (this is actually not my doing, it's one of our researchers) i send him a little pythonscript to use more rsyncprocesses. He modified it and now it's around 5Gbps and 10Gbps. So this helps a lot, but now I'm wondering what's the reason, because I always thought (and had observed it so far with rsync) that it makes full use of the network card (if the data is provided accordingly) Some thoughts about that? Here some specs: DELL Server : 7515 with 1T RAM, sfp+ nic, Software: opensuse leap 15.2 , rsync 3.1.3 DELL Isilon: sfp+ connection, OS: 9.5.0.3 Thanks Sebastian -- Mit freundlichen Grüßen Sebastian Gödecke -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
