Re: Aw: Re: rsync very very slow with multiple instances at the same time.

2018-03-23 Thread Kevin Korb via rsync
storage is'n ssd, is it? there are even more, iscsi is introducing > additional latencies... >   > regards > roland >   >   > *Gesendet:* Freitag, 23. März 2018 um 17:52 Uhr > *Von:* "Jayce Piel via rsync" <rsync@lists.samba.org> > *An:* "Kevin Korb via rsync

Aw: Re: rsync very very slow with multiple instances at the same time.

2018-03-23 Thread devzero--- via rsync
encies...   regards roland     Gesendet: Freitag, 23. März 2018 um 17:52 Uhr Von: "Jayce Piel via rsync" <rsync@lists.samba.org> An: "Kevin Korb via rsync" <rsync@lists.samba.org> Betreff: Re: rsync very very slow with multiple instances at the same time. Ok, so i

Re: rsync very very slow with multiple instances at the same time.

2018-03-23 Thread Jayce Piel via rsync
Ok, so i did some tests. find /path -type f -ls > /dev/null First on my local SSD disk (1.9 millions files) : 1 find : real2m16.743s user0m7.607s sys 0m45.952s 10 concurrent finds (approx same results for each) : real4m48.629s user0m11.013s sys 2m0.288s Almost double

Aw: rsync very very slow with multiple instances at the same time.

2018-03-21 Thread devzero--- via rsync
: Mittwoch, 21. März 2018 um 14:49 Uhr Von: "Jayce Piel via rsync" <rsync@lists.samba.org> An: rsync@lists.samba.org Betreff: rsync very very slow with multiple instances at the same time. I create a new thread, because the issue is not really the same, but i copy here the thread t

Re: rsync very very slow with multiple instances at the same time.

2018-03-21 Thread Kevin Korb via rsync
When rsync has a lot of files to look through but not many to actually transfer most of the work will be gathering information from the stat() function call. You can simulate just the stat call with: find /path -type f -ls > /dev/null You can run one then a few of those to see if your storage has

rsync very very slow with multiple instances at the same time.

2018-03-21 Thread Jayce Piel via rsync
I create a new thread, because the issue is not really the same, but i copy here the thread that made me jump into the list. My issue is not really that it waits before starting copying, but a general performance issue, specially when there are multiple rsync running at the same time. Here is