On Mon, Apr 11, 2016 at 1:34 PM, Marco Atzeri <marco.atz...@gmail.com> wrote: > > > On 11/04/2016 18:57, Greg Freemyer wrote: >> >> On Thu, Mar 31, 2016 at 4:48 PM, Greg Freemyer <greg.freem...@gmail.com> >> wrote: >>> >>> All, >>> >>> I'm not sure this is on-topic, but at least I'm in cygwin when I >>> notice the below: >>> >>> I do a lot of large data transfers between USB drives. Often I get >>> great speeds (70MB/sec or more). >>> >>> Sometimes it falls way off to closer to 20MB/sec with the same class >>> of hardware. >>> >>> I experienced the "slow" transfer speed today, so I thought I'd ask if >>> anyone knew a way to resolve it? >> >> >> I'm trying rsync of a bunch of large files again today. >> >> Initially I saw the same slow copy speed (about 20 MB/sec). My >> destination was connected to a USB 3.1 port, but it seemed to be the >> bottleneck so I moved it to a USB 3 port (3.1 should be 2x the speed >> of 3.0, so it should not have been the bottleneck). >> >> After getting my throughput up above 50 MB/sec I notice in the windows >> "performance monitor" that my source disk is hitting 100% utilization, >> then a few seconds later my destination disk is. And back and forth. >> It seems I'm only reading or writing for a few seconds, then >> alternating. >> >> I assume the issue is that too much data is being read / cached by >> rsync prior to it being written out so I'm getting no advantage of >> reading and writing in parallel. >> >> (I gather iostat isn't available for cygwin?) >> >> Is there some rsync (or cygwin) option / feature that would encourage >> parallel reading/writing? >> >> fyi: I did some linux testing with "dd" over the weekend and I hit 140 >> MB/sec if I used a 100MB blocksize. That was to / from the raw disk >> (/dev/sda => /dev/sdb). >> >> If I bumped my blocksize to 1GB for dd in linux, my throughput dropped >> to 70 MB/sec just as I see right now with rsync in cygwin. >> >> Thanks >> Greg >> > > > Is windows robocopy faster ?
As I test, I just copied 30 GB of 1.5 GB files via robocopy. None of the files had been accessed since a reboot, so none should have been in cache. According to Resource Monitor, yes. I'm getting about 105 MB/sec for read and 105 MB/sec for write vs 70 MB/sec for rsync. Significantly, with robocopy Resource Monitor is showing a consistent read/write speed. With rsync read and write fluctuate back and forth. As I stated before, I believe I should be able to get 140 MB/sec, but I wouldn't complain about 105 MB/sec. 70 MB/sec just seems to highlight a flaw in how rsync manages the data flow since it is 50% of theoretical max. ie. this seems to be what rsync in cygwin is doing: while (files) { read 1.5 GB file to ram write 1.5 GB file from ram fsync() ensure 1.5 GB file is on disk } endwhile I haven't tested in linux. Maybe rsync just isn't as efficient as I expect? There is also the problem that rsync often slows down to below 50 MB/sec I've tried it with no other activity on the PC. Today I was only getting 20 MB/sec at first. (I moved the USB cable and it came up to 70 MB/sec). Thanks Greg -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple