UDT ( https://en.wikipedia.org/wiki/UDP-based_Data_Transfer_Protocol ) was made exactly for this purpose, but I'm not familiar with how well it works as a standalone tool.
Am Donnerstag, 13. September 2018 22:15:23 UTC+2 schrieb Jay Askren: > > Todd Montgomvery, > > Correct, robocopy uses TCP. We have a 10 Gbps terrestrial line and are > working on getting a second line for redundancy. > > > Thomas, > Thanks for the link. I will look at that page. > > Jay > > > > On Thursday, September 13, 2018 at 11:04:54 AM UTC-6, Todd L. Montgomery > wrote: >> >> Hi Jay. >> >> Going to assume Robocopy uses TCP.... >> >> As you had no real issues with things without a WAN, I would assume the >> TCP window sizes, etc. are all good for the rates you need. >> >> Latency will play a role, but more likely loss is a more impactful factor >> as congestion control will be more of a throttle than flow control. With >> TCP (low loss rate), RTT scales linearly with throughput. Well, as RTT goes >> up, throughput goes down, but it is linear. With loss, even low loss, >> throughput scales with sqrt(loss rate). After about 5%, TCP-Reno goes into >> stop-and-wait. 1 MSS per RTT. This scale is non-linear and in the < 5% loss >> rate area is really painful on throughput. >> >> In short, WANs will slow down with loss quite a lot. Latency will also >> have an impact, though. Just not as much potential. >> >> Running multiple TCP connections over the same path will mean that they >> will fight with one another via congestion control trying to find a >> fairness point that jumps around and can end up underutilizing the >> bandwidth at times. This is where things like TCP BBR can be helpful. But >> still, loss will cause quite a slow down. >> >> What can you do? Well, it depends on what your links between the areas >> actually are..... terrestrial vs. satellite, etc. Lots of options. >> >> On Thu, Sep 13, 2018 at 9:41 AM Jay Askren <[email protected]> wrote: >> >>> We need to push 40 TB of images per day from our scanning department in >>> Utah to our storage servers in Virginia and then we download about 4 TB of >>> processed images per day back to Utah. In our previous process we had no >>> problem getting the throughput we needed by using Robocopy which comes with >>> Windows, but our old storage servers were here in Utah. We can get >>> Robocopy to work across the WAN but we have to run 3 or 4 Robocopy >>> processes under different Windows users which is somewhat fragile and feels >>> like a bad hack. The files here in Utah are on a Windows server because of >>> the proprietary software needed to run the scanner. All of our servers in >>> Virginia run Centos. >>> >>> Any thoughts on how to transfer files over long distance and still get >>> high throughput? I believe the issue we are running into is high latency. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "mechanical-sympathy" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- You received this message because you are subscribed to the Google Groups "mechanical-sympathy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
