On Oct 9, 2012, at 10:01 AM, Sill, Alan wrote: > You haven't mentioned the networking conditions under which you are > operating. the transfer time that you mention would be equivalent to an > average transfer speed of 114 Mbytes/sec, or 912 Mbits/sec, for a 120 GByte > file, as implied by the name of the file you cite for the transfer.
Good catch, Alan. > That would be essentially full saturation of a 1 GbE line, for example. > > The parallelism options of gridFTP (which are not part of the free globus > Online service, unless I am mistaken) This is not true. Free Globus Online service use parallelism and other performance optimization parameters in GridFTP as part of its autotuning logic. In addition, it also allows users to play with parallelism, concurrency values etc in the 'transfer' command in CLI. > would not help you exceed the maximum transfer rate possible for your > hardware, but might help, depending on details of network topology, for > longer-distance transfers in which you are not otherwise able to hit network > saturation. > > Magic has not yet been invented, unfortunately (although the folks on the > Globus team are no doubt working on this!) > > Alan > > On Oct 9, 2012, at 9:47 AM, gridftp user <[email protected]> > wrote: > >> >>> Hi Melvin, >>> I recommend that you install Globus Connect >> Multi-User - https://www.globusonline.org/gcmu/ and use Globus Online >> (www.globusonline.org) to do the transfer. >>> >>> In case of >> the transfer you mention below, if you add say '-p 4' or '-p 8' to the >> globus-url-copy command line, you should get much better transfer rate. >> That said, you should really be using Globus Online as it does >> autotuning of the parameters to get good performance. >>> >>> Raj >> >> >> Hi Raj, >> >> >> >> Thank you for taking the time to respond. Adding the -p switch made no real >> difference: >> >> time globus-url-copy -v -p 8 file:/opt/120_GB_file.random >> sshftp://172.22.10.206/home/test/bigfile.random >> >> >> >> real 17m34.074s >> >> user 0m3.116s >> >> sys 1m55.821s >> >> >> I thought I must have managed to miss that in my reading, but that >> information is unfortunately missing >> from all of the on-line documentation that I've read over the past week. In >> fact, even knowing what to look >> for I am unable to find it mentioned in any site documentation at all. I did >> find it by typing globus-url-copy >> -help, which gives me some reading that I've not gone through yet. >> >> Any chance the lack of improvement is due to the fact that the traffic is >> going out one server, through a >> single switch, and back into the receiving server? I ask because I am not a >> network engineer by any means >> and I am trying to get an understanding of what is going on. If the >> infrastructure is an issue then an outside >> test is in order. >> >> I appreciate the suggestion of using Globus Online. We looked into that, but >> we do not want our >> data residing on systems we do not control, even if it is there for a short >> time. That is why I am >> trying so hard to set up a server/client system, so we can have almost >> complete control over both >> end-points and transfer parameters. >> >> I'll keep plugging away. >> >> Melvin >> >> >
