Don, Our experience is parallel streams help TCP significantly for higher RTT (>=20ms) links but not UDT so much (as it gets most of the available bandwidth with single stream). As you say, parallel streams does not help lots of small files data sets. It looks like you are happy with the current UDT performance. Most likely, concurrency (-cc in globus-url-copy) of 2 or 4 will further improve the speed for these data sets.
Raj On Aug 13, 2010, at 2:53 PM, Don Gilbert wrote: > > Raj, > > We tested only one stream for each. I am assuming from what I read of your > reports > that we can speed up both TCP and UDT with multiple streams. Given the many > separate files of bio-data sets, it doesn't matter much how this is done. > Current > bio-mirror.net customers often have 10 simultaneous FTP fetches running, and > that is still a minor load on the system. It however still doesn't get all > the data from here to Australia or China before the data is updated here > (daily updates for some of popular bio-data sets). > > - Don
