Just to prevent misunderstandings: I'm not complaining that the performance is less than without encryption. The GridFTP User Guide clearly states that the performance impact of those options is significant. I'm just curious why scp is so much faster, when I see no real difference in the functionality.
Though the performance improvement when using multiple streams is curious. The CPU load of the GridFTP process also increases with more streams. Those are the commands scp -c aes128-cbc bibgrid1lx:/mnt/sw_src/rhel5/isos/* /scratch/ globus-url-copy -r -vb -data-channel-private -p 4 gsiftp://bibgrid1lx.eu.boehringer.com/mnt/sw_src/rhel5/isos/ file:///scratch/ If you want it detailed: the numbers are im MB/sec as reported by globus-url-copy when it has finished the transfer. P is the number of parallel streams. File p = 0 p = 2 p = 4 p = 8 p = 16 default 1 21,39 37,45 41,68 49,36 41,54 2 27,77 28,77 26,30 25,11 25,52 3 24,71 22,68 26,18 21,77 22,22 4 22,96 25,25 22,94 27,46 24,53 5 23,33 30,94 22,70 21,59 18,96 average 24,03 29,02 27,96 29,06 26,55 safe 1 9,13 17,82 18,62 18,24 19,39 2 9,34 16,26 18,89 20,39 19,05 3 9,35 16,22 17,04 20,86 19,20 4 9,66 15,73 17,97 19,56 19,33 5 9,32 15,98 18,01 19,53 20,49 average 9,36 16,40 18,11 19,72 19,49 private 1 3,23 4,80 6,67 8,24 8,41 2 3,26 4,76 6,84 8,43 9,24 3 3,26 4,76 6,87 8,41 8,79 4 3,28 4,78 7,00 8,28 8,85 5 3,25 4,78 7,07 8,34 8,65 average 3,26 4,78 6,89 8,34 8,79 And one scp run: 1 27,40 2 25,50 3 26,30 4 26,80 5 24,30 Avg 26,06 -----Ursprüngliche Nachricht----- Von: John Bresnahan [mailto:[EMAIL PROTECTED] Gesendet: Dienstag, 18. September 2007 16:54 An: Assfalg,Christian (APER) BIP-DE-B Cc: [EMAIL PROTECTED]; [email protected] Betreff: Re: AW: [gt-user] What encryption algorythmus does GridFTP use with -data-channel-private? can you send the command you use to transfer the file? [EMAIL PROTECTED] wrote: > I'm transferring five ISO images, about 600 MB each, 3.1 GB in total. The > numbers are the ones that GridFTP reports when run with the -vb option, so I > guess they do not include startup delays. The same applies for scp. > > -----Ursprüngliche Nachricht----- > Von: Richard Wellner, Jr. [mailto:[EMAIL PROTECTED] > Gesendet: Dienstag, 18. September 2007 16:32 > An: Assfalg,Christian (APER) BIP-DE-B; [email protected] > Betreff: Re: [gt-user] What encryption algorythmus does GridFTP use with > -data-channel-private? > > > How large a file are you transferring? GridFTP takes more time for start-up > and the performance gains become evident only with large files. > > rw2 > > On 9/18/07 2:47 AM, "[EMAIL PROTECTED]" > <[EMAIL PROTECTED]> wrote: > >> Hello! >> >> I've been doing some performance measurements with GridFTP and safe / > secure >> data channels, that is using the -data-channel-safe and > -data-channel-private >> options. >> >> This showed that the performance between two 2.2 Ghz Opteron machines on > the >> same switch topps at about 10 MB/sec for -data-channel-private and -p 2 / > -p >> 4. I am wondering what kind of encryption algorythm and implementation >> GridFTP uses? >> >> Scp reaches transfer rates of up to 35 MB/sec on the same scenario using > the >> aes128-cbc cipher. This is actually quite near the maximum disk IO I > observed >> using iostat, and can only be achieved on subsequent runs. I'm not shure >> wether CPU or disk is actually the bottleneck here. >> >> Regards, >> Christian Aßfalg >
