This message is being posted on both gt-users and gridftp-users. We're having problems using the "restart" file transfer feature of the globus-url-copy utility. It seems to work with offsets less than 2 Gig, but does not work properly when the offset exceeds 2 Gig. We're wondering if this is a known issue, or if there's something in our building of system is not correct.
Details of our configuration, and usage of globus-url-copy: OS is 32-bit Fedora 9 Version 4.2.1 of the Globus toolkit. When I transfer using an offset and a length with globus-url-copy everything is good until the offset is >2GB, after than the offsets don't seem to be working correctly. When I transfer a file (much larger than 2 Gig) without using offsets it moves flawlessly any size. Compiled with ./configure with a -prefix but no other options. Fedora 9 developer work station install. gcc 4.0.3 Options on the command line: globus-url-copy -q -fast -verbose-perf -p4 -nodcau -ds (string) file://localfile gsiftp://remotemachine/directory In addition, the -off and -len are used to resume the file transfer at a known position. A replication case is a 3500MB file (filled with /dev/urandom characters) Source file is local, and destination across a local LAN. Three separate invocations of globus-url-copy using the following offsets and lengths: -off 0 -len 1000000000 -off 1000000000 -len 1000000000 -off 2000000000 -len 1000000000 These work correctly: after the transfer completes, the file size on the receiving end is correct and a checksum of the file contents matches that on the sending end. Then I set the offset and length to the rest of the file: -off 3000000000 -len 145728000 After the transfer completes, the file size on the receiving end is 3000000000 not the 3145728000 that was expected. No errors where reported and everything looks normal as far as the client is concerned. Likewise the receiving server does not report errors. When I use the -dbg switch on the client all of the offsets in the blocks look correct from the sending side. When I tried the older 4.0.8, the behavior was the same up until the offset 3000000000 and length 145728000, there the server reported that it was eof and the connection had an error no data is transferred. Other things I have tried: If the offset is -off 2000000000 -len 1145728000 everything transfers successfully, the size is correct and the data checksums correctly. Running the transfer as a get instead of a put, so pulling the data down to the server, the behavior is the same 1000MB, 2000MB, and up to 3000MB are correct, then the last transfer take place and the file is suddenly 7440695296 or about 4GB larger than it should be. In -dbg all of the offsets are in the 7440695296 range. I have also tried to force some CFLAGS on the ./configure: CFLAGS='-D_GNU_SOURCE' CFLAGS='-D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' CFLAGS='-D_FILE_OFFSET_BITS=64' All of these resulted in either no change in behavior or errors right away complaining about eof being reached. The best seems to be letting ./configure just do the magic. I'm stuck, it really seems like a 32 bit seek is taking place to start the transfer on the server side. Any help on this issue would be greatly appreciated! Kevin Mahony | Software Engineer Savvis 10900 Hampshire Ave S, Suite 150, Bloomington, MN 55438 Office: 952.852.4804| Fax: 952.852.4951 E-mail: [email protected] This message contains information which may be confidential and/or privileged. Unless you are the intended recipient (or authorized to receive for the intended recipient), you may not read, use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail and delete the message and any attachment(s) thereto without retaining any copies.
