My apologies, I sent you an earlier version of the diagram that showed only three servers (total) running. I actually had six going. However, my port assignments were wrong per your example and the syntax of my copy command was incorrect as well. Now, with your support, the basic transfer mechanism is running and data seems to be going down both wires. My objective now is to fill both 10GigE pipes on route to trying to move data in excess of 2000MB/sec. Any tuning hints at this point would be appreciated. Then looking ahead, we're putting the pieces in place for our 40GigE disk-to-disk test from GSFC (Greenbelt, MD) to the SC10 floor in New Orleans which leads to another question, "how is gridFTP impacted by increasing rtts?".
Thanks again for all the help, Hoot '-----Original Message----- From: Raj Kettimuthu <[email protected]> To: Hoot Thompson <[email protected]> Cc: [email protected] Subject: Re: Call Date: Mon, 13 Sep 2010 17:27:07 -0500 Hoot, Right now, you have GridFTP backends running only on i7test4. You need 2 backends on i7test3 and another frontend process. You need a total of 6 server processes (4 backends and 2 frontends). Here is what you will run on i7test4 globus-gridftp-server -data-interface i7test4-10g1 -S -p 7000 -dn globus-gridftp-server -data-interface i7test4-10g2 -S -p 8000 -dn globus-gridftp-server -p 6000 -S -r i7test4:7000, i7test4:8000 Here is what you will run on i7test3 globus-gridftp-server -data-interface i7test3-10g1 -S -p 7000 -dn globus-gridftp-server -data-interface i7test3-10g2 -S -p 8000 -dn globus-gridftp-server -p 6000 -S -r i7test3:7000, i7test3:8000 Run the client as follows: globus-url-copy -vb -stripe gsiftp://i7test3:6000/i7raid/hoot/file_12GB gsiftp://i7test4:6000/i7raid/hoot/file_12GB Raj On Sep 13, 2010, at 5:05 PM, Hoot Thompson wrote: > Raj et al, > > I'm still missing something. Attached is a diagram of what I have set > up - two servers interconnected with two 10GigE links. I want to pass > one file, striped across both links from one server to the other, in > this case from i7test4 to i7test3 and/or i7test3 to i7test4. Shown on > the attachment are the three gridftp-servers I have running on each > server. Are they correct? Does the command line you gave below still > apply? > > Hoot > > > -----Original Message----- > From: Raj Kettimuthu <[email protected]> > To: Hoot Thompson <[email protected]> > Subject: Re: Call > Date: Thu, 9 Sep 2010 17:04:56 -0500 > > Hoot, Sorry for the slow response. You have to use the following > command-line: $GLOBUS_LOCATION/bin/globus-url-copy -vb -stripe > gsiftp://i7test3-10g1:6000/i7raid/hoot/file_12GB > gsiftp://i7test3-10g2:6000/i7raid/hoot/file_12GB > > The data will flow between the 2 backends at each end and both the > links will be used. Striping also requires that the all source > backends can read the source file and all the destination backends can > write to the destination file i.e, backends at each end have a shared > filesystem. > > The tutorial slides available > at http://www.globus.org/toolkit/data/gridftp/tutorials/ has more > information on striping. > > Please post your questions to gridftp-user list so that you do not > block on me. > > Raj > > > > On Sep 7, 2010, at 3:39 PM, Hoot Thompson wrote: > > > ok, I'm getting close but admittedly I've reached a point of > > confusion. In trying to walk before I run, I have a triangle > > configured of sort where my frontend node calls out the two NIC > > cards in the other server using the -r option. > > > > Here's the frontend... > > sudo -E /home/globus/globus/sbin/globus-gridftp-server -S -p 6000 -r > > 192.168.1.13:7000,192.168.2.13:8000 -logfile /tmp/grid -log-level > > 'ALL' > > > > On the backend I have two servers started.... > > sudo -E /home/globus/globus/sbin/globus-gridftp-server -S -p 7000 > > -dn -logfile /tmp/grid -log-level 'ALL' > > > > sudo -E /home/globus/globus/sbin/globus-gridftp-server -S -p 8000 > > -dn -logfile /tmp/grid -log-level 'ALL' > > > > > > So now when I issue the following (logged into the frontend which > > is .1.12(10g1) and .2.12(10g2)) > > > > $GLOBUS_LOCATION/bin/globus-url-copy -vb -stripe > > gsiftp://i7test3-10g1:6000/i7raid/hoot/file_12GB /i7raid/hoot/file_12GB > > -----data travels over the .1.x subnet > > > > when I issue > > $GLOBUS_LOCATION/bin/globus-url-copy -vb -stripe > > gsiftp://i7test3-10g2:6000/i7raid/hoot/file_12GB /i7raid/hoot/file_12GB > > ------data travels over the .2.x subnet > > > > I assume this makes sense. My confusion is when I bring up a second > > gridftp server on the frontend, what does my globus-url-copy command > > look like to pull data over both links simultanesouly? > > > > BTW, performance is great for the one link, > > > > 945.64 MB/sec avg 1179.98 MB/sec inst > > > > > > -----Original Message----- > > From: Raj Kettimuthu <[email protected]> > > To: Hoot Thompson <[email protected]> > > Subject: Re: Call > > Date: Fri, 3 Sep 2010 16:15:24 -0500 > > > > You have to run 3 instances of gridftp server on each side. > > > > On the sending you run 2 data nodes (with -dn), one each on 1.12 and > > 2.12 and run 1 control node (with -r or remote_nodes 1.12, 2.12) on > > either 1.12 or 2.12 or you can designate another machine for this > > > > On the receiving you do the same - run 2 data nodes (with -dn), one > > each on 1.13 and 2.13 and run 1 control node (with -r or > > remote_nodes 1.13, 2.13) on either 1.13 or 2.13 or you can designate > > another machine for this > > > > Then you run 'globus-url-copy -stripe <source control node> <dest > > control node>' > > > > Hope this helps. > > > > Raj > > On Sep 3, 2010, at 4:06 PM, Hoot Thompson wrote: > > > > > So on the send side I can put something like the following in > > > the .conf file... > > > > > > remote_nodes 192.16.1.13:6000,192.168.2.13:7000 > > > > > > But on the receive side, how do you specify multiple data_node > > > interfaces on one machine? > > > > > > > > > Hoot > > > > > > > > > __________________________________________________________________ > > > > > > > > > From: Raj Kettimuthu [mailto:[email protected]] > > > Sent: Friday, September 03, 2010 4:39 PM > > > To: Hoot Thompson > > > Subject: Re: Call > > > > > > > > > > > > Hoot, > > > The link below should have the information you are looking for: > > > http://www.globus.org/toolkit/docs/latest-stable/data/gridftp/admin/#gridftp-admin-striped > > > If something is not clear, please feel free to contact me. > > > > > > > > > Raj > > > > > > On Sep 3, 2010, at 3:33 PM, Hoot Thompson wrote: > > > > > > > > > > Maybe you'd have a moment to offer a little advice and some of > > > > this will be repetitive. I currently have two servers, each > > > > with two 10GigE NICs. > > > > > > > > i7test3 > > > > 192.168.1.12 > > > > 192.168.2.12 > > > > > > > > i7test4 > > > > 192.168.1.13 > > > > 192.168.2.13 > > > > > > > > I want to stripe a single large file over the two links. What > > > > goes in the in the gridftp.conf files on both ends to make this > > > > happen? Do any command line options need to be invoked when I > > > > initiate the the url to url copy? > > > > > > > > > > > > Thanks, > > > > > > > > hoot > > > > > > > > > > > > ________________________________________________________________ > > > > > > > > > > > > From: Raj Kettimuthu [mailto:[email protected]] > > > > Sent: Friday, September 03, 2010 4:12 PM > > > > To: Hoot Thompson > > > > Subject: Re: Call > > > > > > > > > > > > > > > > Good. I just started replying to your email on the > > > > server-to-server error. Have a good long weekend. > > > > > > > > > > > > Raj > > > > > > > > On Sep 3, 2010, at 3:09 PM, Hoot Thompson wrote: > > > > > > > > > > > > > Got server to server working :-) > > > > > > > > > > Have a good long weekend. > > > > > > > > > > Hoot > > > > > > > > > > > > > > > ______________________________________________________________ > > > > > > > > > > > > > > > From: Raj Kettimuthu [mailto:[email protected]] > > > > > Sent: Thursday, September 02, 2010 4:20 PM > > > > > To: Hoot Thompson > > > > > Subject: Re: Call > > > > > > > > > > > > > > > > > > > > toll-free telecon is 877-950-6599 with passcode 6356785 > > > > > > > > > > On Sep 2, 2010, at 3:19 PM, Hoot Thompson wrote: > > > > > > > > > > > > > > > > Yes but I didn't get the dial in number > > > > > > > > > > > > -----Original Message----- > > > > > > From: Raj Kettimuthu [mailto:[email protected]] > > > > > > Sent: Thursday, September 02, 2010 4:16 PM > > > > > > To: Hoot Thompson > > > > > > Subject: Call > > > > > > > > > > > > Pat and I are on the call. Are you going to join? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > <test_layout.pdf>
