Joe, Thank you for the admin guide. If I understand correctly, our collaborators should set env. variable $GLOBUS_TCP_PORT_RANGE 50000,50099 on their end ? Their firewall does not block incoming traffic on these ports.
Even if they set the env. variable, the command "globus-job-run transpgrid.pppl.gov /bin/date" just hangs. I was able to look at the logs on our firewall and it appears that the requests are coming on ports in the 30000-45000 range (not on the range we allow traffic). Also, when they run "globus-job-submit ....." the command executes, but it takes 4 times longer now (our new server is much faster than the previous). What do you think is the problem? Thank you, Irena On Mon, Mar 14, 2016 at 3:50 PM, Joseph Bester <bes...@mcs.anl.gov> wrote: > This part of the admin guide has information about how to set up the > various GT components to work with a firewall port range: > > http://toolkit.globus.org/toolkit/docs/6.0/admin/install/#gt-admin-firewall > > > Joe > > > On Mar 14, 2016, at 3:47 PM, Irena Johnson <ijohn...@pppl.gov> wrote: > > > > It seems that our collaborator is not able to run globus-job-submit (or > globus-job-run) successfully, the command just hangs. > > > > When I run "globus-job-run transpgrid.pppl.gov /bin/date" from my > desktop, I get the output after some delay. However, on the server the job > doesn't seem to complete (see below). > > > > 40491 7362 1 0 14:55 ? 00:00:00 globus-job-manager -conf > /etc/globus/globus-gram-job-manager.conf -type fork -seg-module fork > > > > > > So my question is why globus-job-submit or globus-job-run hang? Are > there any ports that need to be opened so that someone from outside our > network would be able to submit "globus-job-run" or "globus-job-submit" > successfully? > > > > /var/log/globus/gram_*.log > > > > ts=2016-03-14T17:47:16.183966Z id=30243 event=gram.job.end level=ERROR > gramid=/16505988311912234441/18179527716261904531/ job_status=4 status=-73 > reason="the job manager failed to open stdout" > > ts=2016-03-14T17:48:14.051020Z id=30243 event=gram.script_idle.info > level=DEBUG msg="closing idle script handle after 58.0 seconds > > ts=2016-03-14T17:48:16.184372Z id=30243 event=gram.end level=DEBUG > > > > ts=2016-03-14T18:55:45.516392Z id=6403 event=gram.script_read.start > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ result=0 > nbytes=56 > > ts=2016-03-14T18:55:45.516545Z id=6403 event=gram.script.log level=DEBUG > msg="filestageout staging failed with " > > ts=2016-03-14T18:55:45.516662Z id=6403 event=gram.script_read.start > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ result=0 > nbytes=295 > > ts=2016-03-14T18:55:45.516969Z id=6403 event=gram.script_read.start > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ result=0 > nbytes=22 > > ts=2016-03-14T18:55:45.517039Z id=6403 event=gram.job.info level=DEBUG > gramid=/16505999305245686836/18179527716261915071/ job_status=4 > > ts=2016-03-14T18:55:45.517120Z id=6403 event=gram.script_read.start > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ result=0 > nbytes=1 > > ts=2016-03-14T18:55:45.517206Z id=6403 event=gram.job.info level=DEBUG > gramid=/16505999305245686836/18179527716261915071/ job_status=4 > > ts=2016-03-14T18:55:45.517261Z id=6403 event=gram.callback.start > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ state=4 > restart_when_done=false > > ts=2016-03-14T18:55:45.518783Z id=6403 event=gram.callback.end > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ state=4 > status=0 msg="Done queuing callback messages" > > ts=2016-03-14T18:55:45.699160Z id=6403 event=gram.directory_destroy.end > level=DEBUG gramid=/16505999305245686836/18179527716261915071/ > path="/u/tr_nhoward/.globus/tr_nhoward/.globus/job/transpgrid2/16505999305245686836.18179527716261915071" > failures=0 status=0 > > ts=2016-03-14T18:55:45.743314Z id=6403 event=gram.job.end level=ERROR > gramid=/16505999305245686836/18179527716261915071/ job_status=4 status=-73 > reason="the job manager failed to open stdout" > > ts=2016-03-14T18:55:52.105262Z id=6403 > event=gram.add_reference_by_jobid.end level=DEBUG > jobid="5ed6bf0a-ea16-11e5-91d3-288023a84afc:7395" status=-156 msg="Unknown > job ID" reason="the job contact string does not match any which the job > manager is handling" > > ts=2016-03-14T18:55:52.105382Z id=6403 > event=gram.add_reference_by_jobid.end level=DEBUG > jobid="5ed6bf0a-ea16-11e5-91d3-288023a84afc:7395" status=-156 msg="Unknown > job ID" reason="the job contact string does not match any which the job > manager is handling" > > > > > > Thank you! > > Irena > > > > On Mon, Mar 14, 2016 at 2:58 PM, Irena Johnson <ijohn...@pppl.gov> > wrote: > > Raj, > > > > This time globus-url-copy worked (see output below). However, they are > now not able to run "globus-job-run transpgrid.pppl.gov /bin/date". > Command just hangs. > > > > > > 227 Entering Passive Mode (192,55,106,199,195,88) > > > > debug: sending command to > > gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > > ALLO 348 > > > > debug: response from > > gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > > 200 ALLO command successful. > > > > debug: sending command to > > gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > > STOR ~/incoming/281948_CMOD.REQUEST > > > > debug: response from > > gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > > 150 Beginning transfer. > > > > debug: writing buffer 0x7ffaba8c0010, length 348, offset=0, eof=true > > debug: data callback, no error, buffer 0x7ffaba8c0010, length 348, > > offset=0, eof=true > > debug: response from > > gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > > 226 Transfer Complete. > > > > debug: operation complete > > > > > > On Mon, Mar 14, 2016 at 2:27 PM, Raj Kettimuthu <ketti...@anl.gov> > wrote: > > From the 227 response to PASV command below, the server is listening on > port 38198 (149*256 + 54) for the data channel connection. It means that > GLOBUS_TCP_PORT_RANGE is not set as intended. Try adding ‘port_range > 50000,50099’ in gridftp.conf > > > > On Mar 14, 2016, at 1:21 PM, Irena Johnson <ijohn...@pppl.gov> wrote: > > > >> Here is the output. > >> > >> > >> 227 Entering Passive Mode (192,55,106,199,149,54) > >> > >> debug: sending command to > >> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >> ALLO 348 > >> > >> debug: response from > >> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >> 200 ALLO command successful. > >> > >> debug: sending command to > >> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >> STOR ~/incoming/281948_CMOD.REQUEST > >> > >> > >> I am thinking something is not configured correctly on my server. I > have xinetd running (with /etc/xinetd.d/gridftp, which should run gsiftp) > as well as globus-gridftp-server. > >> If I stop the service globus-gridftp-server, I am getting this error: > "error: globus_xio: Unable to connect to transpgrid.pppl.gov:2811", and > nothing is listening on port 2811. > >> > >> > >> If I start the service globus-gridftp-server, at least I can see that > the server is listening on the port: > >> > >> # netstat -tupln |grep 2811 > >> tcp 0 0 :::2811 :::* > LISTEN 1975/globus-gridftp > >> > >> > >> On Mon, Mar 14, 2016 at 2:04 PM, Raj Kettimuthu <ketti...@anl.gov> > wrote: > >> Can you send the output of globus-url-copy -dbg now? Are you sure the > ports 50000-50099 are open? > >> > >> On Mar 14, 2016, at 1:00 PM, Irena Johnson <ijohn...@pppl.gov> wrote: > >> > >>> Hello Raj, > >>> > >>> Our collaborator is still not able to use globus-url-copy (command > still hangs). > >>> > >>> Do I need to set GLOBUS_TCP_PORT_RANGE in /etc/gridftp.conf (the range > is set in /etc/xinetd.d/gridftp) ? > >>> > >>> > >>> # more /etc/gridftp.conf > >>> # globus-gridftp-server configuration file > >>> > >>> # this is a comment > >>> > >>> # option names beginning with '$' will be set as environment > variables, e.g. > >>> # $GLOBUS_ERROR_VERBOSE 1 > >>> # $GLOBUS_TCP_PORT_RANGE 50000,51000 > >>> > >>> # port > >>> port 2811 > >>> log_level ERROR,WARN,INFO > >>> log_single /var/log/gridftp-auth.log > >>> log_transfer /var/log/gridftp.log > >>> > >>> > >>> Thanks, > >>> Irena > >>> > >>> > >>> On Mon, Mar 14, 2016 at 12:44 PM, Irena Johnson <ijohn...@pppl.gov> > wrote: > >>> I was missing the line "env = GLOBUS_TCP_PORT_RANGE=50000,50099" in > /etc/xinetd.d/gridftp > >>> > >>> # cat /etc/xinetd.d/gridftp > >>> service gsiftp > >>> { > >>> instances = 100 > >>> socket_type = stream > >>> wait = no > >>> user = root > >>> env = GLOBUS_TCP_PORT_RANGE=50000,50099 > >>> server = /usr/sbin/globus-gridftp-server > >>> server_args = -i > >>> log_on_success += DURATION > >>> nice = 10 > >>> disable = yes > >>> } > >>> > >>> I restarted globus-gridftp-server -- and will try again. > >>> > >>> > >>> My hosts file is below: > >>> # cat /etc/hosts > >>> 127.0.0.1 localhost > >>> > >>> > >>> Thank you! > >>> > >>> On Mon, Mar 14, 2016 at 12:41 PM, José Luis Gordillo Ruiz < > j...@super.unam.mx> wrote: > >>> Hi all, > >>> > >>> I recently got into a similar problem. It’s a hostname resolution > issue. > >>> see the thread for > https://lists.globus.org/mailman/htdig/gt-user/2007-May/003429.html > >>> > >>> saludos, > >>> > >>> José Luis Gordillo Ruiz > >>> Coordinación de Supercómputo > >>> DGTIC - UNAM > >>> > >>> > >>> > >>>> El 14/03/2016, a las 9:57 a.m., Raj Kettimuthu <ketti...@anl.gov> > escribió: > >>>> > >>>> Looks like a firewall issue to me. Do you have a set of ports open on > the server for data channel connections? If you have done that already, you > have to set that port range either in the server command line using > ‘-port-range startport,endport’ or set set the environment variable > GLOBUS_TCP_PORT_RANGE (GLOBUS_TCP_PORT_RANGE=startport,endport). > >>>> > >>>> On Mar 14, 2016, at 10:46 AM, Irena Johnson <ijohn...@pppl.gov> > wrote: > >>>> > >>>>> Dear Globus User Support, > >>>>> > >>>>> Our collaborator is having problem copying a file via command > "globus-url-copy". > >>>>> Please see below. > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> It hangs up after this: > >>>>> > >>>>> [nthoward@cmodws87 ~]$ globus-url-copy -v -dbg -nodcau > >>>>> file:///home/nthoward/281948_CMOD.REQUEST > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST > >>>>> Source: file:///home/nthoward/ > >>>>> Dest: gsiftp://transpgrid.pppl.gov/~/incoming/ > >>>>> 281948_CMOD.REQUEST > >>>>> debug: starting to put > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST > >>>>> debug: connecting to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 220 transpgrid2.pppl.gov GridFTP Server 7.20 (gcc64, 1420641370-85) > >>>>> [Globus Toolkit 6.0] ready. > >>>>> > >>>>> debug: authenticating with > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 230 User tr_nhoward logged in. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> SITE HELP > >>>>> > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 214-The following commands are recognized: > >>>>> ALLO APPE REST CWD CDUP DCAU EPSV FEAT > >>>>> ERET MDTM STAT ESTO HELP LIST MODE NLST > >>>>> MLSC MLSD PASV RNFR MLSR MLST NOOP OPTS > >>>>> STOR PASS PBSZ PORT PROT SITE EPRT RETR > >>>>> SPOR MFMT SCKS TREV PWD QUIT SBUF SIZE > >>>>> SPAS STRU SYST RNTO TYPE USER LANG MKD > >>>>> RMD DELE CKSM DCSC > >>>>> 214 End > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> FEAT > >>>>> > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 211-Extensions supported > >>>>> HTTP > >>>>> DCSC P,D > >>>>> MFMT > >>>>> AUTHZ_ASSERT > >>>>> MLSR > >>>>> MLSC > >>>>> UTF8 > >>>>> LANG EN > >>>>> DCAU > >>>>> PARALLEL > >>>>> SIZE > >>>>> MLST > >>>>> > Type*;Size*;Modify*;Perm*;Charset;UNIX.mode*;UNIX.owner*;UNIX.uid*;UNIX.group*;UNIX.gid*;Unique*;UNIX.slink*;X.count; > >>>>> ERET > >>>>> ESTO > >>>>> SPAS > >>>>> SPOR > >>>>> REST STREAM > >>>>> MDTM > >>>>> PASV AllowDelayed; > >>>>> 211 End. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> SITE CLIENTINFO scheme=gsiftp;appname="globus-url-copy";appver="9.18 > >>>>> (gcc64, 1448068506-85) [unknown]"; > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 250 OK. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> TYPE I > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 200 Type set to I. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> DCAU N > >>>>> > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 200 DCAU N. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> PASV > >>>>> > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 227 Entering Passive Mode (192,55,106,199,164,70) > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> ALLO 348 > >>>>> > >>>>> debug: response from > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> 200 ALLO command successful. > >>>>> > >>>>> debug: sending command to > >>>>> gsiftp://transpgrid.pppl.gov/~/incoming/281948_CMOD.REQUEST: > >>>>> STOR ~/incoming/281948_CMOD.REQUEST > >>>>> > >>>>> > >>>>> > >>>>> Thank you, > >>>>> Irena > >>>> > >>> > >>> > >>> > >>> > >>> -- > >>> Irena > >>> > >>> > >> > >> > >> > >> > >> -- > >> Irena > > > > > > > > > > -- > > Irena > > > >
