I'll have to test the situation when the jobs are submitted in the background. Don't have a GT available at the moment, so it might take a while. What is the GT server and client version? Also: You said that the error when running the commands in a plain script is 'similar'. Can you paste it?
Martin Belaid MOA wrote: > Thanks a lot Lukasz. I completely agree. I was talking, however, at the > service level not at the client side. > Since using fork (with &) and pthread generates the same error, there is > some how a problem when > a single credential epr is shared between jobs simultaneously. > > ~Belaid. > > > > >> Date: Wed, 21 Jul 2010 15:44:52 +0200 >> From: [email protected] >> To: [email protected] >> CC: [email protected] >> Subject: Re: [gt-user] Threads and > globusrun-ws/globus-credential-delegate. >> >> Hi Belaid, >> >> Thread safety has nothing to do with this. You creates completely >> separated processes. Threads from one process do not interact with >> threads from another process. Every process uses a separated memory >> space allocated by kernel for a process. >> >> (Processes can use shared variables if a method of inter-process >> communication by shared memory is implemented what, I am sure, is not a >> case here). >> >> Regards, >> Lukasz >> >> Belaid MOA wrote: >> > Indeed, the delegation is done first (without &) and then the set of >> > globusrun-ws are with &. >> > >> > The following sentence from >> > http://www.globus.org/toolkit/releasenotes/4.0.4/ may explain why: >> > "The service engine and clients are not thread-safe" >> > >> > Is this means that any client call is not thread-safe? >> > >> > ~Belaid. >> > >> > > Date: Wed, 21 Jul 2010 15:16:59 -0500 >> > > From: [email protected] >> > > CC: [email protected] >> > > Subject: Re: [gt-user] Threads and >> > globusrun-ws/globus-credential-delegate. >> > > >> > > You must not add & at the end of the globus-credential-delegate >> > > command, because the job submission commands require the delegation >> > > command to have finished. Otherwise there won't be an epr of a > delegated >> > > credential. >> > > >> > > Ok, I think what I get out of this is: It works sequentially (that >> > is what >> > > I wanted to confirm, so the usage of the commands is ok), but > maybe not >> > > using pthreads. I don't know what the problem might be. >> > > >> > > Maybe Joe Bester who wrote the command-line tools can provide more > input >> > > on this. >> > > >> > > Martin >> > > >> > > Belaid MOA wrote: >> > > > In the plain shell script as is, no error is thrown. But when we > add & >> > > > at the end of each line, we get the error similar to the one we >> > got from >> > > > using pthreads. >> > > > >> > > > ~Belaid. >> > > > >> > > >> Date: Wed, 21 Jul 2010 14:48:41 -0500 >> > > >> From: [email protected] >> > > >> To: [email protected] >> > > >> CC: [email protected] >> > > >> Subject: Re: [gt-user] Threads and >> > > > globusrun-ws/globus-credential-delegate. >> > > >> >> > > >> I'm sorry, I might be a bit dense but it's still not entirely >> > > >> clear to me: if you run the following in a plain shell script: >> > > >> >> > > >> globus-credential-delegate -h <scheduler> <eprFileName> >> > > >> globusrun-ws -submit -batch -F <scheduler> -Ft <factory> -S -Jf >> > > > <eprFileName> -o JobIdFile -f <jobDescFile> >> > > >> globusrun-ws -submit -batch -F <scheduler> -Ft <factory> -S -Jf >> > > > <eprFileName> -o JobIdFile -f <jobDescFile> >> > > >> globusrun-ws -submit -batch -F <scheduler> -Ft <factory> -S -Jf >> > > > <eprFileName> -o JobIdFile -f <jobDescFile> >> > > >> >> > > >> Do the jobs succeed or fail? >> > > >> >> > > >> Martin >> > > >> >> > > >> >> > > >> Belaid MOA wrote: >> > > >> > Thanks a lot Martin for looking at this. >> > > >> > >> > > >> > 1- In the shell script, I run: >> > > >> > >> > > >> > globus-credential-delegate -h $scheduler $eprName >> > > >> > >> > > >> > 2- The command I call in each thread is: >> > > >> > string sysCommand = "globusrun-ws -submit -batch -F "+scheduler+" >> > > >> > -Ft "+factory+" -S -Jf "; >> > > >> > sysCommand.append(eprName); >> > > >> > sysCommand.append(" -o JobIdFile"); >> > > >> > sysCommand.append(" -f "); >> > > >> > sysCommand.append(jobDescFile); >> > > >> > //submit the request >> > > >> > system(command.c_str()); >> > > >> > >> > > >> > 3- The error is: >> > > >> > >> > > >> > $ globusrun-ws -status -j >> > > >> > JobId: 94110cc2-9376-11df-9044-0019d1a >> > > >> > Current job state: Failed >> > > >> > globusrun-ws: Job failed: Staging error for RSL element >> > fileStageIn. >> > > >> > Connection creation error [Caused by: java.io.EOFException] >> > > >> > Connection creation error [Caused by: java.io.EOFException] >> > > >> > >> > > >> > I do not have access to the GT4 log container on the PBS head >> > node :(. >> > > >> > >> > > >> > ~Belaid. >> > > >> > >> > > >> >> Date: Wed, 21 Jul 2010 10:40:26 -0500 >> > > >> >> From: [email protected] >> > > >> >> CC: [email protected] >> > > >> >> Subject: Re: [gt-user] Threads and >> > > >> > globusrun-ws/globus-credential-delegate. >> > > >> >> >> > > >> >> Can you paste the exact commands you use in the system calls, >> > and the >> > > >> >> error you get in the concurrent scenario? >> > > >> >> >> > > >> >> Martin >> > > >> >> >> > > >> >> Belaid MOA wrote: >> > > >> >> > That's right Martin. For each thread, I just call >> > > >> >> > system("globus-credential-delegate ... ") and use the epr in >> > > >> >> > system("globusrun-ws ...."). That's where I do not get any >> > error. >> > > >> >> > If, however, >> > > >> >> > I call system("globusrun-ws ...") on each thread using a >> > single epr >> > > >> >> > (created in the shell script before running the C program), >> > then I >> > > >> >> > started getting RSL stagein element error. >> > > >> >> > >> > > >> >> > Thanks a lot Martin for looking at this. >> > > >> >> > >> > > >> >> > ~Belaid. >> > > >> >> > >> > > >> >> > >> > > >> >> >> Date: Wed, 21 Jul 2010 09:55:16 -0500 >> > > >> >> >> From: [email protected] >> > > >> >> >> CC: [email protected] >> > > >> >> >> Subject: Re: [gt-user] Threads and >> > > >> >> > globusrun-ws/globus-credential-delegate. >> > > >> >> >> >> > > >> >> >> Hi, >> > > >> >> >> >> > > >> >> >> I'm not sure I get this question right, and I'm also not a >> > C guy >> > > >> > anymore. >> > > >> >> >> >> > > >> >> >> Does it work if you run globus-credential-delegate and >> > globusrun-ws >> > > >> >> > sequentially >> > > >> >> >> as command-line tools? >> > > >> >> >> >> > > >> >> >> I.e. >> > > >> >> >> 1. Call globus-credential-delegate and store the EPR > somewhere. >> > > >> >> >> 2. Then use it for several globusrun-ws job submissions. >> > > >> >> >> >> > > >> >> >> Martin >> > > >> >> >> >> > > >> >> >> Belaid MOA wrote: >> > > >> >> >> > Hi everyone, >> > > >> >> >> > Just a quick question, I am using pthreads in C to run >> > > > globusrun-ws >> > > >> >> >> > and globus-credential-delegate concurrently on a GT4 PBS >> > cluster. >> > > >> >> >> > I noticed that using a single system call to >> > > >> > globus-credential-delegate >> > > >> >> >> > when submitting a set of jobs produces RSL stagein element >> > > > error (The >> > > >> >> >> > jobs are using >> > > >> >> >> > the same epr produced by the single call to >> > > >> > globus-credential-delegate). >> > > >> >> >> > This does not happen when globus-credential-delegate is >> > called >> > > >> > for every >> > > >> >> >> > job (each job has its own unique epr). >> > > >> >> >> > >> > > >> >> >> > Is that mean that globusrun-ws/globus-credential-delegate >> > are not >> > > >> >> >> > thread-safe? >> > > >> >> >> > >> > > >> >> >> > Thanks a lot in advance. >> > > >> >> >> > ~Belaid. >> > > >> >> >> > >> > > >> >> >> > >> > > >> > >> > ------------------------------------------------------------------------ >> > > >> >> >> > Turn down-time into play-time with Messenger games Play > Now! >> > > >> >> > > <http://go.microsoft.com/?linkid=9734381> >> > > >> >> >> >> > > >> >> > >> > > >> >> > >> > > > >> > ------------------------------------------------------------------------ >> > > >> >> > Look 'em in the eye: FREE Messenger video chat Chat Now! >> > > >> >> > <http://go.microsoft.com/?linkid=9734382> >> > > >> >> >> > > >> > >> > > >> > >> > ------------------------------------------------------------------------ >> > > >> > Look 'em in the eye: FREE Messenger video chat Chat Now! >> > > >> > <http://go.microsoft.com/?linkid=9734382> >> > > >> >> > > > >> > > > >> > ------------------------------------------------------------------------ >> > > > Turn down-time into play-time with Messenger games Play Now! >> > > > <http://go.microsoft.com/?linkid=9734381> >> > > >> > >> > ------------------------------------------------------------------------ >> > Look 'em in the eye: FREE Messenger video chat Chat Now! >> > <http://go.microsoft.com/?linkid=9734382> >> > ------------------------------------------------------------------------ >> > Turn down-time into play-time with Messenger games Play Now! >> > <http://go.microsoft.com/?linkid=9734381> >> > > ------------------------------------------------------------------------ > Look 'em in the eye: FREE Messenger video chat Chat Now! > <http://go.microsoft.com/?linkid=9734382>
