You could speed it up considerably, if you can split your jobs and submit more jobs with a smaller count number instead of one job with a large count number:
Some numbers from my environment with a dummy job: 1 job, count=1000: 6min 9s 10 jobs, each with count=100: 57s Reason for that is that Gram4 has to lock a job resource to process an event (e.g. "job done"), but this does not impact other job resources. I can't tell yet what's the optimal job/count ratio. I'll check if there's room for improvement. Martin ----- Original Message ----- From: "Martin Feller" <[EMAIL PROTECTED]> To: "wtk" <[EMAIL PROTECTED]> Cc: [email protected] Sent: Thursday, June 5, 2008 6:29:02 PM GMT -06:00 US/Canada Central Subject: Re: [gt-user] GRAM Puzzle Hi, I checked a bit, and it seems as follows: The jobs are done quickly and the information about all these jobs is gathered quickly. But the processing inside Gram4 takes the time: For each of the <count> jobs the job resource has to be processed. I don't know yet why this is so time-consuming and have to check if and how this could be improved performance-wise. Martin ----- Original Message ----- From: "wtk" <[EMAIL PROTECTED]> To: [email protected] Sent: Monday, June 2, 2008 12:21:52 AM GMT -06:00 US/Canada Central Subject: [gt-user] GRAM Puzzle Hi, I encountered a strange problem recently. After i submitted one job continuously for several timeswith the same user ,I found that most of my jobs finished successfully,but the rest of themkeep the status of "stageIn"/"Active" for hours ,and it's far beyond the job's execution time. I can not figure out what happendbecause the container.log contains no error information. I hopesomeonecantell me how to handle this problem or how did this happen Thanks here is my job file: ============================================= <job> <factoryEndpoint xmlns:gram=" http://www.globus.org/namespaces/2004/10/gram/job " xmlns:wsa=" http://schemas.xmlsoap.org/ws/2004/03/addressing "> <wsa:Address> https://serverIP:8443/wsrf/services/ManagedJobFactoryService </wsa:Address> <wsa:ReferenceProperties> <gram:ResourceID>Fork</gram:ResourceID> </wsa:ReferenceProperties> </factoryEndpoint> <executable>/home/job/seqret/seqret</executable> <directory>/home/job/seqret/</directory> <argument>fasta::job136.stgin</argument> <argument>phylip::job136.stgout</argument> <stdout>job136.stdout</stdout> <stderr>job136.stderr</stderr> <count>3000</count> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <fileStageIn> <transfer> <sourceUrl>gsiftp://ServerIP:2811/tmp/workload/Outfiles/outseq.stgin</sourceUrl> <destinationUrl>file:///home/job/seqret/job136.stgin</destinationUrl> </transfer> </fileStageIn> <fileStageOut> <transfer> <sourceUrl>file:///home/job/seqret/job136.stdout</sourceUrl> <destinationUrl>gsiftp://ServerIP:2811/tmp/workload/Infiles/job136.stdout</destinationUrl> </transfer> <transfer> <sourceUrl>file:///home/job/seqret/job136.stderr</sourceUrl> <destinationUrl>gsiftp://ServerIP:2811/tmp/workload/Infiles/job136.stderr</destinationUrl> </transfer> <transfer> <sourceUrl>file:///home/job/seqret/job136.stgout</sourceUrl> <destinationUrl>gsiftp://ServerIP:2811/tmp/workload/Infiles/job136.answer</destinationUrl> </transfer> </fileStageOut> </job>
