Dougal, To the best of my knowledge Gram4.x/RFT does not have such a detection mechanism. I don't know if tools on top of Gram4 (Swift, Gridway, others?) provide mechanisms for your use-case.
A general note: If you are not tied to web-services, I'd check with the Gram5 folks if they still support what Steve described, and maybe consider going with Gram5 (based on and improved Gram2) in the long term, because in medium terms Gram4.x won't be supported anymore, but Gram5 will (back to the future! :) ) Martin Steven Timm wrote: > Dougal--I am not sure what the GT4 equivalent is for file > stage-in but I know that the GT2 stagein does detect that the > same file has previously been staged in, and not stage it in again. > The cache on the far end has many hard links to the same file. > > Steve > > > On Mon, 31 May 2010, Dougal Ballantyne wrote: > >> Dear GT, >> >> I have been working on a project for several months now researching >> and developing a grid solution based on Globus Toolkit 4. Many thanks >> to people who have helped me with previous issues. >> >> I have a slightly Off-Topic question related to how others handle a >> particular scenario. >> >> We have a job generation and control application that we have added >> support for Globus through some perl modules that call globusrun-ws. >> When a job is generated, the program pulls from the job database the >> associated input files and creates an XML file which lists the input >> files in StageIn and the requested results file in StageOut. This >> works great for a single job and jobs that all use different input >> data. However we often have a scenario when we generate several >> hundred jobs that all use the same input data. In our current setup we >> would StageIn the same input file several hundred times. >> >> I was wondering if that was a method or known best practice within the >> Globus Toolkit for handling this sort of scenario. I am aware that we >> could modify the tool to stage the data first, run the jobs and then >> remove the input file BUT that would also be a change of workflow for >> the users. >> >> Your thoughts or comments greatly appreciated. >> >> Kind regards, >> >> Dougal Ballantyne >> >
