On Mar 12, 2012, at 9:44 AM, Louise-Amélie Schmitt wrote:
> Hi Nate, and thanks for the reply
>>> - We saw the outputs_to_working_directory option in the .ini file, but it
>>> only concerns output files, is there a way to make a local copy of all the
>>> input files in the job working directory? (including indices)
>> Hi L-A,
>> Unfortunately no. This option was only created so that you could mount the
>> file_path directory read-only on your cluster nodes for security purposes.
> In the end it shouldn't be necessary. The guy in charge of the cluster said
>>> - Can we set the job working directory as an absolute path so it's local to
>>> the nodes, like /tmp ?
>> No, things like the external metadata inputs and outputs are written to the
>> working directory and are expected to be available on both the cluster and
>> the application server at the same path.
> Ok, I see. How do you manage that then? Is it actually not an issue I/O-wise?
Most of the things written to the working directory aren't large enough to
cause performance problems. The worst culprit, before we split up the load
across a pool of servers, was tophat. Ideally, Galaxy needs a mechanism to
define whether local or network temp space should be used (or perhaps more
abstractly, a way to estimate the size needed based on the inputs).
>>> - If it's possible, will the job working directories created for each job
>>> be cleaned properly at the end?
>> Yes, depending on the value of "cleanup_job" in the config file.
>> Enhancements that remove this "shared directory" limitation are on our
>> roadmap for this year.
> Ok, thanks a lot for all this information!
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client. To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at: