On Dec 20, 2011, at 5:04 AM, Cittaro Davide wrote:
> Hi developers, I have a question that may be an OT, but since galaxy can work
> in a clustered environment withh queueing system, I'll try to ask here.
> Is there anibody here who copies data in a local temporary directory before
> performing any analysis step and copy it back into the "final results"?
We did this for a while when we had a poorly performing fileserver. It can
reduce load in that environment, but in cases where you are only going to read
small portions of input files, you'll probably have longer execution time.
Likewise if you'll simply be writing the output(s) in one big stream, since you
then have to write it once locally and then back over the network.
That said, if you have a lot interim steps that produce large data that then
get merged via some process back to final outputs, it absolutely makes sense to
use local disk for those steps (assuming local disk is large enough - another
problem that we sometimes encounter).
> Sent from my iPad
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at: