File system performance varies wildly between storage architectures.
There are storage server setups that can easily scale to orders of
magnitude beyond the compute that backs - suffice to say
we are currently bound by the number of cores we have available and
not by IO/network performance. (Nate may elaborate on the specifics of
the public servers setup, but I am not sure it is useful to you unless
you have hundreds or thousands (or millions) of dollars to spend on
new storage and network hardware :) ).

Also my idea was not a second Galaxy instance - sorry I did not make
that clearer. It was to restrict your current Galaxy instance to a
smaller portion of your cluster. If your cluster is completely
dedicated to Galaxy however this idea doesn't make sense, but if this
is a shared condor cluster used for other things (besides Galaxy) it
could make sense.

Sorry have not been more helpful.


On Tue, Dec 17, 2013 at 6:12 PM, Ben Gift <> wrote:
> How do you have it set up on the main public galaxy install? I imagine that
> people run enough big jobs that there there is enormous use of your shared
> file system. How did you scale that to so many nodes without bogging down
> the file system with large dataset transfers?
> It seems that for now the solution of having a second Galaxy instance will
> work well, thank you very much John :) . But I'm still interested in a more
> permanent scaled solution. After reading up more on our shared file system
> it still seems like heavy traffic is bad, so could my initial idea still be
> good?
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to