On Fri, Apr 8, 2016 at 9:55 AM, Poole, Richard <r.po...@ucl.ac.uk> wrote:

> Could somebody point me to a good explanation of how to setup and use
> GALAXY_SLOTS correctly on my server?
>
> A basic explanation is good but I also make use of Pulsar to stage some
> jobs on our cluster here (my machine is 4-core and cluster I use is
> 12-core) so I am wondering if GALAXY_SLOTS can handle this (so I don't need
> to specify exact thread numbers in e.g. tool wrappers)
>

Hi Richard,

Is your cluster running a distributed resource manager (DRM) like PBS, grid
engine, etc.? If yes, then $GALAXY_SLOTS is handled automatically based on
whatever options you submit to your cluster with. If you request that a job
be allocated 12 cores on a node, $GALAXY_SLOTS will be set to 12 and any
tools which respect $GALAXY_SLOTS (which include all of the multicore
devteam and IUC tools) will use 12 cores accordingly.

It is important to only submit tools which can use multiple cores with the
multicore option, otherwise you may allocate 12 cores for a tool which will
only use 1, wasting resources. Here is the job configuration file we use
for usegalaxy.org which shows how to map multicore tools to multicore
destinations running the Slurm DRM:


https://github.com/galaxyproject/usegalaxy-playbook/blob/master/templates/galaxy/usegalaxy.org/config/job_conf.xml.j2

For example, `bowtie2` (line 201) runs on the `slurm_multi` destination
(126) via the `dynamic_local_stampede_select_dynamic_walltime` dynamic
destination (in another file, but the details are not relevant, in your
case you can map directly from a tool to multicore destination defined in
job_conf.xml).

If your cluster is running a DRM, you most likely do not need to run Pulsar
(Galaxy has native support for pretty much all commonly used DRMs) unless
you need the ability to stage files to/from the cluster or do not have
direct submit access to the cluster from the Galaxy server.

--nate


> Thanks,
>
> Richard
>
> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to