Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Shane Sturrock
Thanks Peter, that was the hint I was needing. It looks like the solution for 
my LSF cluster was to use drmaa://-n 8/ in the universe_wsgi.ini and now I can 
see BLAST using 8 cores. It doesn’t scale perfectly so going higher isn't 
sensible but turnaround time for my test job has improved markedly.

Shane

Dr. Shane Sturrock
NZGL BioIT Admin
n...@biomatters.com

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Peter Cock
On Thu, Jul 21, 2016 at 10:35 AM, Shane Sturrock  wrote:
>
> I’m just using the drmaa plugin so I’m guessing I need to either specify a
> job runner for BLAST which sets GALAXY_SLOTS. The documentation isn’t
> entirely clear although it seems I need to create a job_conf.xml which sets
> local_slots to a value for runner drmaa unless there’s a way to set that in
> the galaxy.ini file instead.

There is a legacy job setting via galaxy.ini aka universe_wsgi.ini where
we used to use this:

[galaxy:tool_runners]
ncbi_blastp_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastn_wrapper  = drmaa://-V -pe smp 4/
ncbi_blastx_wrapper  = drmaa://-V -pe smp 4/
ncbi_tblastn_wrapper = drmaa://-V -pe smp 4/
ncbi_tblastx_wrapper = drmaa://-V -pe smp 4/
blast_reciprocal_best_hits = drmaa://-V -pe smp 4/

The -V was an SGE switch to copy the environment variables,
and -pe smp 4 is for a 4 core job.

For job_conf.xml I am trying something similar with less repetition:

...


-pe smp 4


...








...


i.e. I've set the computationally heavy BLAST+ tools to use a
destination configured to use just four cores. Because this is
SGE, we do this via the native syntax of -pe smp 4, i.e.
just like at the command line we'd do:

$ qsub -pe smp 4 ...

This detail will be different for your LSF cluster.

Peter
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Shane Sturrock
Hi Peter,

> On 21/07/2016, at 8:50 PM, Peter Cock  wrote:
> 
> Hi Shane,
> 
> We've not touched anything on the BLAST+ wrapper here for
> a while - the command line is always built using:
> 
> -num_threads "\${GALAXY_SLOTS:-8}"
> 
> That means use the environment variable $GALAXY_SLOTS
> if set, defaulting to 8 threads if not. See:
> 
> https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/ncbi_macros.xml#L352
> 
> Galaxy itself should be setting $GALAXY_SLOTS via your
> cluster configuration - and from your description this seems
> to be set to 1 thread/slot only.

I suspect since I’m running an old configuration that my cluster setup isn’t 
optimal. I didn’t really want my users going nuts since our cluster is small (6 
nodes, 16 cores each) so I wasn’t keen to give them too much power because and 
was happy that tools would only run on single cores although I’ve set things up 
as per the docs way back when so it uses multiple handlers and so on and with 
default_cluster_job_runner = drmaa:/// all in the galaxy.ini (universe_wsgi.ini 
actually)

> 
> Can you tell us more about how your Galaxy LSF is setup?
> Have you got BLAST specific cluster job settings?

Nothing specific, but it would seem that the GALAXY_SLOTS variable has gone 
from being unset to set with the install I’ve got now and that would explain 
why BLAST has slowed down a lot.
> 
> (I don't use LSF so don't have the details to hand)

I’m just using the drmaa plugin so I’m guessing I need to either specify a job 
runner for BLAST which sets GALAXY_SLOTS. The documentation isn’t entirely 
clear although it seems I need to create a job_conf.xml which sets local_slots 
to a value for runner drmaa unless there’s a way to set that in the galaxy.ini 
file instead.

Shane

> 
> Peter
> 
> On Wed, Jul 20, 2016 at 10:03 PM, Shane Sturrock  wrote:
>> Previously, the BLAST wrapper was able to use multiple cores but recently
>> users have started complaining it has got really slow and when I look at the
>> cluster a job is only using a single core. I don’t want jobs split across
>> multiple cluster nodes, but each node has 16 cores so it would be good if
>> they could be used. I’m still using the older 16.01 release and this has
>> been upgraded repeatedly over the last few years so I’m still using the
>> universe_wsg.ini file (sym linked to config/galaxy.ini) and I don’t have a
>> job_conf.xml set up. I’m using drmaa to drive an LSF cluster. BLAST is the
>> main issue here so I was wondering if there’s a way to pass the -num_threads
>> flag without breaking everything until I can build up a new server?
>> 
>> Dr. Shane Sturrock
>> NZGL BioIT Admin
>> n...@biomatters.com
>> 
>> 
>> ___
>> Please keep all replies on the list by using "reply all"
>> in your mail client.  To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
>>  https://lists.galaxyproject.org/
>> 
>> To search Galaxy mailing lists use the unified search at:
>>  http://galaxyproject.org/search/mailinglists/


Dr. Shane Sturrock
NZGL BioIT Admin
n...@biomatters.com

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] BLAST only using single core

2016-07-21 Thread Peter Cock
Hi Shane,

We've not touched anything on the BLAST+ wrapper here for
a while - the command line is always built using:

-num_threads "\${GALAXY_SLOTS:-8}"

That means use the environment variable $GALAXY_SLOTS
if set, defaulting to 8 threads if not. See:

https://github.com/peterjc/galaxy_blast/blob/master/tools/ncbi_blast_plus/ncbi_macros.xml#L352

Galaxy itself should be setting $GALAXY_SLOTS via your
cluster configuration - and from your description this seems
to be set to 1 thread/slot only.

Can you tell us more about how your Galaxy LSF is setup?
Have you got BLAST specific cluster job settings?

(I don't use LSF so don't have the details to hand)

Peter

On Wed, Jul 20, 2016 at 10:03 PM, Shane Sturrock  wrote:
> Previously, the BLAST wrapper was able to use multiple cores but recently
> users have started complaining it has got really slow and when I look at the
> cluster a job is only using a single core. I don’t want jobs split across
> multiple cluster nodes, but each node has 16 cores so it would be good if
> they could be used. I’m still using the older 16.01 release and this has
> been upgraded repeatedly over the last few years so I’m still using the
> universe_wsg.ini file (sym linked to config/galaxy.ini) and I don’t have a
> job_conf.xml set up. I’m using drmaa to drive an LSF cluster. BLAST is the
> main issue here so I was wondering if there’s a way to pass the -num_threads
> flag without breaking everything until I can build up a new server?
>
> Dr. Shane Sturrock
> NZGL BioIT Admin
> n...@biomatters.com
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] BLAST only using single core

2016-07-20 Thread Shane Sturrock
Previously, the BLAST wrapper was able to use multiple cores but recently users 
have started complaining it has got really slow and when I look at the cluster 
a job is only using a single core. I don’t want jobs split across multiple 
cluster nodes, but each node has 16 cores so it would be good if they could be 
used. I’m still using the older 16.01 release and this has been upgraded 
repeatedly over the last few years so I’m still using the universe_wsg.ini file 
(sym linked to config/galaxy.ini) and I don’t have a job_conf.xml set up. I’m 
using drmaa to drive an LSF cluster. BLAST is the main issue here so I was 
wondering if there’s a way to pass the -num_threads flag without breaking 
everything until I can build up a new server?

Dr. Shane Sturrock
NZGL BioIT Admin
n...@biomatters.com

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/