Re: [galaxy-dev] question about GALAXY_SLOTS
On Thu, Oct 16, 2014 at 11:05 PM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
Hi,
this is just to make sure: the GALAXY_SLOTS environmental variable set by
Galaxy when running tools will always be a number = 1 with 1 being the
default if nothing else is configured in the job runner settings ?
Correct ?
Thanks,
Wolfgang
Hi Wolfgang,
I believe so, however it is possible it might be unset in a corner case
(please report this as a bug if you see it happen) or a tool could change
the value.
You can use the following bash syntax to set your own default in
the tool's command template, e.g.
-num_threads \${GALAXY_SLOTS:-8}
Note the colon minus is the special bash syntax, here the default
value is 8 (not minus 8) if $GALAXY_SLOTS is not set. Also note
in the command XML you must escape the dollar sign.
Peter
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] question about GALAXY_SLOTS
On 10/17/2014 10:39 AM, Peter Cock wrote:
On Thu, Oct 16, 2014 at 11:05 PM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
Hi,
this is just to make sure: the GALAXY_SLOTS environmental variable set by
Galaxy when running tools will always be a number = 1 with 1 being the
default if nothing else is configured in the job runner settings ?
Correct ?
Thanks,
Wolfgang
Hi Wolfgang,
I believe so, however it is possible it might be unset in a corner case
(please report this as a bug if you see it happen) or a tool could change
the value.
You can use the following bash syntax to set your own default in
the tool's command template, e.g.
-num_threads \${GALAXY_SLOTS:-8}
Note the colon minus is the special bash syntax, here the default
value is 8 (not minus 8) if $GALAXY_SLOTS is not set. Also note
in the command XML you must escape the dollar sign.
Peter
Hi Peter,
thanks for confirming. I'm not particularly worried about it being absent.
My use-case is own software that has its own threading config settings,
but when run in Galaxy should respect $GALAXY_SLOTS. To do this, it's
checking for the presence of the environmental variable and gives it
precedence over its own config. What would be problematic though is if
in certain situations GALAXY_SLOTS could be 0 or a negative number or
something that can't be cast to an int, but Björn's and your answer
imply that this won't happen.
By the way, there's one exception from the above precedence rule that
I'm planning to implement: if GALAXY_SLOTS defaults to 1, I'll use the
config settings instead, essentially giving users the choice if they
prefer configuration from Galaxy or from the tool side. Do you think
this is a good idea ?
Best,
Wolfgang
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] question about GALAXY_SLOTS
Hi,
I'm not sure to understand ... I was but not now anymore :P
When you find \${GALAXY_SLOTS:-4} in a wrapper, and if you haven't set a
tool_runner (-pe thread 8) for this tool, by default it will use 4
splots ! No ?
Is there a global GALAXY_SLOTS variable ?
Thanks
Gildas
Le 17/10/2014 02:03, Björn Grüning a écrit :
Hi Wolfgang,
\${GALAXY_SLOTS:-4} in this case will default to 4 if you have not
configured it in job_conf.xml. Otherwise yes, it will be 1.
Ciao,
Bjoern
Am 17.10.2014 um 00:05 schrieb Wolfgang Maier:
Hi,
this is just to make sure: the GALAXY_SLOTS environmental variable set
by Galaxy when running tools will always be a number = 1 with 1 being
the default if nothing else is configured in the job runner settings ?
Correct ?
Thanks,
Wolfgang
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] question about GALAXY_SLOTS
In theory GALAXY_SLOTS is now always set unless Galaxy is pretty old -
I still think it is important for tools to specify defaults using the
\${GALAXY_SLOTS:-4} idiom so that tools themselves don't have explicit
dependencies on the Galaxy runtime and in case new resource managers
or job runners come along that don't set this.
That said - I figured someday someone would ask this question so there
is indeed undocumented magic to determine if the number of slots has
been configured explicitly to 1 or if that is just the fallback
because it was left unspecified or Galaxy didn't know how to determine
it from the resource manager.
The magic environment variable is GALAXY_SLOTS_CONFIGURED - if that is
set and GALAXY_SLOTS is 1 - Galaxy believes that the number of
requested cores is truly and explicitly 1. If GALAXY_SLOTS is 1 and
GALAXY_SLOTS_CONFIGURED is not set - Galaxy is just supplying the
default of 1 to the tool because it doesn't know what else to do. (1
is used by the local runner if local_slots is not set i job_conf or
used by all the other job runners if they cannot figure out what the
environment is requesting).
I don't know how actionable it is from a tool author perspective - but
I did think it was important to track it. Hope this helps.
-John
On Fri, Oct 17, 2014 at 5:19 AM, Gildas Le Corguille
lecorgui...@sb-roscoff.fr wrote:
Hi,
I'm not sure to understand ... I was but not now anymore :P
When you find \${GALAXY_SLOTS:-4} in a wrapper, and if you haven't set a
tool_runner (-pe thread 8) for this tool, by default it will use 4 splots !
No ?
Is there a global GALAXY_SLOTS variable ?
Thanks
Gildas
Le 17/10/2014 02:03, Björn Grüning a écrit :
Hi Wolfgang,
\${GALAXY_SLOTS:-4} in this case will default to 4 if you have not
configured it in job_conf.xml. Otherwise yes, it will be 1.
Ciao,
Bjoern
Am 17.10.2014 um 00:05 schrieb Wolfgang Maier:
Hi,
this is just to make sure: the GALAXY_SLOTS environmental variable set
by Galaxy when running tools will always be a number = 1 with 1 being
the default if nothing else is configured in the job runner settings ?
Correct ?
Thanks,
Wolfgang
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] question about GALAXY_SLOTS
On 10/17/2014 02:47 PM, John Chilton wrote:
In theory GALAXY_SLOTS is now always set unless Galaxy is pretty old -
I still think it is important for tools to specify defaults using the
\${GALAXY_SLOTS:-4} idiom so that tools themselves don't have explicit
dependencies on the Galaxy runtime and in case new resource managers
or job runners come along that don't set this.
That said - I figured someday someone would ask this question so there
is indeed undocumented magic to determine if the number of slots has
been configured explicitly to 1 or if that is just the fallback
because it was left unspecified or Galaxy didn't know how to determine
it from the resource manager.
The magic environment variable is GALAXY_SLOTS_CONFIGURED - if that is
set and GALAXY_SLOTS is 1 - Galaxy believes that the number of
requested cores is truly and explicitly 1. If GALAXY_SLOTS is 1 and
GALAXY_SLOTS_CONFIGURED is not set - Galaxy is just supplying the
default of 1 to the tool because it doesn't know what else to do. (1
is used by the local runner if local_slots is not set i job_conf or
used by all the other job runners if they cannot figure out what the
environment is requesting).
I don't know how actionable it is from a tool author perspective - but
I did think it was important to track it. Hope this helps.
Hi John,
This is great ! I think it is very actionable for my case. I would just
overwrite my tool's internal configuration with GALAXY_SLOTS if it is
set and if GALAXY_SLOTS 1 or (GALAXY_SLOTS == 1 and
GALAXY_SLOTS_CONFIGURED is set).
Much better than having to guess for the == 1 case, thanks!
Have a nice weekend,
Wolfgang
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] question about GALAXY_SLOTS
Hi Wolfgang,
\${GALAXY_SLOTS:-4} in this case will default to 4 if you have not
configured it in job_conf.xml. Otherwise yes, it will be 1.
Ciao,
Bjoern
Am 17.10.2014 um 00:05 schrieb Wolfgang Maier:
Hi,
this is just to make sure: the GALAXY_SLOTS environmental variable set
by Galaxy when running tools will always be a number = 1 with 1 being
the default if nothing else is configured in the job runner settings ?
Correct ?
Thanks,
Wolfgang
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
To search Galaxy mailing lists use the unified search at:
http://galaxyproject.org/search/mailinglists/
