Re: [galaxy-dev] pass user groups to dynamic job runner
Il giorno ven, 20/09/2013 alle 10.25 -0500, John Chilton ha scritto:
> On Fri, Sep 20, 2013 at 9:08 AM, Nicola Soranzo wrote:
> >> First step is to create a new dynamic job destination, this is
> >> demonstrated in job_conf.xml in the above changeset. This demonstrates
> >> the new style job_conf section - this will need to be adapted for the
> >> old style runners but is still doable.
> >>
> >> Then you will want to add a dynamic job runner rule that implements
> >> this logic. This is the file lib/galaxy/jobs/rules/license_checker.py.
> >
> > Is there a way to specify in job_conf.xml a per-tool final destination
> > after the has_license filtering ? E.g.:
> >
> >
> > > final_destination="remote_cluster" />
> >
> > The 'final_destination' attribute would then be used in has_license
> > function instead of returning DEFAULT_JOB_DESTINATION_ID.
> >
>
> There is no way currently to pass information from the tool tag to the
> dynamic runner or to have the dynamic runner "pass" on making a
> decision in some way. It could potentially make sense to add these -
> but I would put forward a workaround that in my opinion is actually a
> little better:
> ...
I have actually found a way to implement my idea of specifying a final
destination! I also generalized your approach giving the possibility to
also specify a per-tool required group in jobs_conf.xml .
job_conf.xml :
is_user_in_group
lib/galaxy/jobs/rules/is_user_in_group.py :
from galaxy.jobs.mapper import JobMappingException
DEFAULT_GROUP = 'have_license'
def is_user_in_group(user, app, tool_id):
# app is a galaxy.app.UniverseApplication object
if user is None:
raise JobMappingException('You must login to use this tool!')
user_group_assocs = user.groups or []
default_destination_id = app.job_config.get_destination(None)
for jtc in app.job_config.get_job_tool_configurations(tool_id): #
app.job_config.get_job_tool_configurations(tool_id) returns a list of
galaxy.jobs.JobToolConfiguration objects
if not jtc.params:
required_group = jtc.get('required_group', DEFAULT_GROUP)
final_destination = jtc.get('final_destination',
default_destination_id)
break
else:
required_group = DEFAULT_GROUP
final_destination = default_destination_id
user_in_group = required_group in [user_group_assoc.group.name for
user_group_assoc in user_group_assocs]
if not user_in_group:
raise JobMappingException('This tool is restricted to users in
the %s group, please contact a site administrator to be authorized!' %
required_group)
else:
return final_destination
Hope that helps,
Nicola
--
Nicola Soranzo, Ph.D.
CRS4
Bioinformatics Program
Loc. Piscina Manna
09010 Pula (CA), Italy
http://www.bioinformatica.crs4.it/
> >> There is no longer any need to display such tools to the user thanks
> >> to dynamic toolbox filters. This is that last file:
> >> lib/galaxy/tools/filters/license_filter.py. This will prevent any tool
> >> in the list 'LICENSED_TOOLS' from even being seen by unlicensed users.
> >> You do need to specify this filter is being used by adding the line:
> >>
> >> tool_filters = license_filter:has_license
> >>
> >> in the [app:main] section of universe_wsgi.ini.
> >
> > I think that something like this:
> >
> > # Tool filtering. Multiple such filters can be specified by giving
> > # module (relative to galaxy.tools.filters) and function names for each
> > # as a comma separated list.
> > # See lib/galaxy/tools/filters/examples.py.sample for some examples.
> > #tool_filters =
> > #tool_label_filters =
> > #tool_section_filters =
> >
> > should be added to universe_wsgi.ini.sample , otherwise this wonderful
> > feature is very difficult to discover.
>
> Do you mean to suggest that merely documenting my contributions to
> Galaxy on my Twitter feed is insufficient? Hmm I don't know about
> that...
>
> Seriously though, Björn has implemented some extensions to dynamic
> toolbox filters that include this:
>
> https://bitbucket.org/galaxy/galaxy-central/pull-request/179/implement-the-ability-to-change-the-tool/diff#chg-universe_wsgi.ini.sample
>
> I have added more options to Galaxy then I have documented in that
> file and I am always unsure whether to add them or not, the hesitation
> is that I think that file has too cluttered. I have missed really
> important modifications that should have been made because I didn't
> see them in that file.
>
> These filters may rise to the level that they belong in there, I am
> not certain, but they should certainly be documented at least on the
> wiki somewhere
>
> -John
>
> >
> >> Hope this helps!
> >>
> >> -John
> >
> > Thanks again!
> >
> > Nicola
> >
> >> On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert
> >> wrote:
> >> > Hi,
> >> >
> >> > Could somebody point me to the
Re: [galaxy-dev] pass user groups to dynamic job runner
On Fri, Sep 20, 2013 at 9:08 AM, Nicola Soranzo wrote:
> Il giorno gio, 19/09/2013 alle 12.42 -0500, John Chilton ha scritto:
>> Feature request received.
>>
>> The following changeset demonstrates how one could implement this
>>
>> https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd
>
> Hi John,
> that's pretty interesting and we are probably going to use something
> similar for some "problematic" tools, thanks for sharing!
>
>> First step is to create a new dynamic job destination, this is
>> demonstrated in job_conf.xml in the above changeset. This demonstrates
>> the new style job_conf section - this will need to be adapted for the
>> old style runners but is still doable.
>>
>> Then you will want to add a dynamic job runner rule that implements
>> this logic. This is the file lib/galaxy/jobs/rules/license_checker.py.
>
> Is there a way to specify in job_conf.xml a per-tool final destination
> after the has_license filtering ? E.g.:
>
>
> final_destination="remote_cluster" />
>
> The 'final_destination' attribute would then be used in has_license
> function instead of returning DEFAULT_JOB_DESTINATION_ID.
>
There is no way currently to pass information from the tool tag to the
dynamic runner or to have the dynamic runner "pass" on making a
decision in some way. It could potentially make sense to add these -
but I would put forward a workaround that in my opinion is actually a
little better:
from galaxy.jobs.mapper import JobMappingException
DEFAULT_DESTINATION_ID = "remoteCluster"
LOCAL_DESTINATION_ID = "local"
LOCAL_TOOLS = ['cat1']
def has_license(user, tool_id):
user_group_assocs = user.groups or []
user_has_license = 'have_license' in [user_group_assoc.group.name
for user_group_assoc in user_group_assocs]
if not user_has_license:
raise JobMappingException("No license, no tool!")
if tool_id in LOCAL_TOOLS:
return LOCAL_DESTINATION_ID
else:
return DEFAULT_DESTINATION_ID
The problem here is that you sort of have special logic for 'cat1' in
two places so what I would probably do is actually just eliminate the
tool mappings all together in job_conf.xml and assign these tools
destination using a default dynamic job runner.
from galaxy.jobs.mapper import JobMappingException
DEFAULT_DESTINATION_ID = "remoteCluster"
LOCAL_DESTINATION_ID = "local"
LOCAL_TOOLS = ['cat1']
REQUIRES_LICENSE = ['cat1', 'cat2']
def default_mapper(user, tool_id):
if tool_id in REQUIRES_LICENSE:
_check_license(user)
if tool_id in LOCAL_TOOLS:
return LOCAL_DESTINATION_ID
else:
return DEFAULT_DESTINATION_ID
def _check_license(user):
user_group_assocs = user.groups or []
user_has_license = 'have_license' in [user_group_assoc.group.name
for user_group_assoc in user_group_assocs]
if not user_has_license:
raise JobMappingException("No license, no tool!")
I don't use the dynamic job runner personally, but I am nevertheless
going to declare it a best practice that at some point the complexity
of the rules is such that it just makes sense to stop assigning any
tools mappings in job_conf.xml and just use a default dynamic job
destination for everything.
>> There is no longer any need to display such tools to the user thanks
>> to dynamic toolbox filters. This is that last file:
>> lib/galaxy/tools/filters/license_filter.py. This will prevent any tool
>> in the list 'LICENSED_TOOLS' from even being seen by unlicensed users.
>> You do need to specify this filter is being used by adding the line:
>>
>> tool_filters = license_filter:has_license
>>
>> in the [app:main] section of universe_wsgi.ini.
>
> I think that something like this:
>
> # Tool filtering. Multiple such filters can be specified by giving
> # module (relative to galaxy.tools.filters) and function names for each
> # as a comma separated list.
> # See lib/galaxy/tools/filters/examples.py.sample for some examples.
> #tool_filters =
> #tool_label_filters =
> #tool_section_filters =
>
> should be added to universe_wsgi.ini.sample , otherwise this wonderful
> feature is very difficult to discover.
Do you mean to suggest that merely documenting my contributions to
Galaxy on my Twitter feed is insufficient? Hmm I don't know about
that...
Seriously though, Björn has implemented some extensions to dynamic
toolbox filters that include this:
https://bitbucket.org/galaxy/galaxy-central/pull-request/179/implement-the-ability-to-change-the-tool/diff#chg-universe_wsgi.ini.sample
I have added more options to Galaxy then I have documented in that
file and I am always unsure whether to add them or not, the hesitation
is that I think that file has too cluttered. I have missed really
important modifications that should have been made because I didn't
see them in that file.
These filters may rise to the level that they belong in there, I am
not certain, but they should certainly be documented at least on the
wiki somewhere
-John
>
>> Hope th
Re: [galaxy-dev] pass user groups to dynamic job runner
Thanks, I integrated parts of these snippets into our general dynamic_jobs function, and it works flawlessly. Best, Geert Van: jmchil...@gmail.com [jmchil...@gmail.com] namens John Chilton [chil...@msi.umn.edu] Verzonden: donderdag 19 september 2013 19:42 Aan: Vandeweyer Geert CC: galaxy-...@bx.psu.edu Onderwerp: Re: [galaxy-dev] pass user groups to dynamic job runner Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. This will do what you asked, you can adjust the tools it targets and the message that gets displayed to the user pretty easily (right now it is just "No license, no tool"). There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. Hope this helps! -John On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert wrote: > Hi, > > Could somebody point me to the place where I can create a ticket for the > following feature: > > - We want to have tools available only to users who provided a licence for > this tool. > - To prevent very long email lists (see example on dev-only tools), I'd like > to have a group 'have_licence' > - in dynamic job runner function : check if user is in usergroup : ok => run > ; fail => give message. > > Or can I hide tools from the menu based on usergroup? > > Best, > > Geert > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] pass user groups to dynamic job runner
Il giorno gio, 19/09/2013 alle 12.42 -0500, John Chilton ha scritto: > Feature request received. > > The following changeset demonstrates how one could implement this > > https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd Hi John, that's pretty interesting and we are probably going to use something similar for some "problematic" tools, thanks for sharing! > First step is to create a new dynamic job destination, this is > demonstrated in job_conf.xml in the above changeset. This demonstrates > the new style job_conf section - this will need to be adapted for the > old style runners but is still doable. > > Then you will want to add a dynamic job runner rule that implements > this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. Is there a way to specify in job_conf.xml a per-tool final destination after the has_license filtering ? E.g.: The 'final_destination' attribute would then be used in has_license function instead of returning DEFAULT_JOB_DESTINATION_ID. > There is no longer any need to display such tools to the user thanks > to dynamic toolbox filters. This is that last file: > lib/galaxy/tools/filters/license_filter.py. This will prevent any tool > in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. > You do need to specify this filter is being used by adding the line: > > tool_filters = license_filter:has_license > > in the [app:main] section of universe_wsgi.ini. I think that something like this: # Tool filtering. Multiple such filters can be specified by giving # module (relative to galaxy.tools.filters) and function names for each # as a comma separated list. # See lib/galaxy/tools/filters/examples.py.sample for some examples. #tool_filters = #tool_label_filters = #tool_section_filters = should be added to universe_wsgi.ini.sample , otherwise this wonderful feature is very difficult to discover. > Hope this helps! > > -John Thanks again! Nicola > On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert > wrote: > > Hi, > > > > Could somebody point me to the place where I can create a ticket for the > > following feature: > > > > - We want to have tools available only to users who provided a licence for > > this tool. > > - To prevent very long email lists (see example on dev-only tools), I'd > > like to have a group 'have_licence' > > - in dynamic job runner function : check if user is in usergroup : ok => > > run ; fail => give message. > > > > Or can I hide tools from the menu based on usergroup? > > > > Best, > > > > Geert > > ___ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] pass user groups to dynamic job runner
Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok => run ; fail => give message. Or can I hide tools from the menu based on usergroup? Best, Geert ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] pass user groups to dynamic job runner
Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. This will do what you asked, you can adjust the tools it targets and the message that gets displayed to the user pretty easily (right now it is just "No license, no tool"). There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. Hope this helps! -John On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert wrote: > Hi, > > Could somebody point me to the place where I can create a ticket for the > following feature: > > - We want to have tools available only to users who provided a licence for > this tool. > - To prevent very long email lists (see example on dev-only tools), I'd like > to have a group 'have_licence' > - in dynamic job runner function : check if user is in usergroup : ok => run > ; fail => give message. > > Or can I hide tools from the menu based on usergroup? > > Best, > > Geert > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
