Re: [galaxy-dev] Towards Galaxy Linux (not) ? [was [RFC] Storing of tarballs and patches for tool_dependencies to enable reproducibility]
Hi Ido, If I might chime in, I am a bit worried about all the automatic installation going on in galaxy, and it seems that the trend is to enhance this. A small R or python script calling into well known libraries that come from well known repositories (bioconductor etc… ) I can check. (Of course I install too much stuff from github, bioconductor etc… without checking). Yes, these are huge security concerns and every admin is advised to check the code beforehand. In case of binaries its hard or not possible at al. That's one reason I want to discuss that issue. I'm not sure it is comparable to a entire Linux distribution, its more like an Appstore, like pypi, bioconductor or gems, and yes that is The app stores are checked by Apple or google for malicious code, the apps are sandboxed. There are many eyes for python, bioconductor packages and gems because much more people interact with them directly compared to galaxy-tools. Sure, the Galaxy Tool Shed is slowly getting there. The IUC (Intergalactic Utilities Commission) was founded in the end of 2012 and should be something like a reviewing instance for tools. Sorry maybe I was misleading. I only want a central storage for binaries/tarballs where the source can not be trusted for long term. 'long term' and 'trusted' needs to be defined in such a discussion here. I do not think we should copy python packages that are stored in pypi. We should make it easy as possible to install them in our repository. If you do not trust pypi, we can offer a mirror. Some goes for gems. Trusted for me means I trust the source not having dangerous code. I trust pypi more than some mirror, bioconductor base packages from more than some freshly published package that few people have used, tools from galaxy core developers more than from tool-shed etc… I know this is not the type of trust you were talking about. That is, its twofold. One to trust the source to not infiltrate the system or do any harm, the other part is to trust the availability of data. Both are important imho. Cheers, Bjoern best, ido ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] UnboundLocalError: local variable 'prior_installation_required' referenced before assignment
Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. Salve! Bjoern Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Hi Carlos, Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. That is new to me. How can that happen? Anyone with an OS-X can confirm that? On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? ATLAS (once untared ;-)) needs cpu-throttling to be disabled to optimizes its library. If it is not disabled ATLAS compilation will fail. For OS-X I found that: http://apple.stackexchange.com/questions/41045/how-can-i-disable-cpu-throttling-and-cpu-disabling Sorry, I never touched a OS-X. Nevertheless, If its not disabled, it is supposed to fail silently and downstream packages will not be affected. But if its crashing during untarring, I can't do much. What I can do is to repack the tarball and host it elsewhere. What brings me to: http://gmod.827538.n3.nabble.com/RFC-Storing-of-tarballs-and-patches-for-tool-dependencies-to-enable-reproducibility-td4036591.html Bad news for a Friday morning, sorry :( Bjoern Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Yes this tar is broken at least on OSX. Other people have the same issue: http://code.google.com/p/libarchive/issues/detail?id=299 On Sep 20, 2013, at 10:41 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Carlos, Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. That is new to me. How can that happen? Anyone with an OS-X can confirm that? On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? ATLAS (once untared ;-)) needs cpu-throttling to be disabled to optimizes its library. If it is not disabled ATLAS compilation will fail. For OS-X I found that: http://apple.stackexchange.com/questions/41045/how-can-i-disable-cpu-throttling-and-cpu-disabling Sorry, I never touched a OS-X. Nevertheless, If its not disabled, it is supposed to fail silently and downstream packages will not be affected. But if its crashing during untarring, I can't do much. What I can do is to repack the tarball and host it elsewhere. What brings me to: http://gmod.827538.n3.nabble.com/RFC-Storing-of-tarballs-and-patches-for-tool-dependencies-to-enable-reproducibility-td4036591.html Bad news for a Friday morning, sorry :( Bjoern Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Hi Ido and Carlos, can you check if that tarball is working? http://downloads.sourceforge.net/project/math-atlas/Developer%20% 28unstable%29/3.11.11/atlas3.11.11.tar.bz2 The chance is low, but if its working for you I will consider to create a new version for it. Thanks, Bjoern Yes this tar is broken at least on OSX. Other people have the same issue: http://code.google.com/p/libarchive/issues/detail?id=299 On Sep 20, 2013, at 10:41 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Carlos, Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. That is new to me. How can that happen? Anyone with an OS-X can confirm that? On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? ATLAS (once untared ;-)) needs cpu-throttling to be disabled to optimizes its library. If it is not disabled ATLAS compilation will fail. For OS-X I found that: http://apple.stackexchange.com/questions/41045/how-can-i-disable-cpu-throttling-and-cpu-disabling Sorry, I never touched a OS-X. Nevertheless, If its not disabled, it is supposed to fail silently and downstream packages will not be affected. But if its crashing during untarring, I can't do much. What I can do is to repack the tarball and host it elsewhere. What brings me to: http://gmod.827538.n3.nabble.com/RFC-Storing-of-tarballs-and-patches-for-tool-dependencies-to-enable-reproducibility-td4036591.html Bad news for a Friday morning, sorry :( Bjoern Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Thanks for posting that Ido, Right now I would suggest manually installing these dependencies, rather than ticking the box for the Tool Shed to do it for you. If you are using the Apple provided Python to run your Galaxy on Mac OS X, it comes with NumPy anyway, so compiling Biopython should be simple (just install XCode and the command line tools first). Note that I have not yet updated my Biopython using tools on the main tool shed to include the automatic dependency on the Tool Shed Biopython package, so they require a manual install for now. As soon as the NumPy/ATLAS packaging issue is solved, I plan to push those updates from the Test Tool Shed though. Peter On Fri, Sep 20, 2013 at 10:04 AM, Ido Tamir ta...@imp.ac.at wrote: Yes this tar is broken at least on OSX. Other people have the same issue: http://code.google.com/p/libarchive/issues/detail?id=299 On Sep 20, 2013, at 10:41 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Carlos, Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. That is new to me. How can that happen? Anyone with an OS-X can confirm that? On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? ATLAS (once untared ;-)) needs cpu-throttling to be disabled to optimizes its library. If it is not disabled ATLAS compilation will fail. For OS-X I found that: http://apple.stackexchange.com/questions/41045/how-can-i-disable-cpu-throttling-and-cpu-disabling Sorry, I never touched a OS-X. Nevertheless, If its not disabled, it is supposed to fail silently and downstream packages will not be affected. But if its crashing during untarring, I can't do much. What I can do is to repack the tarball and host it elsewhere. What brings me to: http://gmod.827538.n3.nabble.com/RFC-Storing-of-tarballs-and-patches-for-tool-dependencies-to-enable-reproducibility-td4036591.html Bad news for a Friday morning, sorry :( Bjoern Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client.
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
tar xvfj atlas3.11.11.tar.bz2 shows no errors on OSX and creates one ATLAS folder. best, ido On Sep 20, 2013, at 11:17 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Ido and Carlos, can you check if that tarball is working? http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2 The chance is low, but if its working for you I will consider to create a new version for it. Thanks, Bjoern Yes this tar is broken at least on OSX. Other people have the same issue: http://code.google.com/p/libarchive/issues/detail?id=299 On Sep 20, 2013, at 10:41 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Carlos, Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. That is new to me. How can that happen? Anyone with an OS-X can confirm that? On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? ATLAS (once untared ;-)) needs cpu-throttling to be disabled to optimizes its library. If it is not disabled ATLAS compilation will fail. For OS-X I found that: http://apple.stackexchange.com/questions/41045/how-can-i-disable-cpu-throttling-and-cpu-disabling Sorry, I never touched a OS-X. Nevertheless, If its not disabled, it is supposed to fail silently and downstream packages will not be affected. But if its crashing during untarring, I can't do much. What I can do is to repack the tarball and host it elsewhere. What brings me to: http://gmod.827538.n3.nabble.com/RFC-Storing-of-tarballs-and-patches-for-tool-dependencies-to-enable-reproducibility-td4036591.html Bad news for a Friday morning, sorry :( Bjoern Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To
Re: [galaxy-dev] Tool Shed packages for BLAST+ binaries
On Wed, Sep 18, 2013 at 11:47 AM, Peter Cock p.j.a.c...@googlemail.com wrote: On Tue, Aug 27, 2013 at 2:18 PM, Dave Bouvier d...@bx.psu.edu wrote: Peter, I also tried running the command that returns error code 64 on the same system that runs the automated tests, and it downloaded the correct file for that operating system and architecture. So I'm not sure why it's failing when run through buildbot, but I'll look into it and get back to you. I've added a trello card to track progress on this issue: https://trello.com/c/9ERDMc7j/1079-toolshed-investigate-cause-of-shell-commands-failing-through-buildbot-when-they-are-valid-commands --Dave B. Hi Dave, Something seems to have changed again on the Test Tool Shed, http://testtoolshed.g2.bx.psu.edu/view/peterjc/ncbi_blast_plus/4ae1bac976f3 Installation errors - no functional tests were run for any tools in this changeset revision Tool dependencies TypeNameVersion blast+ package 2.2.26+ Error /bin/sh: 2: [[: not found /bin/sh: 3: [[: not found /bin/sh: 4: [[: not found /bin/sh: 6: [[: not found /bin/sh: 7: [[: not found tar: option requires an argument -- 'f' Try `tar --help' or `tar --usage' for more information. Is the test cluster really missing /bin/sh? Can you check that and post the full installation log please? I really would like to push an update to ncbi_blast_plus to the main Tool Shed with an updated README file (using RST) and the new citation information - plus of course handling the binary dependency via the new package: http://toolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 http://testtoolshed.g2.bx.psu.edu/view/iuc/package_blast_plus_2_2_26 Once this is done there is a backlog of updates I want to tackle following our productive BoF meeting at GCC2013: http://wiki.galaxyproject.org/Events/GCC2013/BoF/GalaxyBlast Thanks, Peter A related bug for Greg (I think), despite this strange no /bin/sh error, this repository is not listed on the Test Tool Shed under Latest revision: installation errors. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
On Fri, Sep 20, 2013 at 10:17 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi Ido and Carlos, can you check if that tarball is working? http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2 The chance is low, but if its working for you I will consider to create a new version for it. Thanks, Bjoern Hi Bjoern, That archive file works for me on Mac OS X 10.8.5, details below. Peter -- I was fooled for a moment, it is one of the annoying SourceForge URLs which doesn't work at the command line - their mirror system is horrid: $ curl -O http://downloads.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2 % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 So, getting a real link via my browser: $ curl -O http://kent.dl.sourceforge.net/project/math-atlas/Developer%20%28unstable%29/3.11.11/atlas3.11.11.tar.bz2 % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 5115k 100 5115k0 0 1267k 0 0:00:04 0:00:04 --:--:-- 1272k $ tar -jxvf atlas3.11.11.tar.bz2 ... $ cd ATLAS/ $ ./configure ATLAS can no longer be configured in the exact source directory, create a subdir such as MyObj. See ATLAS/INSTALL.txt for help. $ mkdir my_build_dir ; cd my_build_dir $ ../configure gcc -I/tmp/x/ATLAS/my_build_dir/..//CONFIG/include -g -w -c /tmp/x/ATLAS/my_build_dir/..//CONFIG/src/atlconf_misc.c gcc -I/tmp/x/ATLAS/my_build_dir/..//CONFIG/include -g -w -o xconfig /tmp/x/ATLAS/my_build_dir/..//CONFIG/src/config.c atlconf_misc.o ./xconfig -d s /tmp/x/ATLAS/my_build_dir/../ -d b /tmp/x/ATLAS/my_build_dir OS configured as OSX (12) Assembly configured as GAS_x8664 (2) Vector ISA Extension configured as SSE3 (6,448) Architecture configured as Core2 (25) Clock rate configured as 2800Mhz Maximum number of threads configured as 8 Parallel make command configured as '$(MAKE) -j 8' Cannot detect CPU throttling. ... $ time make make -f Make.top build cd bin/ ; make xatlas_build /usr/bin/gcc -DL2SIZE=33554432 -I/tmp/x/ATLAS/my_build_dir/include -I/tmp/x/ATLAS/my_build_dir/..//include -I/tmp/x/ATLAS/my_build_dir/..//include/contrib -DAdd_ -DF77_INTEGER=int -DStringSunStyle -DATL_OS_OSX -DATL_ARCH_Core2 -DATL_CPUMHZ=2800 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 -m64 -DATL_NCPU=8 -O -fomit-frame-pointer -m64 -c /tmp/x/ATLAS/my_build_dir/..//bin/atlas_tee.c /usr/bin/gcc -DL2SIZE=33554432 -I/tmp/x/ATLAS/my_build_dir/include -I/tmp/x/ATLAS/my_build_dir/..//include -I/tmp/x/ATLAS/my_build_dir/..//include/contrib -DAdd_ -DF77_INTEGER=int -DStringSunStyle -DATL_OS_OSX -DATL_ARCH_Core2 -DATL_CPUMHZ=2800 -DATL_SSE3 -DATL_SSE2 -DATL_SSE1 -DATL_USE64BITS -DATL_GAS_x8664 -m64 -DATL_NCPU=8 -O -fomit-frame-pointer -m64 -o xatlas_tee atlas_tee.o ... cc1: error: unrecognized command line option -mcpu=ultrasparc FlagCheck.c:1: error: bad value (ultrasparc) for -mtune= switch cc1: error: unrecognized command line option -maltivec cc1: error: unrecognized command line option -mabi=altivec cc1: error: unrecognized command line option -mcpu=ultrasparc cc1: error: unrecognized command line option -mcpu=ultrasparc cc1: error: unrecognized command line option -mcpu=ultrasparc cc1: error: unrecognized command line option -mcpu=970 cc1: error: unrecognized command line option -mips4 cc1: error: unrecognized command line option -mips4 cc1: error: unrecognized command line option -mips4 cc1: error: unrecognized command line option -mips4 cc1: error: unrecognized command line option -mvsx cc1: error: unrecognized command line option -mfpu=vfpv3 ... ATLAS install complete. Examine ATLAS/bin/arch/INSTALL_LOG/SUMMARY.LOG for details. make clean rm -rf *.dSYM rm -f *.o x* config?.out *core* real8m35.518s user7m33.738s sys2m17.387s $ time make test ... 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed 8 cases: 8 passed, 0 skipped, 0 failed 4 cases: 4 passed, 0 skipped, 0 failed DONE SCOPING FOR FAILURES IN CBLAS TESTS: fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \ interfaces/blas/C/testing/sanity.out | \ fgrep -v PASSED make[1]: [sanity_test] Error 1 (ignored) DONE SCOPING FOR FAILURES IN F77BLAS TESTS: fgrep -e fault -e FAULT -e error -e ERROR -e fail -e FAIL \ interfaces/blas/F77/testing/sanity.out | \ fgrep -v PASSED make[1]: [sanity_test] Error 1 (ignored) DONE real0m38.065s user0m27.856s sys0m6.329s
[galaxy-dev] pgcleanup problem
Hi All, I'm trying to do some cleanup in my test environment (galaxy-dist) and pgcleanup.py ends with Traceback (most recent call last): File ./scripts/cleanup_datasets/pgcleanup.py, line 773, in module cleanup = Cleanup() File ./scripts/cleanup_datasets/pgcleanup.py, line 55, in __init__ self.__connect_db() File ./scripts/cleanup_datasets/pgcleanup.py, line 114, in __connect_db self.conn = psycopg2.connect(**args) TypeError: 'username' is an invalid keyword argument for this function Unless I add args['user'] = args['username'] del args['username'] Is this a bug or a config error? Eric ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
Hi, I tried to things to solve it. 1: I uploaded a new version to: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_10 this time with target_filename=ATLAS.tar.bz2. That is a rather new feature, so you will need a recent Galaxy version. The prosite is that we do not need to call 'tar', we are using the python tarfile module. Hopefully that will work, also on OS-X 2: I have created: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_11 Peter was able to install it on OS-X, at least on commandline, so I have hope that this one is working, well. The downside is, that it is a unstable version and will disappear from the server. Would be great if you can give both a try. For me its working. That means on ubuntu its installed and on Fedora its failing silently with 'It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information'. Thanks for your help! Bjoern On Thu, Sep 19, 2013 at 5:15 PM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Peter and Carlos, On Mon, Sep 16, 2013 at 8:57 PM, Carlos Borroto carlos.borr...@gmail.com wrote: I did an extra test. Started with a clean 'galaxy-dist'. This time both repositories fail with the same error. I guess before something was cached for the repository with version 0.1.4. I used biopython repository as a guide to write my tool dependency definition: http://testtoolshed.g2.bx.psu.edu/view/biopython/package_biopython_1_61 I can confirm biopython repository is failing to install for me with exactly the same error. I wonder if a recently addition in the test toolshed broke the treatment of prior_installation_required. Thanks, Carlos Could be - note that currently Biopython isn't currently installing properly on the Test Tool Shed due to ALTAS failing (a requirement of NumPy which is a requirement of Biopython). Dave and Bjoern are I think looking at this already... I can't do much I tested it again and for me its working fine on my computers I have at hand ... sorry. In case it helps, this is how the INSTALLATION.log file ends on OS X 10.8.4: $ tail -n 3 ~/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/3508de0ebae1/INSTALLATION.log x ATLAS/tune/lapack/lanbsrch.c tar: Error exit delayed from previous errors. # This is the relevant part I can find in Galaxy's log: [localhost] local: tar xfvj atlas3.10.1.tar.bz2 Warning: local() encountered an error (return code 1) while executing 'tar xfvj atlas3.10.1.tar.bz2' After noticing this I got what I'm guessing is the original file from sourceforge: http://downloads.sourceforge.net/project/math-atlas/Stable/3.10.1/atlas3.10.1.tar.bz2 I can confirm that trying to untar this file fails with the exact same error. However, on Ubuntu 13.04 untaring this file works just fine. On Ubuntu 13.04 the error I see is: STDERR It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information # Björn, you said cpu throttling can be easily disable on Ubuntu. Can you comment how? Do I need to disable it completely, or should the install script do it when installing? Thanks, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Using tool_dependencies.xml files directly via Galaxy API
Hello all, I'm looking at how to automatically script setting up a Galaxy instance with selected tools with dependencies as part of automated testing [*]. Can the Galaxy API be used to run the dependency instructions in a tool_dependencies.xml file (without using a Tool Shed)? Or more generally, can the Galaxy API be used to install a Tool Shed package from disk (i.e. point it at the files which would normally be uploaded to a Tool Shed)? Or, for an automated installation of local tools must I first setup a local Tool Shed and upload the tools there? Thanks, Peter [*] I'm using the TravisCI system for automated testing of my Galaxy tools now hosted on GitHub, which works by running the tests on a clean virtual machine: http://blastedbio.blogspot.com/2013/09/using-travis-ci-for-testing-galaxy-tools.html This means I need to automate the installation of Galaxy (easy), the tools I wish to test and their dependencies. This configuration would be less effort (and a more useful test) if it could mimic tool and dependency installation from the Tool Shed by reading my dependency XML files. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
On Fri, Sep 20, 2013 at 1:48 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi, I tried to things to solve it. Was this specifically to solve ATLAS under Mac OS X, or more generally include the problem shown on the Galaxy (Test) Tool Shed? 1: I uploaded a new version to: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_10 this time with target_filename=ATLAS.tar.bz2. That is a rather new feature, so you will need a recent Galaxy version. The prosite is that we do not need to call 'tar', we are using the python tarfile module. Hopefully that will work, also on OS-X Not tried. 2: I have created: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_11 Peter was able to install it on OS-X, at least on commandline, so I have hope that this one is working, well. The downside is, that it is a unstable version and will disappear from the server. Tried on a CentOS machine (running galaxy-central), it appeared to work but on closer inspection: Missing tool dependencies - click the name to install the missing dependency Name Version Type Installation status atlas 3.11.11 package Error $ cat /opt/galaxy-dist-shed-tools/atlas/3.11.11/iuc/package_atlas_3_11/3e1635c33e26/INSTALLATION.log # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDOUT # # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDERR You must be root # Would be great if you can give both a try. For me its working. That means on ubuntu its installed and on Fedora its failing silently with 'It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information'. Like Fedora, on CentOS I am getting a silent failure :( Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
On Fri, Sep 20, 2013 at 1:48 PM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi, I tried to things to solve it. Was this specifically to solve ATLAS under Mac OS X, or more generally include the problem shown on the Galaxy (Test) Tool Shed? Yes. My hope was to use tarfile from python and not 'tar', because bsdtar seems to crash with that package. 1: I uploaded a new version to: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_10 this time with target_filename=ATLAS.tar.bz2. That is a rather new feature, so you will need a recent Galaxy version. The prosite is that we do not need to call 'tar', we are using the python tarfile module. Hopefully that will work, also on OS-X Not tried. 2: I have created: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_11 Peter was able to install it on OS-X, at least on commandline, so I have hope that this one is working, well. The downside is, that it is a unstable version and will disappear from the server. Tried on a CentOS machine (running galaxy-central), it appeared to work but on closer inspection: Missing tool dependencies - click the name to install the missing dependency Name Version Type Installation status atlas 3.11.11 package Error $ cat /opt/galaxy-dist-shed-tools/atlas/3.11.11/iuc/package_atlas_3_11/3e1635c33e26/INSTALLATION.log # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDOUT # # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDERR You must be root # Would be great if you can give both a try. For me its working. That means on ubuntu its installed and on Fedora its failing silently with 'It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information'. Like Fedora, on CentOS I am getting a silent failure :( For whatever reason I get the following in my log. Galaxy tries to continue with the installation, that fine. But in your case Galaxy aborts. Nevertheless, failing silently means, that biopython and numpy will not be affected (hopefully). I can also remove the manuell deactivation of cpu throttling, it was just an idea to support more systems out of the box without admin invention. # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDOUT # # # try to disable cpu throttling if hash cpufreq-selector 2/dev/null; then cpufreq-selector -g performance elif hash cpupower 2/dev/null; then cpupower frequency-set -g performance else echo 'Please deactivate CPU throttling by your own, or install cpufreq-selector' exit fi STDERR Error calling SetGovernor: Caller is not authorized # # mkdir build cd build mkdir /home/gruening/projects/code/galaxy-central/tool_deps/atlas/3.11.11/iuc/package_atlas_3_11/3e1635c33e26/atlas/ ../configure -Fa alg -fPIC --prefix=/home/gruening/projects/code/galaxy-central/tool_deps/atlas/3.11.11/iuc/package_atlas_3_11/3e1635c33e26/atlas/ --with-netlib-lapack-tarfile=../lapack-3.4.2.tgz STDOUT gcc
Re: [galaxy-dev] pass user groups to dynamic job runner
Il giorno gio, 19/09/2013 alle 12.42 -0500, John Chilton ha scritto: Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd Hi John, that's pretty interesting and we are probably going to use something similar for some problematic tools, thanks for sharing! First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. Is there a way to specify in job_conf.xml a per-tool final destination after the has_license filtering ? E.g.: tool id=cat1 destination=has_license final_destination=local / tool id=cat2 destination=has_license final_destination=remote_cluster / The 'final_destination' attribute would then be used in has_license function instead of returning DEFAULT_JOB_DESTINATION_ID. There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. I think that something like this: # Tool filtering. Multiple such filters can be specified by giving # module (relative to galaxy.tools.filters) and function names for each # as a comma separated list. # See lib/galaxy/tools/filters/examples.py.sample for some examples. #tool_filters = #tool_label_filters = #tool_section_filters = should be added to universe_wsgi.ini.sample , otherwise this wonderful feature is very difficult to discover. Hope this helps! -John Thanks again! Nicola On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert geert.vandewe...@uantwerpen.be wrote: Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok = run ; fail = give message. Or can I hide tools from the menu based on usergroup? Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Test Toolshed Biopython package dependency Atlas fails to install (Was: Re: UnboundLocalError: local variable 'prior_installation_required' referenced before assignment)
On Fri, Sep 20, 2013 at 8:48 AM, Bjoern Gruening bjoern.gruen...@gmail.com wrote: Hi, I tried to things to solve it. 1: I uploaded a new version to: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_10 this time with target_filename=ATLAS.tar.bz2. That is a rather new feature, so you will need a recent Galaxy version. The prosite is that we do not need to call 'tar', we are using the python tarfile module. Hopefully that will work, also on OS-X 2: I have created: http://testtoolshed.g2.bx.psu.edu/view/iuc/package_atlas_3_11 Peter was able to install it on OS-X, at least on commandline, so I have hope that this one is working, well. The downside is, that it is a unstable version and will disappear from the server. Would be great if you can give both a try. For me its working. That means on ubuntu its installed and on Fedora its failing silently with 'It appears you have cpu throttling enabled, which makes timings unreliable and an ATLAS install nonsensical. Aborting. See ATLAS/INSTALL.txt for further information'. I don't have access to my OS X box until later tonight. On Ubuntu with an up-to-date instance of galaxy-central is failing now for me with a different error: STDOUT ./xconfig -d s /home/cborroto/src/galaxy-central/database/tmp/tmp-toolshed-mtdyWTv64/ATLAS/build/../ -d b /home/cborroto/src/galaxy-central/database/tmp/tmp-toolshed-mtdyWTv64/ATLAS/build -Fa alg -fPIC -Si lapa ckref 1 xconfig exited with 127 # # mkdir build cd build mkdir /home/cborroto/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/d32504e1aea8/atlas/ ../configure -Fa alg -fPIC --prefix=/home/cborroto/src/shed_tools_dependencies.central/atlas/3.10.1/iuc/package_atlas_3_10/d32504e1aea8/atlas/ --with-netlib-lapack-tarfile=../lapack-3.4.2.tgz STDERR cat: ..//CONFIG/src/Makefile: No such file or directory cp: cannot stat ‘..//CONFIG/src/atlcomp.txt’: No such file or directory make: *** No rule to make target `xconfig'. Stop. /bin/sh: 1: ./xconfig: not found Full INSTALLATION.log: http://pastebin.com/raw.php?i=hvqFWvgJ Somehow this is also a silent failing and numpy, biopython and ngs-tools(my tool package depending on biopython) get Installed(Green) while lapack, atlas and split_by_barcode(my actual tool wrapper depending on ngs-tools package) get Installed, missing tool dependencies(Grey). This means if I try to use my wrapper in this state I get this error: /bin/sh: 1: ngs-tools: not found However, if I do a Repair repository on split_by_barcode it goes into Installed(Green) and everything seems to work from then on. Thanks for working on this, Carlos ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] testtoolshed.g2.bx.psu.edu not compatible with galaxy-central stable
Hi Rico, The test tool shed is now running my latest commit - 10629:ab20415126a7. I was successful with installing the genome diversity repository and all of it's dependencies using that changeset in my local Galaxy environment (although 3 tool dependencies encountered the following errors while attempting to compile). Can you try again and let me know if you encounter additonal problems? atlas 3.10.1 package Error cat: ..//CONFIG/src/Makefile: No such file or directory cp: ..//CONFIG/src/atlcomp.txt: No such file or directory make: *** No rule to make target `xconfig'. Stop. /bin/sh: line 1: ./xconfig: No such file or directory lapack 3.4.2 package Error CMake Error: your Fortran compiler: CMAKE_Fortran_COMPILER-NOTFOUND was not found. Please set CMAKE_Fortran_COMPILER to a valid compiler path or name. CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed phast 1.3 package Error make[1]: *** No rule to make target `/Users/gvk/workspace/tool_dependencies/clapack/3.2.1/rico/package_clapack_3_2_1/56a949e5f998/lapack.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/lib/../../lib/liblapack.a'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/dless/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/dless/../../bin/dless'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/exoniphy/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/exoniphy/../../bin/exoniphy'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastCons/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastCons/../../bin/phastCons'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastOdds/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastOdds/../../bin/phastOdds'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastMotif/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastMotif/../../bin/phastMotif'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phyloFit/../../lib/libphast.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phyloFit/../../bin/phyloFit'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phyloBoot/../../lib/libphast.a', needed by
Re: [galaxy-dev] pass user groups to dynamic job runner
Thanks, I integrated parts of these snippets into our general dynamic_jobs function, and it works flawlessly. Best, Geert Van: jmchil...@gmail.com [jmchil...@gmail.com] namens John Chilton [chil...@msi.umn.edu] Verzonden: donderdag 19 september 2013 19:42 Aan: Vandeweyer Geert CC: galaxy-...@bx.psu.edu Onderwerp: Re: [galaxy-dev] pass user groups to dynamic job runner Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. This will do what you asked, you can adjust the tools it targets and the message that gets displayed to the user pretty easily (right now it is just No license, no tool). There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. Hope this helps! -John On Thu, Sep 19, 2013 at 2:45 AM, Vandeweyer Geert geert.vandewe...@uantwerpen.be wrote: Hi, Could somebody point me to the place where I can create a ticket for the following feature: - We want to have tools available only to users who provided a licence for this tool. - To prevent very long email lists (see example on dev-only tools), I'd like to have a group 'have_licence' - in dynamic job runner function : check if user is in usergroup : ok = run ; fail = give message. Or can I hide tools from the menu based on usergroup? Best, Geert ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] testtoolshed.g2.bx.psu.edu not compatible with galaxy-central stable
The default branch ( i.e., the galaxy-central repository on bitbucket) is backward compatible to the December 20, 2012 Galaxy release with regard to communication between a Galaxy instance and a Tool Shed instance. It is not guaranteed that this communication will be forward compatible as new features are continually introduced. Since the test Tool Shed is generally running the galaxy-central tip, it is most likely necessary to run that in your local Galaxy instance if you want to use the new features running on the test Tool Shed that have been introduced after the December 20, 2012 Galaxy release. With regard to forward-compatiblity between Galaxy and the Tool Shed, there is a Trello card here: https://trello.com/c/cd3hBnnH/30-tool-shed-decoupling-galaxy-and-tool-shed-versions Greg Von Kuster On Sep 20, 2013, at 1:00 PM, Richard Burhans r...@bx.psu.edu wrote: Greg, I still get the same traceback. The issue is with installing a repository from testtoolshed, which is running code on the default branch, to a galaxy instance running on the stable branch. My understanding was that the default branch is supposed to be backward-compatible to the stable branch. Please let me know if this is not the case (which would mean that this is not a bug). The atlas and lapack errors are begin caused by http://testtoolshed.g2.bx.psu.edu/view/iuc/package_numpy_1_7. I'm cc'ing galaxy-iuc so they can look into it further. Thanks for the report on phast. I'll probably need to use the architecture-dependent compilation stuff you're working on to fix this. -rico On Sep 20, 2013, at 11:21 AM, Greg Von Kuster wrote: Hi Rico, The test tool shed is now running my latest commit - 10629:ab20415126a7. I was successful with installing the genome diversity repository and all of it's dependencies using that changeset in my local Galaxy environment (although 3 tool dependencies encountered the following errors while attempting to compile). Can you try again and let me know if you encounter additonal problems? atlas3.10.1 package Error cat: ..//CONFIG/src/Makefile: No such file or directory cp: ..//CONFIG/src/atlcomp.txt: No such file or directory make: *** No rule to make target `xconfig'. Stop. /bin/sh: line 1: ./xconfig: No such file or directory lapack 3.4.2 package Error CMake Error: your Fortran compiler: CMAKE_Fortran_COMPILER-NOTFOUND was not found. Please set CMAKE_Fortran_COMPILER to a valid compiler path or name. CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMak! eFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after E! nableLanguage CMake Error: Internal CMake error, TryCompile co! nfigure of cmake failed phast1.3 package Error make[1]: *** No rule to make target `/Users/gvk/workspace/tool_dependencies/clapack/3.2.1/rico/package_clapack_3_2_1/56a949e5f998/lapack.a', needed by `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/lib/../../lib/liblapack.a'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/dless/../../lib/libphast.a', needed by
Re: [galaxy-dev] testtoolshed.g2.bx.psu.edu not compatible with galaxy-central stable
Greg, I still get the same traceback. The issue is with installing a repository from testtoolshed, which is running code on the default branch, to a galaxy instance running on the stable branch. My understanding was that the default branch is supposed to be backward- compatible to the stable branch. Please let me know if this is not the case (which would mean that this is not a bug). The atlas and lapack errors are begin caused by http:// testtoolshed.g2.bx.psu.edu/view/iuc/package_numpy_1_7. I'm cc'ing galaxy-iuc so they can look into it further. Thanks for the report on phast. I'll probably need to use the architecture-dependent compilation stuff you're working on to fix this. -rico On Sep 20, 2013, at 11:21 AM, Greg Von Kuster wrote: Hi Rico, The test tool shed is now running my latest commit - 10629:ab20415126a7. I was successful with installing the genome diversity repository and all of it's dependencies using that changeset in my local Galaxy environment (although 3 tool dependencies encountered the following errors while attempting to compile). Can you try again and let me know if you encounter additonal problems? atlas 3.10.1 package Error cat: ..//CONFIG/src/Makefile: No such file or directory cp: ..//CONFIG/src/atlcomp.txt: No such file or directory make: *** No rule to make target `xconfig'. Stop. /bin/ sh: line 1: ./xconfig: No such file or directory lapack 3.4.2 package Error CMake Error: your Fortran compiler: CMAKE_Fortran_COMPILER-NOTFOUND was not found. Please set CMAKE_Fortran_COMPILER to a valid compiler path or name. CMake Error at /opt/local/share/cmake-2.8/Modules/ CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/ CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/ CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/ CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed CMake Error at /opt/local/share/cmake-2.8/Modules/ CMakeFortranInformation.cmake:27 (get_filename_component): get_filename_component called with incorrect number of arguments Call Stack (most recent call first): CMakeLists.txt:2 (project) CMake Error: CMAKE_Fortran_COMPILER not set, after EnableLanguage CMake Error: Internal CMake error, TryCompile configure of cmake failed phast 1.3 package Error make[1]: *** No rule to make target `/Users/ gvk/workspace/tool_dependencies/clapack/3.2.1/rico/ package_clapack_3_2_1/56a949e5f998/lapack.a', needed by `/Users/gvk/ workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/ phast-1.3/src/lib/../../lib/liblapack.a'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/ tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/dless/../../lib/ libphast.a', needed by `/Users/gvk/workspace/galaxy-central/ database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/dless/../../bin/ dless'. Stop. make[1]: *** No rule to make target `/Users/gvk/ workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/ phast-1.3/src/exoniphy/../../lib/libphast.a', needed by `/Users/ gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/ phast-1.3/src/exoniphy/../../bin/exoniphy'. Stop. make[1]: *** No rule to make target `/Users/gvk/workspace/galaxy-central/database/ tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastCons/../../lib/ libphast.a', needed by `/Users/gvk/workspace/galaxy-central/ database/tmp/tmp-toolshed-mtdM9orSY/phast-1.3/src/phastCons/../../ bin/phastCons'. Stop. make[1]: *** No rule to make target `/Users/ gvk/workspace/galaxy-central/database/tmp/tmp-toolshed-mtdM9orSY/ phast-1.3/src/phastOdds/../../lib/libphast.a', needed by `/Users/
Re: [galaxy-dev] Tool Shed: get_readme_files bug
Hello Rico, I'm not seeing a problem here. The revision of the genome_diversity you're referring to is the repository tip on the main tool shed. The README file on disk has this content: g2cmnty@montana:~/galaxy_toolshed/database/community_files/000/repo_200$ cat README Source code for the executables needed by these tools can be found in the genome_diversity directory. Additionally, you'll need the following python modules: matplotlib (we used version 1.1.0) http://pypi.python.org/packages/source/m/matplotlib/ mechanize (we used version 0.2.5) http://pypi.python.org/packages/source/m/mechanize/ networkx (we used version 1.6) http://pypi.python.org/packages/source/n/networkx/ fisher (we used version 0.1.4) http://pypi.python.org/packages/source/f/fisher/ And the following software: ADMIXTURE (we used version 1.22) http://www.genetics.ucla.edu/software/admixture/ EIGENSOFT (we used version 3.0) http://genepath.med.harvard.edu/~reich/Software.htm PHAST (we used version 1.2.1) http://compgen.bscb.cornell.edu/phast/ QuickTree (we used version 1.1) http://www.sanger.ac.uk/resources/software/quicktree/ Images used in the tools' documentation are located in the static/images directory. Please copy these to the static/images directory in your Galaxy installation. and the browser displays the following for the same revision: README README Source code for the executables needed by these tools can be found in the genome_diversity directory. Additionally, you'll need the following python modules: matplotlib (we used version 1.1.0) http://pypi.python.org/packages/source/m/matplotlib/ mechanize (we used version 0.2.5) http://pypi.python.org/packages/source/m/mechanize/ networkx (we used version 1.6) http://pypi.python.org/packages/source/n/networkx/ fisher (we used version 0.1.4) http://pypi.python.org/packages/source/f/fisher/ And the following software: ADMIXTURE (we used version 1.22) http://www.genetics.ucla.edu/software/admixture/ EIGENSOFT (we used version 3.0) http://genepath.med.harvard.edu/reich/Software.htm PHAST (we used version 1.2.1) http://compgen.bscb.cornell.edu/phast/ QuickTree (we used version 1.1) http://www.sanger.ac.uk/resources/software/quicktree/ Images used in the tools' documentation are located in the static/images directory. Please copy these to the static/images directory in your Galaxy installation. In addition, the Readme container will display the correct content of a README file that is associated with the selected changeset revision. Can you provide specific changeset revisions of a repository where this is not the case? The following URL is not a proper URL for browsing a tool shed repository Readme file via a browser. http://toolshed.g2.bx.psu.edu/repos/miller-lab/genome_diversity/file/5064f618ec1c/README Greg Von Kuster On Sep 20, 2013, at 4:26 PM, Richard Burhans r...@bx.psu.edu wrote: It appears that the Tool Shed does not show the correct readme file content. 1) get_readme_files() in lib/galaxy/webapps/tool_shed/controllers/repository.py gets the correct metadata from the database 2) build_readme_files_dict() in lib/tool_shed/util/readme_util.py reads data from whatever revision happens to be there (not the requested revision) As an example, compare the output from: http://toolshed.g2.bx.psu.edu/repository/get_readme_files?name=genome_diversityowner=miller-labchangeset_revision=5064f618ec1c and: http://toolshed.g2.bx.psu.edu/repos/miller-lab/genome_diversity/file/5064f618ec1c/README -rico ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] pass user groups to dynamic job runner
On Fri, Sep 20, 2013 at 9:08 AM, Nicola Soranzo sora...@crs4.it wrote: Il giorno gio, 19/09/2013 alle 12.42 -0500, John Chilton ha scritto: Feature request received. The following changeset demonstrates how one could implement this https://github.com/jmchilton/galaxy-central/commit/b500a24bc1aa94bef48db20a7cc1f3d68855b7fd Hi John, that's pretty interesting and we are probably going to use something similar for some problematic tools, thanks for sharing! First step is to create a new dynamic job destination, this is demonstrated in job_conf.xml in the above changeset. This demonstrates the new style job_conf section - this will need to be adapted for the old style runners but is still doable. Then you will want to add a dynamic job runner rule that implements this logic. This is the file lib/galaxy/jobs/rules/license_checker.py. Is there a way to specify in job_conf.xml a per-tool final destination after the has_license filtering ? E.g.: tool id=cat1 destination=has_license final_destination=local / tool id=cat2 destination=has_license final_destination=remote_cluster / The 'final_destination' attribute would then be used in has_license function instead of returning DEFAULT_JOB_DESTINATION_ID. There is no way currently to pass information from the tool tag to the dynamic runner or to have the dynamic runner pass on making a decision in some way. It could potentially make sense to add these - but I would put forward a workaround that in my opinion is actually a little better: from galaxy.jobs.mapper import JobMappingException DEFAULT_DESTINATION_ID = remoteCluster LOCAL_DESTINATION_ID = local LOCAL_TOOLS = ['cat1'] def has_license(user, tool_id): user_group_assocs = user.groups or [] user_has_license = 'have_license' in [user_group_assoc.group.name for user_group_assoc in user_group_assocs] if not user_has_license: raise JobMappingException(No license, no tool!) if tool_id in LOCAL_TOOLS: return LOCAL_DESTINATION_ID else: return DEFAULT_DESTINATION_ID The problem here is that you sort of have special logic for 'cat1' in two places so what I would probably do is actually just eliminate the tool mappings all together in job_conf.xml and assign these tools destination using a default dynamic job runner. from galaxy.jobs.mapper import JobMappingException DEFAULT_DESTINATION_ID = remoteCluster LOCAL_DESTINATION_ID = local LOCAL_TOOLS = ['cat1'] REQUIRES_LICENSE = ['cat1', 'cat2'] def default_mapper(user, tool_id): if tool_id in REQUIRES_LICENSE: _check_license(user) if tool_id in LOCAL_TOOLS: return LOCAL_DESTINATION_ID else: return DEFAULT_DESTINATION_ID def _check_license(user): user_group_assocs = user.groups or [] user_has_license = 'have_license' in [user_group_assoc.group.name for user_group_assoc in user_group_assocs] if not user_has_license: raise JobMappingException(No license, no tool!) I don't use the dynamic job runner personally, but I am nevertheless going to declare it a best practice that at some point the complexity of the rules is such that it just makes sense to stop assigning any tools mappings in job_conf.xml and just use a default dynamic job destination for everything. There is no longer any need to display such tools to the user thanks to dynamic toolbox filters. This is that last file: lib/galaxy/tools/filters/license_filter.py. This will prevent any tool in the list 'LICENSED_TOOLS' from even being seen by unlicensed users. You do need to specify this filter is being used by adding the line: tool_filters = license_filter:has_license in the [app:main] section of universe_wsgi.ini. I think that something like this: # Tool filtering. Multiple such filters can be specified by giving # module (relative to galaxy.tools.filters) and function names for each # as a comma separated list. # See lib/galaxy/tools/filters/examples.py.sample for some examples. #tool_filters = #tool_label_filters = #tool_section_filters = should be added to universe_wsgi.ini.sample , otherwise this wonderful feature is very difficult to discover. Do you mean to suggest that merely documenting my contributions to Galaxy on my Twitter feed is insufficient? Hmm I don't know about that... Seriously though, Björn has implemented some extensions to dynamic toolbox filters that include this: https://bitbucket.org/galaxy/galaxy-central/pull-request/179/implement-the-ability-to-change-the-tool/diff#chg-universe_wsgi.ini.sample I have added more options to Galaxy then I have documented in that file and I am always unsure whether to add them or not, the hesitation is that I think that file has too cluttered. I have missed really important modifications that should have been made because I didn't see them in that file. These filters may rise to the level that they belong in there, I am not certain, but they should certainly be documented at least on the wiki
