Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Hi, has anything changed in galaxy in this regard? Any way to modify an environment before a tools is run? I now have a tool relying on R-devel and bioconductor devel, both of which I can load in a module. The tool comes from the toolshed with xml like: command interpreter=Rscript … I don't want to hack around in the tool itself, but simply load the necessary R-version. thank you very much, ido On Sep 13, 2013, at 3:23 AM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Hi Ido, I do not get your question in all detail, but it is possible to define a tool_dependencies.xml with a specific R version, + R libraries and use only that specific version from your tool with requirement tags. For an example please see various R tools from: https://github.com/bgruening/galaxytools Cheers, Bjoern Am 25.04.2014 14:25, schrieb Ido Tamir: Hi, has anything changed in galaxy in this regard? Any way to modify an environment before a tools is run? I now have a tool relying on R-devel and bioconductor devel, both of which I can load in a module. The tool comes from the toolshed with xml like: command interpreter=Rscript … I don't want to hack around in the tool itself, but simply load the necessary R-version. thank you very much, ido On Sep 13, 2013, at 3:23 AM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Dear Björn, maybe I could change the tool, which is not mine, and which I don't want to maintain to use a specific R-version that is already available on our cluster and which I can put into my path with module load R/3.1.0-devel ( http://modules.sourceforge.net/) there even is a requirements in the wrapper which is not fulfilled (these versions are not available at the moment) and still it installed without problems. requirements requirement type=R-module version=3.5.27edgeR/requirement requirement type=R-module version=3.18.13limma/requirement /requirements This requirements tags looks also rather inflexible to me. With an additional level of user configurable indirection it would be possible to make the tools fit to different infrastructures without having to use binaries provided by somebody else, taking up space for just one tool etc... Currently for jobs that are not run locally there is a general environment_setup_file. There could be one optional environment_setup_file for every job (or destination). In the end I created yet another wrapper. best, ido On Apr 25, 2014, at 2:34 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ido, I do not get your question in all detail, but it is possible to define a tool_dependencies.xml with a specific R version, + R libraries and use only that specific version from your tool with requirement tags. For an example please see various R tools from: https://github.com/bgruening/galaxytools Cheers, Bjoern Am 25.04.2014 14:25, schrieb Ido Tamir: Hi, has anything changed in galaxy in this regard? Any way to modify an environment before a tools is run? I now have a tool relying on R-devel and bioconductor devel, both of which I can load in a module. The tool comes from the toolshed with xml like: command interpreter=Rscript … I don't want to hack around in the tool itself, but simply load the necessary R-version. thank you very much, ido On Sep 13, 2013, at 3:23 AM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Hello Ido, Reading through your e-mail it is not clear to me whether the tool has requirements that could be overridden. If it has package requirements explicitly defined - those are completely configurable - you can write arbitrary python plugins to resolve these any you wish - and (if the modules are available on the galaxy server) there is an existing plugin that can be used to resolve these packages as modules. I think the best documentation of this may be on this pull request (https://bitbucket.org/galaxy/galaxy-central/pull-request/228/tool-dependency-resolver-plugins-revision/diff). Additionally (perhaps more pertinently), I have an open pull request (created just this week) to allow setting environment variables on a per destination basis - this is sort of what you are getting at and might be a better way to go - https://bitbucket.org/galaxy/galaxy-central/pull-request/378/allow-specification-of-environment. With this modification you could hack up the path to reflect the changes made by the module load. That is a bit of hack but would work for this case. That said I think you make a good point about the environment file, I will modify the new env tags to allow a file attribute and just source that file. This will give you your per destination environment files - hopefully this is satisfactory. -John On Fri, Apr 25, 2014 at 8:22 AM, Ido Tamir ta...@imp.ac.at wrote: Dear Björn, maybe I could change the tool, which is not mine, and which I don't want to maintain to use a specific R-version that is already available on our cluster and which I can put into my path with module load R/3.1.0-devel ( http://modules.sourceforge.net/) there even is a requirements in the wrapper which is not fulfilled (these versions are not available at the moment) and still it installed without problems. requirements requirement type=R-module version=3.5.27edgeR/requirement requirement type=R-module version=3.18.13limma/requirement /requirements This requirements tags looks also rather inflexible to me. With an additional level of user configurable indirection it would be possible to make the tools fit to different infrastructures without having to use binaries provided by somebody else, taking up space for just one tool etc... Currently for jobs that are not run locally there is a general environment_setup_file. There could be one optional environment_setup_file for every job (or destination). In the end I created yet another wrapper. best, ido On Apr 25, 2014, at 2:34 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ido, I do not get your question in all detail, but it is possible to define a tool_dependencies.xml with a specific R version, + R libraries and use only that specific version from your tool with requirement tags. For an example please see various R tools from: https://github.com/bgruening/galaxytools Cheers, Bjoern Am 25.04.2014 14:25, schrieb Ido Tamir: Hi, has anything changed in galaxy in this regard? Any way to modify an environment before a tools is run? I now have a tool relying on R-devel and bioconductor devel, both of which I can load in a module. The tool comes from the toolshed with xml like: command interpreter=Rscript … I don't want to hack around in the tool itself, but simply load the necessary R-version. thank you very much, ido On Sep 13, 2013, at 3:23 AM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
On Apr 25, 2014, at 3:55 PM, John Chilton jmchil...@gmail.com wrote: Additionally (perhaps more pertinently), I have an open pull request (created just this week) to allow setting environment variables on a per destination basis - this is sort of what you are getting at and might be a better way to go - https://bitbucket.org/galaxy/galaxy-central/pull-request/378/allow-specification-of-environment. With this modification you could hack up the path to reflect the changes made by the module load. That is a bit of hack but would work for this case. That said I think you make a good point about the environment file, I will modify the new env tags to allow a file attribute and just source that file. This will give you your per destination environment files - hopefully this is satisfactory. That sounds great! best, ido On Fri, Apr 25, 2014 at 8:22 AM, Ido Tamir ta...@imp.ac.at wrote: Dear Björn, maybe I could change the tool, which is not mine, and which I don't want to maintain to use a specific R-version that is already available on our cluster and which I can put into my path with module load R/3.1.0-devel ( http://modules.sourceforge.net/) there even is a requirements in the wrapper which is not fulfilled (these versions are not available at the moment) and still it installed without problems. requirements requirement type=R-module version=3.5.27edgeR/requirement requirement type=R-module version=3.18.13limma/requirement /requirements This requirements tags looks also rather inflexible to me. With an additional level of user configurable indirection it would be possible to make the tools fit to different infrastructures without having to use binaries provided by somebody else, taking up space for just one tool etc... Currently for jobs that are not run locally there is a general environment_setup_file. There could be one optional environment_setup_file for every job (or destination). In the end I created yet another wrapper. best, ido On Apr 25, 2014, at 2:34 PM, Björn Grüning bjoern.gruen...@gmail.com wrote: Hi Ido, I do not get your question in all detail, but it is possible to define a tool_dependencies.xml with a specific R version, + R libraries and use only that specific version from your tool with requirement tags. For an example please see various R tools from: https://github.com/bgruening/galaxytools Cheers, Bjoern Am 25.04.2014 14:25, schrieb Ido Tamir: Hi, has anything changed in galaxy in this regard? Any way to modify an environment before a tools is run? I now have a tool relying on R-devel and bioconductor devel, both of which I can load in a module. The tool comes from the toolshed with xml like: command interpreter=Rscript … I don't want to hack around in the tool itself, but simply load the necessary R-version. thank you very much, ido On Sep 13, 2013, at 3:23 AM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Just been reading a bit more about the Galaxy packaging system. Here's a slight modification to what I was suggesting that might fit in a bit better. Apologies for not being more familiar with the existing system before proposing extensions. Recall that my goal is to support using a system-installed (native) package, at a defined version, which I aim to achieve by loading the appropriate environment module before running a tool. We still have tool_dependencies.xml defining a package at a particular version, but rather than download and build the source code, there's just a directive that says how to pick up the correct program version at runtime, e.g. which environment module to load. So instead of the tool_dependencies.xml fragment: tool_dependency package name=bwa version=0.6.2 install version=1.0 actions action type=download_by_urlhttp://downloads.sourceforge.net/project/bio-bwa/bwa-0.6.2.tar.bz2/action action type=shell_commandmake/action action type=move_file sourcebwa/source destination$INSTALL_DIR/bin/destination /action action type=set_environment environment_variable name=PATH action=prepend_to$INSTALL_DIR/bin/environment_variable /action /actions /install /package /tool_dependency We have something like this (NB: element and attribute names are for illustrative purposes only): tool_dependency package name=bwa version=0.6.2 use_native actions action type=module_loadbwa/0.6.2/action /actions /use_native /package /tool_dependency This causes the right thing (module load bwa/0.6.2) to be stuck into the dependencies env.sh file when this package is installed from the toolshed. We could call this toolshed tool native_package_bwa_0_6_2, to avoid confusion with the existing download-and-make one. We might want a bit of flexibility on what actions are supported (in case we want to support Software Collections, for example). What do you think? cheers, Simon PS: In case it wasn't already clear, solving this problem well is quite important to us here at AgResearch. ;-) === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
I'd like to propose a slightly different approach. I don't like the idea of introducing a different type of requirement for this because it really isn't a different type of requirement. You are still requiring a package, you just want to use a different mechanism to provide it. So, instead, what about creating a plugin for tool requirement handling? You could replace the default dependency injection approach with one based on modules. So, the tool configurations would not need to change, you would just change the configuration of your instance to inject module loads instead of source env.shs. And yes, we would definitely accept something like this as long as it is a plugin that is not enabled by default. -- James Taylor, Associate Professor, Biology/CS, Emory University On Thu, Sep 12, 2013 at 7:03 PM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Another vote for the excellent modules system from us at AgResearch Galaxy's dependency injection works in a similar way, without requiring all the external dependencies. We don't need unload since we compose a unique environment for every tool execution. This is much cleaner that loading everything into the environment before running Galaxy, we want tools to be executed with as clean an environment as possible. Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. So far, I have done this in the tool XML file by prefixing the command with a module load command, e.g. command. /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence [snipped] /command The ugly . /etc/profile is needed for technical reasons. I don't like this, as it's a bit hacky, but what it achieves is useful and flexible. I envisage an extension to the tool XML syntax which would let me write it like this, say: requirementsrequirement type=module version=5.0.0EMBOSS/requirement/requirements commandinfoseq -sequence [snipped] /command This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? Here's where I'm coming from. I already have most of the applications I want installed on my system. I am working on updating the RPMs to support side-by-side installs of multiple versions, with version selection by the environment modules facility. This is a small change (and improvement) on what I have today. It's a work in progress, but it's a manageable amount of work. It's looking likely that this multi-version packaging approach may be adopted by an official CentOS Scientific Repo, and this could change the landscape in terms of packaging of scientific applications for major Linux distributions. Who knows, that's all in the future. In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) The big question is, will the Galaxy team accept contributions to Galaxy to support different ways of solving this packaging problem? In particular, allowing for environment modules to be loaded as part of running a tool, and having this functionality in the toolshed? cheers, Simon === Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. === ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] A proposed modules extension for toolshed wrappers
Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. On our HPC cluster, I have configured Galaxy to source the following from our modules system: 1) Global file as defined in universe_wsgi.ini environment_setup_file=setup-modules-for-galaxy.sh ^^ This setups the module environment so commands like module load work. It does something similar to: source /data/apps/modules/Modules/default/init/bash 21 /dev/null 2) Configured tool_dependency_dir as described here: http://wiki.g2.bx.psu.edu/Admin/Config/Tool%20Dependencies tool_dependency_dir=tool_dependency For each requirement as specified the xml, galaxy will automatically look in tool_dependency/name/default/env.sh Each dependancy's env.sh then use our favorite command: module load blah/version Cumbersome? Yup! But I believe its a nice elegant way of getting thing to work with no mucking around in the xml file. This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? I truly like the idea and think it would. However, stepping into the Galaxy devs shoes, a solution like this might only be best for those using modules (small subset of users). Hopefully others can give their thoughts. The con I see in this is that Galaxy already has the env.sh solution which in my setup described above, does support modules In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) I saw that post last week and while I like the idea, folks like me who really do care about the version of GCC we use (and compile our own versions of gcc) will not use it. However, that is going off topic now :-) Will wait and see what others say... -- Adam Brenner Computer Science, Undergraduate Student Donald Bren School of Information and Computer Sciences Research Computing Support Office of Information Technology http://www.oit.uci.edu/rcs/ University of California, Irvine www.ics.uci.edu/~aebrenne/ aebre...@uci.edu On Thu, Sep 12, 2013 at 4:03 PM, Guest, Simon simon.gu...@agresearch.co.nz wrote: Another vote for the excellent modules system from us at AgResearch Galaxy's dependency injection works in a similar way, without requiring all the external dependencies. We don't need unload since we compose a unique environment for every tool execution. This is much cleaner that loading everything into the environment before running Galaxy, we want tools to be executed with as clean an environment as possible. Setting the environment for each tool just prior to running the tool is clearly better. What I'm doing currently is loading an environment module just before running a tool in Galaxy. So far, I have done this in the tool XML file by prefixing the command with a module load command, e.g. command. /etc/profile; module load EMBOSS/5.0.0; infoseq -sequence [snipped] /command The ugly . /etc/profile is needed for technical reasons. I don't like this, as it's a bit hacky, but what it achieves is useful and flexible. I envisage an extension to the tool XML syntax which would let me write it like this, say: requirementsrequirement type=module version=5.0.0EMBOSS/requirement/requirements commandinfoseq -sequence [snipped] /command This will enable my native EMBOSS wrapper to run the defined version of the (system-installed) EMBOSS suite. However, it would only be worth me writing this extension if there's a likelihood of getting it into Galaxy. Is there? Here's where I'm coming from. I already have most of the applications I want installed on my system. I am working on updating the RPMs to support side-by-side installs of multiple versions, with version selection by the environment modules facility. This is a small change (and improvement) on what I have today. It's a work in progress, but it's a manageable amount of work. It's looking likely that this multi-version packaging approach may be adopted by an official CentOS Scientific Repo, and this could change the landscape in terms of packaging of scientific applications for major Linux distributions. Who knows, that's all in the future. In the meantime, there's work going on in Galaxy land to package every application wanted in Galaxy, by writing install scripts, encapsulating environment variables in tool XML files, etc. (There is difficulty here in writing install scripts which work on every platform, which I have commented on previously.) The big
