I have issued a pull request with a specific implementation of these ideas:


Please feel free to comment.


On Fri, Sep 27, 2013 at 12:23 AM, John Chilton <chil...@msi.umn.edu> wrote:
> Simon,
> What is the advantage of putting that XML definition in the tool shed?
> It is not 100% true because of prior_install_required dependencies,
> but for the most part sourcing/load the environment for tools is a
> Galaxy problem, not so much a tool shed one. What if we did this
> instead?
> Add an option to Galaxy's universe_wsgi.ini with the following default:
> tool_dependency_resolution_order = gx_package_manual, gx_package_toolshed
> Which essentially implements my idea above, with James' additional
> configuration. But which can be overridden as:
> tool_dependency_resolution_order = plugin_module, gx_package_manual,
> gx_package_toolshed
> If set this way then placing <requirement
> package="0.5.9">bwa</requirement> in a tool will result in the module
> bwa/0.5.9 being loaded if it is 'avail'able, else it will check for a
> manually installed env.sh (which is where MSI is currently putting its
> module loads), and else it will fallback to source the tool shed
> installed dependency.
> I feel like this will give you everything you want without any extra
> XML or configuration. Let me know if I am wrong.
> -John
> On Thu, Sep 26, 2013 at 11:39 PM, Guest, Simon
> <simon.gu...@agresearch.co.nz> wrote:
>> At Thu, 26 Sep 2013 22:03:09 -0400,
>> Greg Von Kuster wrote:
>>> Hi John,
>>> On Sep 26, 2013, at 9:15 PM, John Chilton <chil...@msi.umn.edu> wrote:
>>> > I was not even thinking we needed to modify the tool shed to implement
>>> > this. I was hoping (?) you could just modify:
>>> Nothing in the Tool Shed itself would be affected or require modification 
>>> for this new feature as this is completely on the Galaxy side.
>>> >
>>> > lib/galaxy/tools/deps/__init__.py
>>> >
>>> > to implement this. If some tool contains the tag
>>> >
>>> >  <requirement type="package" version="1.7.1">numpy</requirement>
>>> >
>>> > then if there is a manually installed tool_dependency in
>>> > `tool_dependency_dir/numpy/1.7.1/env.sh` that would take precedence
>>> > over the tool shed installed version (would that be something like
>>> > `tool_dependency_dir/numpy/1.7.1/owner/name/changeset/env.sh`)? Let me
>>> > know if this is way off base.
>>> This is a possibility perhaps, but there seems to be a potential weakness 
>>> in that it doesn't require the Tool Dependency object to exist since the 
>>> tool will function without a installed dependency from the Tool Shed.  Or, 
>>> if the installed depndency s required, then it is meaningless because it 
>>> won't be used.  If the former case, then the tool dependency cannot be 
>>> shared via the Tool Shed's dependency mechanism because none of the 
>>> relationships will be defined  since nothing is installed.  Wouldn't it be 
>>> better to allow the Galaxy admin to point the ToolDependency object to a 
>>> specified binary on disk?  In this way, all relationships defined by Tool 
>>> Shed installations will work as expected, with all contained tools with 
>>> that dependency point to that same shared location on disk.
>> Hi Greg, John,
>> What you are discussing is pretty close to what I am after, and what I
>> am prepared to spend some time working on, if that can help.  This is
>> what I have posted about a couple of times previously.  The option to
>> use a pre-installed package rather than a Galaxy installed one I think
>> would be very useful in general.  I think it can be done in a way that
>> doesn't break the tool dependency model.
>> I envisage providing access to existing programs on disk by loading
>> the relevant environment module.  Platforms will differ greatly on
>> where they have programs installed (usr/local/bwa-0.5.9,
>> /usr/libexec/bwa-0.5.9, etc.).  However, we could arrange to have a
>> conventionally named environment module available, so Galaxy just has
>> to be told somehow to do a module load bwa/0.5.9, say, prior to
>> trying to run that tool, which will then be found on the PATH.
>> Hooking this in without killing the dependency on named and versioned
>> toolshed repos could work like this.  In parallel with the repo sets
>> named, e.g. package_bwa_0_5_9, which download and build the package,
>> we have a set named like native_package_bwa_0_5_9.  This could have a
>> tool_dependencies.xml file like this:
>> <?xml version="1.0"?>
>> <tool_dependency>
>>     <native_package name="bwa" version="0.5.9">
>>         <actions>
>>             <action type="load_module">
>>                 <module name="bwa" version="0.5.9"</module>
>>             </action>
>>         </actions>
>>         <readme>
>> Uses native BWA via environment module bwa/0.5.9
>>         </readme>
>>     </native_package>
>> </tool_dependency>
>> What this does is ensure that any repo which requires version 0.5.9 of
>> bwa can have this dependency resolved by either package_bwa_0_5_9, or
>> native_package_bwa_0_5_9.
>> I envisage a Galaxy configuration setting that enables native package
>> and/or Galaxy package dependency resolution, and which one should be
>> preferred.  Of course, if one of these has been manually installed, it
>> will be used as is.
>> I would really like to be able to install tools from the toolshed, and
>> have them resolve their dependencies automatically using my
>> preinstalled application suite.  I see that would also meet the needs
>> being discussed here.
>> How does that sound?
>> cheers,
>> Simon
>> =======================================================================
>> Attention: The information contained in this message and/or attachments
>> from AgResearch Limited is intended only for the persons or entities
>> to which it is addressed and may contain confidential and/or privileged
>> material. Any review, retransmission, dissemination or other use of, or
>> taking of any action in reliance upon, this information by persons or
>> entities other than the intended recipients is prohibited by AgResearch
>> Limited. If you have received this message in error, please notify the
>> sender immediately.
>> =======================================================================

Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

To search Galaxy mailing lists use the unified search at:

Reply via email to