Hi John, It's really too bad that we didn't find time to discuss this in person at the GCC. Until now, I've not heard from anyone that installation from the tool shed without requiring a Galaxy database is important, so I'm lacking some context on this (I assume your statement "without a database present" refers to the Galaxy database).
Some concerns that immediately pop into my mind are the following - this is not a complete list. 1) How do you ensure dependency relationships between installed repositories? 2) How do you manage locating installed repository contents? 3) How do you manage the current state of an installed repostiroy and determine if it can be updated? 4) How do you manage tool version lineage for installed repositories that contain tools (this plays an importnt role in ensuring reproducibility)? 5) How do you maintain the state of an installed repository, enabling it to be repaired if it or any of it's dependencies are in an error state? Since I've never considered re-engineering the tool shed installation process so that it would function in an environment without a Galaxy database, I'm not sure how much effort would need to go into doing so, or where to start. I'll have to think about this for a while. Greg Von Kuster On Jul 15, 2013, at 7:27 PM, John Chilton <chil...@msi.umn.edu> wrote: > One of my goals for the GCC was to sell the idea that tool shed > repositories need to be installable without a database present. I > talked with James Taylor and Enis Afgan about this idea briefly and they > seemed > to believe this was a good idea - I kept meaning to discuss it with Greg but I > never got a good opportunity. Though in past Greg has made this sound > potentially doable and has never objected to the goal overtly. > > I have two specific use cases in mind (CloudBioLinux and LWR), but perhaps the > higher-level justification is something along the lines that a lot of effort > from Greg and others (Dave, Bjorn, Peter, Nate) has gone into building a > modular > dependency system that could very easily be leveraged by applications other > than > Galaxy, so the extra steps that could be taken to make this possible should to > make the codebase as broadly useful and to encourage adoption. The Galaxy > community could benefit from other applications potentially utilizing and > populating the tool shed and Galaxy tool developers would be further > incentized > to write good, modular dependencies and publish them to the tool shed. > > A high-level task decomposition would be something like this: > > 1. Rework installing tool shed repositories to not require a database. A kind > of messy way to do this might be adding a use_database flag throughout. A > cleaner way might be to use allow the core functionality to work with > callbacks > or plugins that performed the database interactions. > > 2. Separate the core functionality out of the Galaxy code base entirely into > a reusable, stand-alone library. > > I would love buy in from the Galaxy team on item 2 above, but it is not > strictly needed for my goals - I imagine I could write a script to pull it out > Galaxy and build the library automatically or even just have the Galaxy > codebase > present when using Galaxy-less tool shed dependencies. > > Buy in on item 1 by the Galaxy team (specifically Greg and Dave B.) > however is needed, are there any objections to this idea? Do you have any > broad > advice on how to approach this to ensure the changes make sense, work with > your > long term vision, and end up in Galaxy? > > Of all the things on my TODO list for the next year, this is probably the most > potentially broadly interesting to this weeks BOSC codefest attendees, so I > was > going to attempt to sell this as something to work on. The sales pitch would > include building a little tool shed version of the module command - > http://linux.die.net/man/1/module to demonstrate this work and have something > immediately useful produced. > > The idea would be to create a command-line tool for utilizing tool shed > dependencies. > > # Unlike standard module, install procedure is available. Probably could > # default to main tool shed and latest installable revision > > % tsmodule repo:install galaxyp/tint > % tsmodule repo:install toolshed.g2.bx.psu.edu/galaxyp/tint/ab43b5ba7a4e > > # module lets you list packages, I guess tool shed version would need > # repository and package listings: > > % tsmodule repo:list > toolshed.g2.bx.psu.edu/galaxyp/tint/ab43b5ba7a4e > % tsmodule package:list > tint_proteomics_scripts/1.19.19/galaxyp/tint/ab43b5ba7a4e > > # Finally, a use command would source the env.sh script and make dependency > # available in the command-line (might require starting new shell?): > > % tsmodule package:use tint_proteomics_scripts > % tsmodule package:use tint_proteomics_scripts/1.19.19 > % tsmodule package:use > int_proteomics_scripts/1.19.19/galaxyp/tint/ab43b5ba7a4e > > # use apps that would be available to tools with valid requirements tags. > % iQuantCLI > > This would be different from using the API scripts because there would be no > API, Galaxy instance, or Galaxy database involved - just the Galaxy code. If > this was able to split into its own Python library, one could imagine even > allowing something like tsmodule to be installable right from pip and > recursively fetch a toolshed_client library or something like that. > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
