I'd +1 to separating out workflows, if the tools are already available the process shouldn't require an admin to install them (not when you can download/upload manually). Should definitely have some separation there...unfortunately all of my ideas consist of me putting aside other work projects for a month to work on toolshed stuff. It's, likewise, no secret that I desperately want to overhaul it.
2015-04-07 13:34 GMT-05:00 John Chilton <jmchil...@gmail.com>: > Yes. Exactly. > > I created a few cards from some of your specific suggestions: > > https://trello.com/c/oC9iC4dx > https://trello.com/c/j9RQxd6M > https://trello.com/c/MFyTxcgq > > From my perspective - empowering users to find workflows however and > request missing tools and giving deployers access to more easily > install missing tools from the workflow screen would be bigger wins - > but are perhaps a little more challenging. > > There is a lot that needs to be done to improve workflow importing > generally and workflow importing from the Tool Shed specifically. I > would call the current implementation something of a prototype - the > deployer experience is confusing, the user experience is less than > ideal, I am pretty sure Galaxy's concept of the workflows and their > lineages and disconnected from the tool sheds - so things like updates > haven't been thought through, etc.... > > The problem is probably obvious - the ambitions of the tool shed and > workflow subsystems are exceeded by the number of developer hours > dedicated to them. Given that more people use tools than workflows in > the tool shed and by wide margin - I think the current devteam focus > on updating tools, improving the experience for tool developers, > improving tool search, trying to fix tool testing, imporving > dependency management, etc... makes sense. On the flip side of the > coin - I have put a lot of effort into workflows but focused on > expressiveness, etc... - again because I think those are more pressing > problems right now than tool shed publication. I doesn't mean anyone > thinks this isn't important - it is just we have to triage right? > > Three questions I have when thinking about workflows in the tool shed - > > 1) I don't think anyone would disagree that we need to spend more time > on these things - but do you think we are missing allocating our time > on other things related to the tool shed and workflows and should > instead we be more focused on this use case - workflows coming from > the tool shed? > > 2) Is there anything I or anyone else can do to encourage more outside > development? It is no secret that my focus is generally on doing > things that will encourage other developers to build awesome stuff. > Any ideas how to create incentives for hacking on the tool shed. > > 3) Does putting workflows in the tool shed make sense fundamentally - > if we do find time to invest a lot more work on this - should > enhancing that process be the focus or should we find/build something > more tailored to workflow distribution and optimized for user facing > applications (the tool shed is meant to be deployer facing instead of > user facing IMO). > > -John > > > On Wed, Apr 1, 2015 at 9:56 AM, Peter Cock <p.j.a.c...@googlemail.com> > wrote: > > Hello all, > > > > I've been playing with the Tool Shed lately for publishing > > Galaxy workflows - and realised the download process is > > still very complicated, especially the final hurdle - importing > > the workflow in order to actually use it. > > > > For the following example I am starting from a clean state - > > an almost fresh Galaxy installation with NOTHING as yet > > installed from the Tool Shed (see end of email), and I want > > to install a single workflow: > > > > https://toolshed.g2.bx.psu.edu/view/peterjc/blast_top_hit_species > > > > 1. Goto Galaxy in browser, eg http://localhost/galaxy-dev/ > > 2. Goto "User", "Register" > > 3. Create account using address added to admin_users > > earlier in config/galaxy.ini > > 4. Click "Return to the home page" > > 5. Since logged in as admin, can goto "Admin" settings > > 6. Select "Search and browse tool sheds" > > 7. Select "Galaxy Main Tool Shed" > > 8. Enter "species" in the search box > > 9. Select "blast_top_hit_species" > > 10. Note the list of dependencies > > 11. Click "Install into Galaxy" > > 12. Leave the dependency handling ticked. > > 13. Ponder that the section selection will apply to? The current > > repository has no tools but some of the dependencies do... > > I don't want to bundle the BLAST tools in the same section as > > the text manipulation tools. Go with the default (don't pick a section). > > 14. Click "install" > > 15. Wait as the following are installed: > > > > Name Description Owner Revision > > blast_top_hit_species Workflow to count species of top nr BLASTX hits > > of a transcriptome peterjc db0c1bb92308 > > ncbi_blast_plus NCBI BLAST+ devteam 2fe07f50a41e > > blast_datatypes BlastXml and other blast datatypes devteam 5482a8cd0f36 > > package_blast_plus_2_2_29 NCBI BLAST+ 2.2.29 (binaries only) iuc > a2ec897aac2c > > package_biopython_1_65 Downloads and compiles version 1.65 of the > > Biopython package. biopython dc595937617c > > package_numpy_1_9 Contains a tool dependency definition that downloads > > and compiles version 1.9 of the the python numpy package. iuc > > 84e97f5cd3ab > > package_atlas_3_10 Contains a tool dependency definition that > > downloads and compiles version 3.10 of the the Automatically Tuned > > Linear Algebra Software. iuc 37ff27ea37ed > > fasta_to_tabular FASTA-to-Tabular converter devteam 9d189d08f2ad > > sample_seqs Sub-sample sequences files (e.g. to reduce coverage) > > peterjc 02c13ef1a669 > > unique Advanced unique lines program using GNU sort bgruening > 7ce75adb93be > > > > 16. Get bored, especially if the page appears to have stalled. > > Try reloading the page which goes back to the "Admin" welcome. > > Now how do I get back to the installation status... > > 17. Click on "Manage installed tool shed repositories" > > 18. If at this point some of the dependency requirements are in > > error state, try a round of repairing repositories (and watch at > > the terminal, e.g. compiling numpy is slow). > > 19. Admire all the nice green "tick" icons and "Installed" entries. > > 20. Notice the tiny grey "cog" icon against the workflow > > repository, "blast_top_hit_species". > > 21. Read the legend, "This repository contains exported workflows", > > which is encouraging but there is nothing on the page hinting at > > *how* to actually use the new workflow. > > 22. Try the context menu to see if that lets you use the workflow. No. > > 23. At this point I fully explored the workflow menus looking for an > > import option, and gave up and returned to "Manage installed tool > > shed repositories". > > 23. Ignore the context menu, click on "blast_top_hit_species". > > 24. Click on the top right "repository actions" menu > > 25. Find nothing about importing a workflow. > > 26. Click on the "Species of top BLAST hits" entry under > > "Contents of this repository" > > 27. Observe pretty SVG image of workflow. > > 24. Click on the top right "repository actions" menu > > 25. Notice the menu now offers "import workflow to galaxy". > > 26. Click on "import workflow to galaxy". > > 26. Back at SVG with green message "Workflow Species of top > > BLAST hits imported successfully. " > > 27. Return to "Analyze Data" (or "Workflows") and finally the > > workflow is available (although probably only for the Galaxy > > administrator's personal account). > > 28. Cry because all the new tools are dumped in the top level > > tool menu (see step 13). > > > > Possible suggestions to make this easier: > > > > At step 21, include a clue in the legend? > > > > At step 22 and 25, include "Import all these workflows" in menu? > > > > At step 24, change the button to "repository/workflow actions" > > since it includes some actions specific to the viewed workflow? > > > > How about the common use case where the Galaxy Admin > > was asked to import the workflow for a specific user? Can > > we have an action "Import workflow for specific user"? > > > > Also I think the tool section behaviour for dependencies needs > > clarification (step 13/28). From a tool author perspective, I would > > like to be able to set the default section for each tool (see also > > past requests to set the order they are added to the tool panel). > > > > Regards, > > > > Peter > > > > -- > > > > > > This was how I setup my current galaxy development instance > > tracking the default dev branch on github: > > > > $ cd ~/repositories > > $ git clone https://github.com/galaxyproject/galaxy.git > > $ cd galaxy > > > > I made minor config changes, including adding myself as > > an admin user via the admin_users setting: > > > > $ emacs config/galaxy.ini > > ... > > > > I also setup Apache to serve this instance at > > http://localhost/galaxy-dev/ and set ToolShed path to be > > ~/repositories/shed_tools/ > > > > This was working fine. To hopefully reset this back to a what > > is almost a clean slate (for this bug report), I next removed > > the auto-generated XML files, data files and SQLite database, > > and tool shed files: > > > > $ rm -f ~/repositories/galaxy/*.xml > > $ rm -f ~/repositories/galaxy/config/*.xml > > $ rm -f ~/repositories/galaxy/database/universe.sqlite > > $ rm -rf ~/repositories/galaxy/database/files/* > > $ rm -rf ~/repositories/shed_tools/ > > > > (Dangerous - which is why I explicitly used the full paths!) > > > > I then restart Galaxy, > > > > $ cd ~/repositories/galaxy > > $ ./run.sh > > ... > > > > Note it creates a new empty SQLite database etc. > > ___________________________________________________________ > > Please keep all replies on the list by using "reply all" > > in your mail client. To manage your subscriptions to this > > and other Galaxy lists, please use the interface at: > > https://lists.galaxyproject.org/ > > > > To search Galaxy mailing lists use the unified search at: > > http://galaxyproject.org/search/mailinglists/ > ___________________________________________________________ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > https://lists.galaxyproject.org/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > -- Eric Rasche Programmer II Center for Phage Technology Rm 312A, BioBio Texas A&M University College Station, TX 77843 404-692-2048 e...@tamu.edu rasche.e...@yandex.ru
___________________________________________________________ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: https://lists.galaxyproject.org/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/