Re: [galaxy-dev] to put or not to put a tool suite into the toolshed
Hi Wolfgang short answer: Please, put your package into the Galaxy Tool Shed. This makes it visible to the whole Galaxy community and everybody can profit from it. long answer: I have been in the same boat. We never put the Galaxy wrappers and the perl scripts of our old NGS pipeline into the Tool Shed. I do feel bad about this. My excuse has always been that too much local customization would be necessary, eg making possible to store data outside the Galaxy directory tree. Hence, we only distributed the package as bunch of files on request. Obviously, there were hardly any requests. Now we are in a similar situation. We are in the process of making our new R/Bioconductor based NGS pipeline available on our internal Galaxy server. And the code is far from being ready to be put into the Tool Shed. But this time we will make the effort. So the Galaxy community can benefit from it (@Dave Clements: Unfortunately, it won't be ready for Baltimore). Regards, Hans-Rudolf On 06/05/2014 11:52 PM, Wolfgang Maier wrote: Dear all, I am maintaining a (still young) suite of command line tools (written in Python) for identification of mutations in model organism genomes through whole-genome sequencing (https://sourceforge.net/projects/mimodd/). MiModD aims at geneticists that do not have much background in bioinformatics, and it's supposed to make WGS analysis for small model organism genomes (anything from yeast to fish) possible on regular PCs, so it's not a cloud/cluster solution. We do support Galaxy though through a complete set of tool wrappers for three reasons: - to provide a graphical user interface - to offer labs the possibility to install the software on one dedicated machine, but run analyses from any machine (typically with Windows installed) - to keep analysis workflows documented and reproducible. Currently, we advise users to take advantage of these features and install a local instance of Galaxy even though it will be running then only on a single desktop PC (or even just a notebook). After installation of Galaxy our software simply copies its wrapper xmls over to the tools folder and modifies the tool_conf file to integrate itself. In addition, users have to install a bit of other third-party software (samtools, snap aligner, optionally snpeff) that our code relies on. [End of lengthy introduction] So my question is: in what way could the package profit from being uploaded to a Galaxy toolshed ? I guess it would mean quite some extra work from my side since I'm not familiar with the whole procedure, so are there benefits (visibility, ease of installation, etc.) that are worth the effort ? Thanks a real lot for any feedback, Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installing galaxy with Apache
Hmm... sorry for the delay I hope you got this to work. I feel like someone should respond to this but it is difficult to remotely debug web server configuration issues. Should your apache config line: RewriteRule ^(.*) http://localhost:8080$1 [P] be: RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] instead? That is the only guess I have. Thanks for your interest in deploying Galaxy. -John On Mon, Jun 2, 2014 at 7:37 AM, Matthias Enders m.end...@german-seed-alliance.de wrote: Hi, this thread is likely related to this one: http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td4654499.html I installed Galaxy on a Ubuntu Server 12.04 system. I followed the instruction on the link mentioned above and this one: https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Now my issue is: When I got this configuration in my /etc/apache2/sites-available/default : RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible at www.my_server_url.de but not at www.my_server_url.de/galaxy Following the later URL I got this message: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /galaxy. Reason: DNS lookup failure for: localhost:8080http: Apache/2.2.22 (Ubuntu) Server at www.my_server_url.de Port 80 When I change the configuration like this: (make the second line a comment): RewriteEngine on #RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible both at www.my_server_url.de and at www.my_server_url.de/galaxy When I completely remove this part, the “It works!” standard page of apache is displayed. The configuration in my universe.ini is as follows: host = 127.0.0.1 [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy Can you please help me with this issue? I want to see the “It works!” page at www.my_server_url.de and galaxy only at www.my_server_urla.de/galaxy. Thanks in advance! Kind regards Matthias Enders ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Installing galaxy with Apache
Hi John, your solution perfectly solved my problem. Thank you! -Ursprüngliche Nachricht- Von: John Chilton [mailto:jmchil...@gmail.com] Gesendet: Friday, June 06, 2014 3:32 PM An: Matthias Enders Cc: galaxy-...@bx.psu.edu Betreff: Re: [galaxy-dev] Installing galaxy with Apache Hmm... sorry for the delay I hope you got this to work. I feel like someone should respond to this but it is difficult to remotely debug web server configuration issues. Should your apache config line: RewriteRule ^(.*) http://localhost:8080$1 [P] be: RewriteRule ^/galaxy(.*) http://localhost:8080$1 [P] instead? That is the only guess I have. Thanks for your interest in deploying Galaxy. -John On Mon, Jun 2, 2014 at 7:37 AM, Matthias Enders m.end...@german-seed-alliance.de wrote: Hi, this thread is likely related to this one: http://user.list.galaxyproject.org/Installing-galaxy-with-Apache-td465 4499.html I installed Galaxy on a Ubuntu Server 12.04 system. I followed the instruction on the link mentioned above and this one: https://wiki.galaxyproject.org/Admin/Config/ApacheProxy Now my issue is: When I got this configuration in my /etc/apache2/sites-available/default : RewriteEngine on RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible at www.my_server_url.de but not at www.my_server_url.de/galaxy Following the later URL I got this message: Proxy Error The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET /galaxy. Reason: DNS lookup failure for: localhost:8080http: Apache/2.2.22 (Ubuntu) Server at www.my_server_url.de Port 80 When I change the configuration like this: (make the second line a comment): RewriteEngine on #RewriteRule ^/galaxy$ /galaxy/ [R] RewriteRule ^/galaxy/static/style/(.*) /home/galaxy/galaxy-dist/static/june_2007_style/blue/$1 [L] RewriteRule ^/galaxy/static/scripts/(.*) /home/galaxy/galaxy-dist/static/scripts/packed/$1 [L] RewriteRule ^/galaxy/static/(.*) /home/galaxy/galaxy-dist/static/$1 [L] RewriteRule ^/galaxy/favicon.ico /home/galaxy/galaxy-dist/static/favicon.ico [L] RewriteRule ^/galaxy/robots.txt /home/galaxy/galaxy-dist/static/robots.txt [L] RewriteRule ^(.*) http://localhost:8080$1 [P] Galaxy is visible both at www.my_server_url.de and at www.my_server_url.de/galaxy When I completely remove this part, the “It works!” standard page of apache is displayed. The configuration in my universe.ini is as follows: host = 127.0.0.1 [filter:proxy-prefix] use = egg:PasteDeploy#prefix prefix = /galaxy [app:main] filter-with = proxy-prefix cookie_path = /galaxy Can you please help me with this issue? I want to see the “It works!” page at www.my_server_url.de and galaxy only at www.my_server_urla.de/galaxy. Thanks in advance! Kind regards Matthias Enders ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] ToolShed for Visualisation Registry?
I was wondering if there is a possibility of a 'Tool'Shed for Visualisation registry plugins(custom plugins, developed by users/developers)? The motivation for this comes from a small 'tool' I wrote for visualiing fastq phred scores using box plots[Still under development] here: http://saketkc.github.io/biojs/src/test/javascript/TestPhredScoreViewer.html I am relying on d3.text to read my fastq, but I believe I could handle this in a much better way had I written it for Galaxy's Visualisation registry specifically. For e.g. I could fetch fastq scores line by line using Galaxy's API, which would make my task a bit easier. I could always keep this to my local instance of Galaxy, but a 'toolshed' or 'registry' for visualisation would help. Saket ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ToolShed for Visualisation Registry?
Hello Saket, Thanks for your interest - I totally agree, I think a lot of tools like this would be better served as visualization plugins. There is a Trello card for this request https://trello.com/c/YLA70rZx - please vote up if you are interested. This has been discussed by the core team and I some member of the team was largely in favor though there was some descent. I think both the tool shed and visualization framework are well suited to allow this integration (not that there aren't many little hurdles, but I think at a high level it should be possible). That said - I don't think this is a top priority for the core team right now (though others should definitely correct me if I am wrong) - and I would be surprised to see movement on it this year. Perhaps until this is available perhaps the community should just maintain a centralized list of repositories on the wiki? I think seeing a list of community developed plugins would also help the team guage how important automating install of new plugins should be. -John On Fri, Jun 6, 2014 at 8:43 AM, Saket Choudhary sake...@gmail.com wrote: I was wondering if there is a possibility of a 'Tool'Shed for Visualisation registry plugins(custom plugins, developed by users/developers)? The motivation for this comes from a small 'tool' I wrote for visualiing fastq phred scores using box plots[Still under development] here: http://saketkc.github.io/biojs/src/test/javascript/TestPhredScoreViewer.html I am relying on d3.text to read my fastq, but I believe I could handle this in a much better way had I written it for Galaxy's Visualisation registry specifically. For e.g. I could fetch fastq scores line by line using Galaxy's API, which would make my task a bit easier. I could always keep this to my local instance of Galaxy, but a 'toolshed' or 'registry' for visualisation would help. Saket ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] to put or not to put a tool suite into the toolshed
Thanks a lot for all the encouraging feedback. More than anything else the friendly atmosphere that seems to prevail on this mailing list makes me think that it may indeed be worthwhile to make the package available to this community ! I will have a closer look at the toolshed instructions and see if I get along with them. Have a nice weekend, Wolfgang ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] ClustalW jobs aren't working
Hi Malcolm, That error makes me think you might be running an outdated version of galaxy code - the toolshed code has undergone extensive revision over the last few months? I just tested that repository on a freshly updated galaxy-central clone and it installed without drama, so I wonder what: hg tip shows? I just tested using : (vgalaxy)rlazarus@rlazarus-UX31A:~/galaxy$ hg tip changeset: 13756:84a00e4f7d06 tag: tip user:Dannon Baker dannonba...@me.com date:Fri Jun 06 17:12:30 2014 -0400 summary: Clarify language in DeleteIntermediateDataset PJA. If your clone is not up to date, I'd recommend completely removing the failed installation (through the admin menu - check the box for complete removal), shut down galaxy, backup your database, do the usual hg pull -u dance and any necessary database upgrade steps then try a clean install? Thanks for reporting this - if it persists on recent Galaxy code we'll need to do some deeper investigation. On Fri, Jun 6, 2014 at 11:32 PM, Malcolm Tobias mtob...@wustl.edu wrote: Ross, Thanks for the reply! Unfortunately I am the local Galaxy admin ;-) I had tried installing the clustalw tool from the toolshed, but that failed with an error (more on that later). I disabled the local tool: [galaxy@login002 galaxy-dist]$ diff tool_conf.xml tool_conf.xml.bkup 226a227,229 section name=Multiple Alignments id=clustal tool file=rgenetics/rgClustalw.xml / /section bounced galaxy in case that's necessary, then retried installing from the toolshed. Shortly after clicking install, I get this message: Internal Server Error Galaxy was unable to sucessfully complete your request An error occurred. This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem. The error has been logged to our team. The logs appear to be complaining about 'prior_installation_required' which I'm assuming means the package_clustalw_2_1 dependency. I was able to install that, and I can verify by looking at the local toolshed: [galaxy@login002 ~]$ ls galaxy-toolshed/ toolshed.g2.bx.psu.edu/repos/devteam/ bowtie_wrappers package_clustalw_2_1 package_vcftools_0_1_11 bwa_wrappers package_fastx_toolkit_0_0_13 Again, I'll post the logs from when the install fails in case that helps. Any suggestions are much appreciated. Cheers, Malcolm 10.28.56.101 - - [06/Jun/2014:08:20:38 -0500] GET /admin_toolshed/prepare_for_install?tool_shed_url= http://toolshed.g2.bx.psu.edu/repository_ids=0e5d027cf47ecae0changeset_revisions=7cc64024fe92 HTTP/1.1 500 - http://toolshed.g2.bx.psu.edu/repository/preview_tools_in_changeset?repository_id=0e5d027cf47ecae0changeset_revision=7cc64024fe92; Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Firefox/24.0 Error - type 'exceptions.UnboundLocalError': local variable 'prior_installation_required' referenced before assignment URL: http://galaxy.chpc.wustl.edu/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=0e5d027cf47ecae0changeset_revisions=7cc64024fe92 File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/error.py', line 149 in __call__ app_iter = self.application(environ, sr_checker) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/recursive.py', line 84 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/eggs/Paste-1.7.5.1-py2.7.egg/paste/httpexceptions.py', line 633 in __call__ return self.application(environ, start_response) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 132 in __call__ return self.handle_request( environ, start_response ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py', line 190 in handle_request body = method( trans, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/web/framework/__init__.py', line 221 in decorator return func( self, trans, *args, **kwargs ) File '/home/galaxy/galaxy-dist/lib/galaxy/webapps/galaxy/controllers/admin_toolshed.py', line 896 in prepare_for_install common_install_util.get_dependencies_for_repository( trans, tool_shed_url, repo_info_dict, includes_tool_dependencies ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 84 in get_dependencies_for_repository installed_rd, missing_rd = get_installed_and_missing_repository_dependencies_for_new_install( trans, repo_info_tuple ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/common_install_util.py', line 190 in get_installed_and_missing_repository_dependencies_for_new_install tool_shed, name, owner, changeset_revision, prior_installation_required = suc.parse_repository_dependency_tuple( rd_tup ) File '/home/galaxy/galaxy-dist/lib/tool_shed/util/shed_util_common.py', line 1132 in parse_repository_dependency_tuple prior_installation_required = str(
[galaxy-dev] Are there examples of tools that run processes in the background?
Hi folks, We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications). Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time. Or is there a better approach I didn't consider? Assuming the answer is no, then are there any tools in the toolshed that launch background processes? If so, I'd like to look at them. And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down. Is there a way to configure a process to be run when it's time to quit Galaxy? Thanks! Melissa ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Are there examples of tools that run processes in the background?
Hi Melissa, The question in the title is easy to answer - most large galaxy instances run (most) jobs on a connected cluster which is effectively 'background' I guess so yes - all of them on cluster backed Galaxy instances might be a reasonable response. The questions in the text are more complex, but (eg) AFAIK there's no technical reason why some batch process owned by the Galaxy user shouldn't issue a sh run.sh --stop-daemon at any time to stop Galaxy. Exactly when that might be a good idea is not so clear to me so I'm probably missing something here - the Galaxy process doesn't consume a lot of resources and is typically left running permanently so users can take advantage of the persistence and reproducibility. Tools are AFAIK not generally written to launch their own background processes because they are managed by the Galaxy job runner which interfaces with (eg) SGE or Torque to run each separate tool invocation and clean up when it has completed. I'm sure you could run a transient Galaxy to batch up some workflows or other processes and I'm sure a VM might be part of that and the API allows a lot of automation. There's a new Galaxy docker image project which might be even more efficient and helpful - see http://lists.bx.psu.edu/pipermail/galaxy-dev/2014-April/019289.html On Sat, Jun 7, 2014 at 10:18 AM, Melissa Cline cl...@soe.ucsc.edu wrote: Hi folks, We're working on developing a VM which we'd like to be tightly coordinated with Galaxy, such that VM users can export data from the VM into Galaxy, perform further analyses on that data, and then import the analysis results into the VM for storage (from where it will serve additional applications). Right now, I'm picturing this as a set of Galaxy tools that launch the VM, manage the data import and export, and shut down the VM at the appropriate time. Or is there a better approach I didn't consider? Assuming the answer is no, then are there any tools in the toolshed that launch background processes? If so, I'd like to look at them. And assuming that my VM will be running in the background, I'd like it to shut down when Galaxy shuts down. Is there a way to configure a process to be run when it's time to quit Galaxy? Thanks! Melissa ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
