Re: [galaxy-dev] repeat tag enhancement?
Thanks for your answers, we made a Trello card for this, feel free to vote. https://trello.com/c/RllVlsfg/989-repeat-tag-enhancement Best regards, Thomas -Ursprüngliche Nachricht- Von: ja...@taylorlab.org [mailto:ja...@taylorlab.org] Im Auftrag von James Taylor Gesendet: Dienstag, 16. Juli 2013 17:42 An: Berner, Thomas Cc: galaxy-dev@lists.bx.psu.edu Betreff: Re: [galaxy-dev] repeat tag enhancement? Rather than adding additional increments to add/remove, what about allowing the user to directly set the number of elements in the repeat? -- James Taylor, Assistant Professor, Biology/CS, Emory University On Tue, Jul 16, 2013 at 7:34 AM, Berner, Thomas thomas.ber...@jki.bund.de wrote: Hey guys, i was searching the mailing list and wiki for some hints if there is a possibility to enhance the repeat tag. We want to give the option to our users to add more than one additional set of the contained parameters at one go e.g. 10 or 20 or 50, because it can be very frustrating to click the add new dataset button 50 times. Is there any option to do this with a text field or preset like add 20 new datasets or something like that? I would be very thankful for an advice. Greetings, Thomas ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA Illumina Mapping / BWA Reference Genome
Hi Moritz I am struggling to follow what exactly you have done. As far as I can see, you did not use the toolshed (http://wiki.galaxyproject.org/Tool%20Shed) to install the BWA alinger tool, but did all manually? If so, have you added the following line: tool file=sr_mapping/bwa_wrapper.xml / to the tool_conf.xml file, and restarted Galaxy? see also: http://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial Hope this helps Hans-Rudolf On 07/16/2013 08:35 PM, Moritz Juchler wrote: Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to choose a bioinformatic pipeline management tool to find SNP's in genomes from hcc patients. My decision was made in favor of galaxy. I have a 64-bit openSuse 11.3 server. I have installed Galaxy locally, since we have a) very large files (30GB per patient) and b) the data is protection sensitive. I kept close to http://wiki.galaxyproject.org/Admin/Get%20Galaxy Now I would like to run this bpipe pipeline: http://pastebin.com/sZd5vfdL And the first step is to align my genome to a _hg19 reference genome_ which I have locally under /genedata/human_genome_GRCh37/. trr@portalmoritz:~ ls -l /genedata/human_genome_GRCh37/ total 8486312 -rw-r--r-- 1 trr root 3199905909 2013-06-25 16:44 hg19.fa -rw-r--r-- 1 trr root 8591 2013-07-01 16:06 hg19.fa.amb -rw-r--r-- 1 trr root 4040 2013-07-01 16:06 hg19.fa.ann -rw-r--r-- 1 trr root 3137161344 2013-07-01 16:05 hg19.fa.bwt -rw-r--r-- 1 trr root 784290318 2013-07-01 16:06 hg19.fa.pac -rw-r--r-- 1 trr root 1568580688 2013-07-01 16:31 hg19.fa.sa http://hg19.fa.sa _bwa is installed and gives me:_ trr@portalmoritz:~ bwa Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.5a-r405 Contact: Heng Li l...@sanger.ac.uk mailto:l...@sanger.ac.uk Then I tried to follow this guide: http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup to get the reference files and http://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies. This is my _$PATH_ trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/home/trr/bpipe-0.9.8/bin:/home/trr/bwa-0.7.5a:/home/trr/samtools-0.1.19 _In the universe_wsgi.ini I changed:_ tool_dependency_dir = /home/trr/galaxy-dist/tool_dependency_dir debug = False use_interactive = True library_import_dir = /genedata/ allow_library_path_paste = True admin_users = ... This is my _tool_dependency_dir:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa ls -l total 4 drwxr-xr-x 3 trr users 4096 2013-07-16 14:28 0.7.4 lrwxrwxrwx 1 trr users6 2013-07-16 14:17 default - 0.7.4/ This is the_version folder of bwa:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4 ls -l total 8 drwxr-xr-x 2 trr users 4096 2013-07-16 14:18 bin -rw-r--r-- 1 trr users 47 2013-07-16 14:22 env.sh This is the _content of env.sh:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4 cat env.sh PATH=/home/trr/bwa-0.7.5a/:$PATH export PATH And this is the _content of the bin folder:_ trr@portalmoritz:~/galaxy-dist/tool_dependency_dir/bwa/0.7.4/bin ls -l total 3896 -rw-r--r-- 1 trr users 6098 2013-07-16 14:18 bamlite.c -rw-r--r-- 1 trr users 3124 2013-07-16 14:18 bamlite.h -rw-r--r-- 1 trr users 24816 2013-07-16 14:18 bamlite.o -rw-r--r-- 1 trr users 11508 2013-07-16 14:18 bntseq.c -rw-r--r-- 1 trr users 2557 2013-07-16 14:18 bntseq.h -rw-r--r-- 1 trr users 37440 2013-07-16 14:18 bntseq.o -rwxr-xr-x 1 trr users 998217 2013-07-16 14:18 bwa -rw-r--r-- 1 trr users 24225 2013-07-16 14:18 bwa.1 -rw-r--r-- 1 trr users 9416 2013-07-16 14:18 bwa.c -rw-r--r-- 1 trr users 1381 2013-07-16 14:18 bwa.h I got the xmls and .py from https://bitbucket.org/galaxy/galaxy-dist/src/da9d740fce31/tools/sr_mapping and i didnt change them at all and put them into ~/galaxy-dist/_tools/sr_mapping_ (since they were missing in this folder) bwa_color_wrapper.xml bwa_wrapper.py bwa_wrapper.xml I added _bwa_index_color.loc and bwa_index.loc_ to ~galaxy-dist/_tool-data_ (they were missing as well, there/*were no*/ bwa_index_color.loc.sample or bwa_index.lox.sample files!!!) I only have this single line in both bwa_index_color.loc and bwa_index.loc trr@portalmoritz:~/galaxy-dist/tool-data cat bwa_index_color.loc #This is a sample file distributed with Galaxy that enables tools # #unique_build_id dbkey display_name file_path hg19hg19hg19/genedata/human_genome_GRCh37/hg19.fa (Spaces are actually tabs!) After all that, I neither have the Map with BWA for Illumina https://main.g2.bx.psu.edu/tool_runner?tool_id=toolshed.g2.bx.psu.edu/repos/devteam/bwa_wrappers/bwa_wrapper/1.2.3 in my local Galaxy version, nor do I find
Re: [galaxy-dev] Galaxy Local Admin user
Please keep all replies on the list by using reply all ! for my reply see below... On 07/17/2013 06:58 AM, Chadi Abdul Kader El Farran wrote: Dear Hans-Rudolf, Thank you for your reply. The problem was solved by restarting. Now my problem is that I am unable to upload files into my Galaxy. I have put the following code in Universe_wsgi.ini: # Directories of files contained in the following directory can be uploaded to a library from the Admin view library_import_dir = /upload However when I upload files to a data library, all files will seem to have no information (0 bytes) That is very crypticand you need to provide more details - did you get an error? - how do you know the files are empty - what happens if you work with the files make sure to follow the wiki page: http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries Hans-Rudolf Can you kindly help me? Thanks a lot for the support. With warmest regards Chadi EL Farran (Post Graduate Student) Dr. Jonathan Yuin-Han LOH's Laboratory #08-01, Institute of Molecular and Cell Biology (IMCB) 61 Biopolis Drive, Proteos Singapore 138673 Contact No: +65-91944793 E-mail: ch...@nus.edu.sg From: Hans-Rudolf Hotz [h...@fmi.ch] Sent: Tuesday, July 16, 2013 9:39 PM To: Chadi Abdul Kader El Farran Cc: galaxy-...@bx.psu.edu Subject: Re: [galaxy-dev] Galaxy Local Admin user Hi Just double checking: Have you restarted Galaxy? Regards, Hans-Rudolf On 07/16/2013 07:23 AM, Chadi Abdul Kader El Farran wrote: Dear Madam/Sir, I have installed Galaxy Local in our system. I registered as a user. However when I try to give admin status to some users, I am unable despite following the steps mentioned in the help. To give a user Galaxy admin privileges, add their Galaxy login ( email ) to the list in the following config setting in the Galaxy configuration file universe_wsgi.ini. # this should be a comma-separated list of valid Galaxy users admin_users = us...@bx.psu.edu,us...@bx.psu.edu I have done what is mentioned above and the universe_wsgi.ini is edited accordingly. Still when I log on to Galaxy Local I don't see the admin tab. Can you kindly help me. With warmest regards Chadi EL Farran (Post Graduate Student) Dr. Jonathan Yuin-Han Loh's Laboratory #08-01, Institute of Molecular and Cell Biology (IMCB) 61 Biopolis Drive, Proteos Singapore 138673 Contact No: +65-91944793 E-mail: ch...@nus.edu.sg ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Postgres renaming to postgresql
Hi all After updating to the last stable galaxy when I'm trying to upgrade the database I'm getting the following message: /steno-internal/projects/galaxy/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.6-linux-x86_64-ucs4.egg/sqlalchemy/engine/url.py:105: SADeprecationWarning: The SQLAlchemy PostgreSQL dialect has been renamed from 'postgres' to 'postgresql'. The new URL format is postgresql[+driver]://user:pass@host/dbname If I change the line in universe_wsgi.ini from database_connection = postgresql://galaxy:password@localhost:5432/galaxydb to ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] nglims, error pages and permissions
Hi, I am using nglims, and I am getting Internal Error pages sometimes. I am not sure if it is because of Galaxy itself or nglims though. The latest is when I create a new role and hit submit. It is repeatable if I fill in the name as test, no description, I add in a single role (others are present but excluded from being associated), if I check create a new group... and then click save. However, when I created a group and checked create a new role ..., it did not create an error. Next, I tried associating data with this new user. However, her new roles (both the new role I created and her username-role) did not show up, and so I could not give the data to her. Is there any reason why a new user's roles would not show up? -- Lee Katz, Ph.D. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] tool inheritance
One more question for the Galaxy gurus. I have a series of tools that 1) interleave raw reads, 2) assemble them, and 3) predict genes from the assembly. How can I display inheritance on the data produced from these tools? It would be really nice if a set of genes showed that they came from an assembly, which came from a set of raw reads. I can't find this on the site, and so I am sure I am using the wrong keywords or something... Thank you! -- Lee Katz, Ph.D. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
[galaxy-dev] Galaxy load problem
Hello, I'm running Galaxy on a Linux server and have been getting warning messages about heavy loads on the CPU due to Galaxy. This is a dedicated Galaxy server and is currently used very lightly. There seem to be six idle postgres processes running, but only two jobs running. Any idea why the load is so high? Thank you. Ken. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] BWA Illumina Mapping / BWA Reference Genome
Hey, no thats correct I did not use Tool Shed. If I would use it, could I skip all these manual steps? Is the installation of BWA with Tool Shed also this complicated or is it more simle? How did you not follow my steps :) I wrote down everything clearly, at least thats what I was hoping. And no I didnt write this line tool file=sr_mapping/bwa_wrapper.**xml / Thats the first time, I am seeing this tutorial :( I will try this out right now Best Moritz On 17 July 2013 09:42, Hans-Rudolf Hotz h...@fmi.ch wrote: Hi Moritz I am struggling to follow what exactly you have done. As far as I can see, you did not use the toolshed (http://wiki.galaxyproject.**org/Tool%20Shedhttp://wiki.galaxyproject.org/Tool%20Shed) to install the BWA alinger tool, but did all manually? If so, have you added the following line: tool file=sr_mapping/bwa_wrapper.**xml / to the tool_conf.xml file, and restarted Galaxy? see also: http://wiki.galaxyproject.org/**Admin/Tools/Add%20Tool%**20Tutorialhttp://wiki.galaxyproject.org/Admin/Tools/Add%20Tool%20Tutorial Hope this helps Hans-Rudolf On 07/16/2013 08:35 PM, Moritz Juchler wrote: Hello Ladies and Gentlemen, I am Moritz Juchler from University Heidelberg. For my Bachelor thesis I have to choose a bioinformatic pipeline management tool to find SNP's in genomes from hcc patients. My decision was made in favor of galaxy. I have a 64-bit openSuse 11.3 server. I have installed Galaxy locally, since we have a) very large files (30GB per patient) and b) the data is protection sensitive. I kept close to http://wiki.galaxyproject.org/**Admin/Get%20Galaxyhttp://wiki.galaxyproject.org/Admin/Get%20Galaxy Now I would like to run this bpipe pipeline: http://pastebin.com/sZd5vfdL And the first step is to align my genome to a _hg19 reference genome_ which I have locally under /genedata/human_genome_GRCh37/**. trr@portalmoritz:~ ls -l /genedata/human_genome_GRCh37/ total 8486312 -rw-r--r-- 1 trr root 3199905909 2013-06-25 16:44 hg19.fa -rw-r--r-- 1 trr root 8591 2013-07-01 16:06 hg19.fa.amb -rw-r--r-- 1 trr root 4040 2013-07-01 16:06 hg19.fa.ann -rw-r--r-- 1 trr root 3137161344 2013-07-01 16:05 hg19.fa.bwt -rw-r--r-- 1 trr root 784290318 2013-07-01 16:06 hg19.fa.pac -rw-r--r-- 1 trr root 1568580688 2013-07-01 16:31 hg19.fa.sa http://hg19.fa.sa _bwa is installed and gives me:_ trr@portalmoritz:~ bwa Program: bwa (alignment via Burrows-Wheeler transformation) Version: 0.7.5a-r405 Contact: Heng Li l...@sanger.ac.uk mailto:l...@sanger.ac.uk Then I tried to follow this guide: http://wiki.galaxyproject.org/**Admin/NGS%20Local%20Setuphttp://wiki.galaxyproject.org/Admin/NGS%20Local%20Setupto get the reference files and http://wiki.galaxyproject.org/**Admin/Config/Tool%**20Dependencieshttp://wiki.galaxyproject.org/Admin/Config/Tool%20Dependencies . This is my _$PATH_ trr@portalmoritz:~ echo $PATH /home/trr/bin:/usr/local/bin:/**usr/bin:/bin:/usr/bin/X11:/** usr/X11R6/bin:/usr/games:/**home/trr/bpipe-0.9.8/bin:/** home/trr/bwa-0.7.5a:/home/trr/**samtools-0.1.19 _In the universe_wsgi.ini I changed:_ tool_dependency_dir = /home/trr/galaxy-dist/tool_**dependency_dir debug = False use_interactive = True library_import_dir = /genedata/ allow_library_path_paste = True admin_users = ... This is my _tool_dependency_dir:_ trr@portalmoritz:~/galaxy-**dist/tool_dependency_dir/bwa ls -l total 4 drwxr-xr-x 3 trr users 4096 2013-07-16 14:28 0.7.4 lrwxrwxrwx 1 trr users6 2013-07-16 14:17 default - 0.7.4/ This is the_version folder of bwa:_ trr@portalmoritz:~/galaxy-**dist/tool_dependency_dir/bwa/**0.7.4 ls -l total 8 drwxr-xr-x 2 trr users 4096 2013-07-16 14:18 bin -rw-r--r-- 1 trr users 47 2013-07-16 14:22 env.sh This is the _content of env.sh:_ trr@portalmoritz:~/galaxy-**dist/tool_dependency_dir/bwa/**0.7.4 cat env.sh PATH=/home/trr/bwa-0.7.5a/:$**PATH export PATH And this is the _content of the bin folder:_ trr@portalmoritz:~/galaxy-**dist/tool_dependency_dir/bwa/**0.7.4/bin ls -l total 3896 -rw-r--r-- 1 trr users 6098 2013-07-16 14:18 bamlite.c -rw-r--r-- 1 trr users 3124 2013-07-16 14:18 bamlite.h -rw-r--r-- 1 trr users 24816 2013-07-16 14:18 bamlite.o -rw-r--r-- 1 trr users 11508 2013-07-16 14:18 bntseq.c -rw-r--r-- 1 trr users 2557 2013-07-16 14:18 bntseq.h -rw-r--r-- 1 trr users 37440 2013-07-16 14:18 bntseq.o -rwxr-xr-x 1 trr users 998217 2013-07-16 14:18 bwa -rw-r--r-- 1 trr users 24225 2013-07-16 14:18 bwa.1 -rw-r--r-- 1 trr users 9416 2013-07-16 14:18 bwa.c -rw-r--r-- 1 trr users 1381 2013-07-16 14:18 bwa.h I got the xmls and .py from https://bitbucket.org/galaxy/**galaxy-dist/src/da9d740fce31/**
Re: [galaxy-dev] Galaxy Local Admin user
Hi Chadi
I am not familiar with the this tool
I recommend to try running the tool manually on the command line first,
and then post the error on the mailing list (with a new subject) IF the
error has something to do with Galaxy.
Hans-Rudolf
On 07/17/2013 05:00 PM, Chadi Abdul Kader El Farran wrote:
Dear Hans-Rudolf,
Thank you for your reply once again.
It appears that the problem is from a tool I installed in Galaxy. It is called
ngsplot.
Whenever I try to do any analysis, I get the following error:
Traceback (most recent call last):
File /home/swarna/galaxy-dist/lib/galaxy/jobs/runners/local.py, line 94,
in queue_job
job_wrapper.finish( stdout, stderr, exit_code )
File /home/swarna/galaxy-dist/lib/galaxy/jobs/__init__.py, line 1009, in
finish
self.sa_session.flush()
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/scoping.py,
line 114, in do
return getattr(self.registry(), name)(*args, **kwargs)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py,
line 1718, in flush
self._flush(objects)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/session.py,
line 1789, in _flush
flush_context.execute()
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py,
line 331, in execute
rec.execute(self)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/unitofwork.py,
line 475, in execute
uow
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py,
line 59, in save_obj
mapper, table, update)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/orm/persistence.py,
line 485, in _emit_update_statements
execute(statement, params)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1449, in execute
params)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1698, in _execute_context
context)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/base.py,
line 1691, in _execute_context
context)
File
/home/swarna/galaxy-dist/eggs/SQLAlchemy-0.7.9-py2.7-linux-x86_64-ucs4.egg/sqlalchemy/engine/default.py,
line 331, in do_execute
cursor.execute(statement, parameters)
ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless
you use a text_factory that can interpret 8-bit bytestrings (like text_factory
= str). It is highly recommended that you instead just switch your application
to Unicode strings. u'UPDATE job SET update_time=?, stdout=?, stderr=? WHERE
job.id = ?' ('2013-07-17 14:56:30.112896', '\nzip error: Nothing to do!
(data.zip)\n', 'mv: cannot stat \xe2\x80\x98data.zip\xe2\x80\x99: No such file
or directory\n', 41)
Tool execution generated the following error message:
Unable to finish job
Can you kindly help me?
With warmest regards,
Chadi EL Farran (Post Graduate Student)
Dr. Jonathan Yuin-Han LOH's Laboratory
#08-01, Institute of Molecular and Cell Biology (IMCB)
61 Biopolis Drive, Proteos
Singapore 138673
Contact No: +65-91944793
E-mail: ch...@nus.edu.sg
From: Hans-Rudolf Hotz [h...@fmi.ch]
Sent: Wednesday, July 17, 2013 3:54 PM
To: Chadi Abdul Kader El Farran
Cc: galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy Local Admin user
Please keep all replies on the list by using reply all !
for my reply see below...
On 07/17/2013 06:58 AM, Chadi Abdul Kader El Farran wrote:
Dear Hans-Rudolf,
Thank you for your reply.
The problem was solved by restarting.
Now my problem is that I am unable to upload files into my Galaxy.
I have put the following code in Universe_wsgi.ini:
# Directories of files contained in the following directory can be uploaded to
a library from the Admin view
library_import_dir = /upload
However when I upload files to a data library, all files will seem to have no
information (0 bytes)
That is very crypticand you need to provide more details
- did you get an error?
- how do you know the files are empty
- what happens if you work with the files
make sure to follow the wiki page:
http://wiki.galaxyproject.org/Admin/DataLibraries/Libraries
Hans-Rudolf
Can you kindly help me?
Thanks a lot for the support. With warmest regards
Chadi EL Farran (Post Graduate Student)
Dr. Jonathan Yuin-Han LOH's Laboratory
#08-01, Institute of Molecular and Cell Biology (IMCB)
61 Biopolis Drive,
[galaxy-dev] Dynamically Generated Tools
TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however, if that's not the recommended course of action, then I'd like some advice on what to do? Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
On Wed, Jul 17, 2013 at 4:58 PM, Michael E. Cotterell mepcotter...@gmail.com wrote: TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. For those not at GCC2013, this was the talk presented by Jessica C. Kissinger, Extension of Galaxy to Utilize Web Services and A Semantic Suggestion Engine Abstract: http://wiki.galaxyproject.org/Events/GCC2013/Abstracts#Events.2FGCC2013.2FAbstracts.2FTalks.Extension_of_Galaxy_to_Utilize_Web_Services_and_A_Semantic_Suggestion_Engine Slides: http://wiki.galaxyproject.org/Documents/Presentations/GCC2013?action=AttachFiledo=viewtarget=KissingerWebServices.pdf Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however, if that's not the recommended course of action, then I'd like some advice on what to do? There is a feature request, that you can specify in your toolshed-repository the preferred location in the tool panel. For now I would ignore it and just install the tools somewhere and hope that feature request is quiet high in Gregs ToDo list :) Cheers, Bjoern Thanks! Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] uninstalling migrated tool
Hello Shantanu, I've created a Trello card for this - we'll take a look as soon as possible. https://trello.com/c/1hXOJPlZ/994-toolshed-from-shantanu-pavgi-on-7-12-13-it-seems-like-if-we-uninstall-a-migrated-tool-then-it-s-entry-in-the-migrated-tools-conf Thanks, Greg Von Kuster On Jul 12, 2013, at 12:42 PM, Shantanu Pavgi (Campus) pa...@uab.edu wrote: It seems like if we uninstall a migrated tool then it's entry in the migrated_tools_conf.xml file doesn't get deleted. Does it need to be removed manually? -- Thanks, Shantanu ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in a tool shed repo, however, if that's not the recommended course of action, then I'd like some advice on what to do? There is a feature request, that you can specify in your toolshed-repository the preferred location in the tool panel. For now I would ignore it and just install the tools somewhere and hope that feature request is quiet high in Gregs ToDo list :) Cheers, Bjoern ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Galaxy load problem
Hi, Ken, You don't reveal much about your local configuration making it hard to offer useful advice - and of course Postgres is a different ball of wax. FWIW I noticed much lower idle cpu loads when I switched from the default setup to running 2 handlers on our lightly loaded system here. eg http://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer?action=showredirect=Admin%2FConfig%2FPerformance YMMV On Thu, Jul 18, 2013 at 12:56 AM, Auerbach, Kenneth R. kauerb...@research.bwh.harvard.edu wrote: Hello, I'm running Galaxy on a Linux server and have been getting warning messages about heavy loads on the CPU due to Galaxy. This is a dedicated Galaxy server and is currently used very lightly. There seem to be six idle postgres processes running, but only two jobs running. Any idea why the load is so high? Thank you. Ken. The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/ -- Ross Lazarus MBBS MPH; Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444 http://scholar.google.com/citations?hl=enuser=UCUuEM4J ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto:bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2. What's the best way to add the generated tools to Galaxy? My predecessors modified the tool_config.xml directly and required users to restart Galaxy. Now, If these tools are in a tool shed repo, I know that I can use the add_to_tool_panel() method in lib/tool_shed/util/tool_util.py if these tools are in
[galaxy-dev] Java JAXB for Tool Configuration Files
Hey guys, I don't know if there will be much interest in this, but in the course of revising code that my predecessors wrote, I discovered that they were hand crafting tool XML configuration files. Since we all know that makes reading and writing such files hard, I started work on creating a Java model for the tool config XML using JAXB. In short, I can create and populate some Java objects and then marshal them directly to XML, knowing that they'll be in the correct format. Also, it makes reading/unmarshalling tool config files easy as well. If there's any interest in this, I can probably separate out the model code into a separate project. At the moment, it's a subset of the tool config XML model that I need for my Web service tools, but I could foresee completing it in hopes of it being more useful to others. Imagine: define your tool using Java and have it print out the pretty XML! Also, there should be an XSD for the tool config files. I can generate one once my Java model is complete, but it's kind of important. In the wiki, it says something to the effect that all is needed is valid XML… That's not entirely correct. It should valid XML that conforms to an XML schema definition (now I'm ranting :P). If one already exists, then all the better! Anyway, let me know if there's any interest. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Dynamically Generated Tools
Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is the best place to put the tool config (XML) files that are generated by my tool? My predecessors just placed them in a subdirectory under our tool's directory. I'm ok with this, but does that mean that all the generated files will get removed if they upgrade our tool via the tool shed (haven't taken the time to see if Galaxy just does an hg pull or if it's more complicated than that)? It is mainly a hg pull. But if you deinstall that repository and reinstall it, all your files are lost. Would it be better to have my tool create a local tool shed repository (just for these generated tools), and if so, can the tool shed API let my tool install this repository? Having not looked at install_tool_shed_repositories.py yet, it's unclear from the wiki documentation whether or not POST /api/tool_shed_repositories/install_repository_revision requires that the tool shed already registered in tool_sheds_conf.xml)? I think that is a better way to do it. The API can be used to install your tools. But it need to be configured beforehand from the user, administrator. 2.
Re: [galaxy-dev] Dynamically Generated Tools
I didn't mean to imply it was unclear. It definitely made sense. I completely agree with you. However, is there for a way (maybe via the toolshed api) for my tool to push these tools to a TS repo? Also, doesn't that add some more complexity for the user? Not everyone runs a local tool shed. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote: Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto:bjoern.gruen...@pharmazie.uni-freiburg.de) (mailto:bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to offer that service to every user and is it save? TL;DR I have a tool that generates other tools. Where should those tools go, and how should I add them (without modifying Galaxy code)? I apologize, in advance, for the wall of text. I would like some advice on what the best way to go about setting up a tool that generates other tools... As some of you (especially those who saw my boss's talk at GCC) already know, I'm working on a tool that lets galaxy users add web service operations as tools to Galaxy. We have a general purpose client for invoking web services, and we have a tool that let's a user enter a WSDL/WADL URL, select the operations they want to add, and generates tool config XML files for Galaxy. Right now, I'm in the process of making our tool tool shed compatible, and that's going along great (I especially like the tool dependency installation stuff), but I'm trying to determine the best course of action on the following: 1. Where is
[galaxy-dev] Fwd: Dynamically Generated Tools
AFAIK it's not yet possible, but a tool shed API to expose subversion hooks for repository creation and update is a logical development which is already on the radar AFAIK. For the bigger picture of complexity for the user, I think some key clarifications required include: which 'user' is experiencing complexity; how big a barrier it is to generating tools; how readily can we make it go away; how much will it help? Tool builders (even 'users' of the TF) might reasonably be expected to have a capacity for substantial complexity that the end user of the generated tools should not need to possess - after all, once a generated tool is installed in Galaxy, the Galaxy user can't tell the difference and sees no more complexity from the generated tool than a hand written one. On absent local tool sheds: it's trivial to run a toolshed on a dev laptop and not hard to run in production - but tool makers can always use the public test and main TS if they want to distribute new tools so I'm not sure how big a barrier a missing local toolshed really is. On Thu, Jul 18, 2013 at 12:08 PM, Michael E. Cotterell mepcotter...@gmail.com wrote: I didn't mean to imply it was unclear. It definitely made sense. I completely agree with you. However, is there for a way (maybe via the toolshed api) for my tool to push these tools to a TS repo? Also, doesn't that add some more complexity for the user? Not everyone runs a local tool shed. Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 10:05 PM, Ross wrote: Sorry if my post was rambling and unclear. Here's the executive summary for what it's worth: IMHO, a Galaxy tool that generates tools should emit them as tool shed compatible artefacts, ready for uploading to a TS repository for automated installation to any target instance. On Thu, Jul 18, 2013 at 11:25 AM, Michael E. Cotterell mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) wrote: Ross, Thanks for the comments. I've already made my main tool tool_shed compatible. What I'm wondering about is what do I do with the tools that my tool creates? Sincerely, Michael E. Cotterell Ph.D. Student in Computer Science, University of Georgia Instructor of Record, Graduate RA TA, University of Georgia Faculty Liaison, CS Graduate Student Association, University of Georgia mepcotter...@gmail.com (mailto:mepcotter...@gmail.com) (mailto: mepcotter...@gmail.com) mepc...@uga.edu (mailto:mepc...@uga.edu) (mailto:mepc...@uga.edu) m...@cs.uga.edu (mailto:m...@cs.uga.edu) (mailto:m...@cs.uga.edu) http://michaelcotterell.com/ On Wednesday, July 17, 2013 at 7:44 PM, Ross wrote: Hi, Michael, As others have said, in the long term, tool wrappers and their dependencies will be distributed and installed through tool sheds rather than being distributed with Galaxy source, so you might want to plan to generate and upload tool shed compatible archives from the get-go. The test and main tool sheds contain plenty of examples and Björn's tools exercise pretty much all of the functionality - you can browse the repo structure which is identical to the gz archive structure you need to upload to a new repo, or download the repo as a gz, unpack it and get exactly the kind of directory structure and contents you need to emulate for your tools. Once you have a working tool packaging the archive up is straightforward. The format for tool shed repo uploads and syntax for the tags used to define dependencies is very well documented in the tool shed section of the wiki. As Björn points out, the tool factory python wrapper might be a useful source of ideas and perhaps code. Your tool generator will need to do something similar to write the content and generate complete tool shed archives. When generating a new tool, the TF uses an ugly XML wrapper generator (contributed improvements would be very welcome!) and (probably more usefully) the few lines of code you need to package up the functional test data and the XML and wrapper if you need one in toolshed archive format. On Thu, Jul 18, 2013 at 5:30 AM, Björn Grüning bjoern.gruen...@pharmazie.uni-freiburg.de (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) (mailto: bjoern.gruen...@pharmazie.uni-freiburg.de) wrote: Hi Michael, I think you will enter new ground with your tool. The closest tool that will do something similar is Ross toolfactory, I think: http://toolshed.g2.bx.psu.edu/view/fubar/toolfactory For me one question is, do you really want to
