[galaxy-dev] Changing job command line.
Hi All, I am currently working on a clustering interface for galaxy that will hopefully enable users to pick the grid in the tool form. With tools that need access to files within the galaxy directory, I have an idea to create a symlinked fake galaxy root with all the files the tools needs for galaxy. This is done currently in my own xml format for each grid. Problem is when parsing a tool_wrapper variable I need to prepare the job and add my additional commands to the command line so that the fake galaxy dir is created with the symlinked databases. Is it bad practice to disable preparation in each tool runner if the job has come from my clustering module or is there a better way to do this. Will the job runners wrapper.prepare() overwrite any of my changes to the command-line when the job makes its way to the runner say local.py. Cheers James. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Programming error in my new tool shape for shape it
ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, state=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-01-14 07:17:20.417990', 'error', '', '\n\n\x1b[33mS\x1b[0megmented \x1b[33mHAP\x1b[0mlotype \x1b[33mE\x1b[0mstimation \x1b[33mI\x1b[0mmputation \x1b[33mT\x1b[0mool\ That is just a sample of the stderr from the galaxy stack trace. I think that shapeit is using 8 bit character codes in the stdout and galaxy is failing on adding to the database. Has anyone experienced this problem and could help me. Cheers James ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Programming error in my new tool for shapeit.
ProgrammingError: (ProgrammingError) You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. u'UPDATE job SET update_time=?, state=?, stdout=?, stderr=? WHERE job.id = ?' ['2013-01-14 07:17:20.417990', 'error', '', '\n\n\x1b[33mS\x1b[0megmented \x1b[33mHAP\x1b[0mlotype \x1b[33mE\x1b[0mstimation \x1b[33mI\x1b[0mmputation \x1b[33mT\x1b[0mool\ That is just a sample of the stderr from the galaxy stack trace. I think that shapeit is using 8 bit character codes in the stdout and galaxy is failing on adding to the database. Has anyone experienced this problem and could help me. Cheers James ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool error occurred when adding my tool to galaxy
Hi Tan, If you go into your tool xml file I think you will find a tag error the traceback. Something like type=select you may have put thype a simple typo. Happy Holidays James Boocock On Sat, 2012-12-29 at 16:36 +0800, Tan Jun wrote: Hi, all I'm new here. These days I'm trying to add a tool in Galaxy, as showed in screencast. I did the same thing with that in screencast. But an error occurred. The error says: tool error An error occurred running this job:failure preparing job The message in the terminal says: galaxy.jobs.handler DEBUG 2012-12-29 15:54:25,504 dispatching job 7 to local runner galaxy.jobs.handler INFO 2012-12-29 15:54:25,733 (7) Job dispatched galaxy.jobs.runners.local DEBUG 2012-12-29 15:54:25,818 Local runner: starting job 7 galaxy.jobs.runners.local ERROR 2012-12-29 15:54:26,061 failure running job 7 Traceback (most recent call last): File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/runners/local.py, line 72, in run_job job_wrapper.prepare() File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/__init__.py, line 180, in prepare config_filenames = self.tool.build_config_files( param_dict, self.working_directory ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/tools/__init__.py, line 2430, in build_config_files f.write( fill_template( template_text, context=param_dict ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, line 1004, in __str__ return getattr(self, mainMethName)() File cheetah_DynamicallyCompiledCheetahTemplate_1356767666_06_36079.py, line 149, in respond KeyError: 'thype' 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b/contents?ids=ebfb8f50c6abde6d HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 Could anybody help me? Thanks! __ === Regards! School of Computer Engineering and Science Shanghai University, Shanghai China Tan Jun 谭军 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Tool error occurred when adding my tool to galaxy
Hi Tan, If you go into your tool xml file I think you will find a tag error the traceback. Something like type=select you may have put thype a simple typo. Happy Holidays James Boocock On Sat, 2012-12-29 at 16:36 +0800, Tan Jun wrote: Hi, all I'm new here. These days I'm trying to add a tool in Galaxy, as showed in screencast. I did the same thing with that in screencast. But an error occurred. The error says: tool error An error occurred running this job:failure preparing job The message in the terminal says: galaxy.jobs.handler DEBUG 2012-12-29 15:54:25,504 dispatching job 7 to local runner galaxy.jobs.handler INFO 2012-12-29 15:54:25,733 (7) Job dispatched galaxy.jobs.runners.local DEBUG 2012-12-29 15:54:25,818 Local runner: starting job 7 galaxy.jobs.runners.local ERROR 2012-12-29 15:54:26,061 failure running job 7 Traceback (most recent call last): File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/runners/local.py, line 72, in run_job job_wrapper.prepare() File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/jobs/__init__.py, line 180, in prepare config_filenames = self.tool.build_config_files( param_dict, self.working_directory ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/tools/__init__.py, line 2430, in build_config_files f.write( fill_template( template_text, context=param_dict ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/lib/galaxy/util/template.py, line 9, in fill_template return str( Template( source=template_text, searchList=[context] ) ) File /home/tanjun/galaxy/galaxy-galaxy-dist-36ad8aa7b922/eggs/Cheetah-2.2.2-py2.7-linux-x86_64-ucs4.egg/Cheetah/Template.py, line 1004, in __str__ return getattr(self, mainMethName)() File cheetah_DynamicallyCompiledCheetahTemplate_1356767666_06_36079.py, line 149, in respond KeyError: 'thype' 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 127.0.0.1 - - [29/Dec/2012:15:54:27 +0800] GET /api/histories/f597429621d6eb2b/contents?ids=ebfb8f50c6abde6d HTTP/1.1 200 - http://localhost:8080/history; Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.19 (KHTML, like Gecko) Ubuntu/11.10 Chromium/18.0.1025.168 Chrome/18.0.1025.168 Safari/535.19 Could anybody help me? Thanks! __ === Regards! School of Computer Engineering and Science Shanghai University, Shanghai China Tan Jun 谭军 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] XGRID and Galaxy
Hi Galaxy Dev Team. We are looking to put xgrid into the galaxy api as part of a summer project. We are hoping to add this functionality directly into galaxy so that all users can benefit. We are curious where to start investigating how galaxy interfaces with grids. We hope to integrate the functionality into galaxy so that a user can specify xgrid in the config file and galaxy will deal with the rest of the interfacing with xgrid. This would of course only be avaliable to MACOSX galaxy installs. Cheers James. Research Assistant Department of Biochemistry University Of Otago ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] VCF compression.
Hi All, Just a quick question regarding file compression. We have a galaxy instance with limited space and a few people are trying to start working on our server. They will mainly be working with vcfs is there a way to make galaxy keep the vcfs in gz format until they are needed. I know this will slow down operations but it is a requirement as the will be copying the vcfs around etc. Cheers James. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Galaxy Tool Creation Help
Hi All,
Just a quick question regarding galaxy tool creation.
In the command tags
Is it possible to get a datasets history number for the history
the user initiated the job from?
Seems unlikely, the reason I ask is
that I have a tool that creates and output file for every input file
and the program can take multiple inputs. It would be useful to
be able to label the outputs related to the history number so the
user could easily identify which output is for which file.
Using the multiple output guide something like. primary_${id}_${history
dataset number}_visible_txt
Cheers
James.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Galaxy tool creation help
Hi Guys
Regarding my earlier question about history names.
I found my answer on the wiki. ${input1.name} was
what I was looking for.
Cheers
James.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] broken pipe when running VCF-merge.
Hi,
We are attempting to integrate vcftools specifically vcf-merge into
galaxy. We have two or files that we
are attempting to merge, the command when run through terminal does not
give a broken pipe, but
when galaxy runs the command a broken pipe is given at the beginning and
the start of the execution.
The output from the merge appears to be exactly the same. But the broken
pipe error causes galaxy
to go read on the output wondering if anyone can help that would be much
appreciated.
The shell script is pasted below.
_SHELL SCRIPT THAT GALAXY RUNS
_
1 #!/bin/bash
2 # @Date 9/01/2012
3 # @Author Ed Hills, James Boocock
4 #
5 # This file will take a white space separated list of file names,
6 # and run the vcf-merge tool and merge all files into a new file.
7 #
8 # INPUTS
9 # $1 = First input File
10 # $2 = Second input File
11 # $N = Extra input files
12
13 FILE_LIST=
14
15 cat $1 | bgzip -c ~tmp1.vcf.gz
16 cat $2 | bgzip -c ~tmp2.vcf.gz
17
18 tabix -p vcf ~tmp1.vcf.gz
19 tabix -p vcf ~tmp2.vcf.gz
20
21 FILE_LIST=~tmp1.vcf.gz ~tmp2.vcf.gz
22
23 if [ $# 2 ]
24 then
25 for ((i=3; i = $#; i++))
26 do
27 eval EXTRA_FILE=\$${i}
28 cat $EXTRA_FILE | bgzip -c ~tmp${i}.vcf.gz
29 tabix -p vcf ~tmp${i}.vcf.gz
30 FILE_LIST=${FILE_LIST} ~tmp${i}.vcf.gz
31 done
32 fi
33
34 perl ~/galaxy-dist/tool-data/shared/vcfperltools/vcf-merge
${FILE_LIST} 2 /dev/null
35
36 rm -f ~tmp*
37
_ERROR MESSAGE_
Warning: The AC tag(s) will not be merged correctly for multiallelic sites.
To be handled correctly, please redefine with Number=. or set the environment
variable DONT_FIX_VCF40_AG_TAGS=0.
gzip: stdout: Broken pipe
Using column name 'HG00096' for ~tmp1.vcf.gz:HG00096
gzip: stdout: Broken pipe
Using column name 'NA06986' for ~tmp2.vcf.gz:NA06986
Cheers James.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] broken pipe when integrating VCF-merge from vcf-tools
Hi,
We are attempting to integrate vcftools specifically vcf-merge into
galaxy. We have two or files that we
are attempting to merge, the command when run through terminal does not
give a broken pipe, but
when galaxy runs the command a broken pipe is given at the beginning and
the start of the execution.
The output from the merge appears to be exactly the same. But the broken
pipe error causes galaxy
to go read on the output wondering if anyone can help that would be much
appreciated.
The shell script is pasted below.
_SHELL SCRIPT THAT GALAXY RUNS
_
1 #!/bin/bash
2 # @Date 9/01/2012
3 # @Author Ed Hills, James Boocock
4 #
5 # This file will take a white space separated list of file names,
6 # and run the vcf-merge tool and merge all files into a new file.
7 #
8 # INPUTS
9 # $1 = First input File
10 # $2 = Second input File
11 # $N = Extra input files
12
13 FILE_LIST=
14
15 cat $1 | bgzip -c ~tmp1.vcf.gz
16 cat $2 | bgzip -c ~tmp2.vcf.gz
17
18 tabix -p vcf ~tmp1.vcf.gz
19 tabix -p vcf ~tmp2.vcf.gz
20
21 FILE_LIST=~tmp1.vcf.gz ~tmp2.vcf.gz
22
23 if [ $# 2 ]
24 then
25 for ((i=3; i = $#; i++))
26 do
27 eval EXTRA_FILE=\$${i}
28 cat $EXTRA_FILE | bgzip -c ~tmp${i}.vcf.gz
29 tabix -p vcf ~tmp${i}.vcf.gz
30 FILE_LIST=${FILE_LIST} ~tmp${i}.vcf.gz
31 done
32 fi
33
34 perl ~/galaxy-dist/tool-data/shared/vcfperltools/vcf-merge
${FILE_LIST} 2 /dev/null
35
36 rm -f ~tmp*
37
_ERROR MESSAGE_
Warning: The AC tag(s) will not be merged correctly for multiallelic sites.
To be handled correctly, please redefine with Number=. or set the environment
variable DONT_FIX_VCF40_AG_TAGS=0.
gzip: stdout: Broken pipe
Using column name 'HG00096' for ~tmp1.vcf.gz:HG00096
gzip: stdout: Broken pipe
Using column name 'NA06986' for ~tmp2.vcf.gz:NA06986
Cheers James.
___
Please keep all replies on the list by using reply all
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] error when running UNIT_TEST script
Hi When running the run_unit_tests.sh script on my server i get the following Error. Wondering what It means and whether It is something I have done or something within galaxy #Start Error Doctest: galaxy.web.form_builder.TextArea ... ok TextField (galaxy.web.form_builder) Doctest: galaxy.web.form_builder.TextField ... ok == ERROR: Failure: ImportError (No module named tables) -- Traceback (most recent call last): File /home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/loader.py, line 379, in loadTestsFromName addr.filename, addr.module) File /home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 39, in importFromPath return self.importFromDir(dir_path, fqname) File /home/james/galaxy-dist/eggs/nose-0.11.1-py2.7.egg/nose/importer.py, line 86, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File /home/james/galaxy-dist/lib/galaxy/datatypes/converters/bed_to_genetrack_converter.py, line 16, in module from genetrack.scripts import tabs2genetrack File /home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/__init__.py, line 38, in module from genetrack.hdflib import PositionalData File /home/james/galaxy-dist/eggs/GeneTrack-2.0.0_beta_1_dev_48da9e998f0caf01c5be731e926f4b0481f658f0-py2.7.egg/genetrack/hdflib.py, line 5, in module from tables import openFile ImportError: No module named tables -- Ran 60 tests in 11.380s #End Error Cheers James ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] administrator Interface.
Hi, I have setup a galaxy production server and created a admins.txt file with comma separated account names. When running galaxy and logged into one of the activated accounts I cannot view the interface. Configs Listed Below Cheers James. #Config File file:universe config path:/home/galaxy/galaxy-dist/universe-wsgi. Config line related to the admins folder. admin_users = /home/galaxy/galaxy-dist/tools/SOER1000genes/galaxy/admins.txt #Admin.txt. file:admins.txt path:/home/galaxy/galaxy-dist/tools/SOER1000genes/galaxy/admins.txt #file contents# admin_users=sfk2...@gmail.com ## When I Log in to the account above in galaxy I cannot view the admin interface. I am confused as to what I have done wrong or possibly stupid. ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] FTP GALAXY
Hi We have set up galaxy in a production environment and our galaxy users can connect to the FTP and upload files. No worries. The settings in the universe file seem correct and are. ftp_upload_dir = /home/galaxy/galaxy-dist/database/ftp It creates the directory and uploads the file. But when the user signs into galaxy they cannot view their file in the get data - from files on my computer. Proftp config is listed below. Cheers James Boocock ServerName Public Galaxy FTP ServerType standalone DefaultServer on Port21 Umask 077 SyslogFacility DAEMON SyslogLevel debug MaxInstances30 Usernobody Group nogroup DisplayConnect /etc/opt/local/proftpd_welcome.txt # Passive port range for the firewall PassivePorts3 4 # Cause every FTP user to be jailed (chrooted) into their home directory DefaultRoot ~ # Automatically create home directory if it doesn't exist CreateHome on dirmode 1700 # Allow users to overwrite their files AllowOverwrite on # Allow users to resume interrupted uploads AllowStoreRestart on # Bar use of SITE CHMOD Limit SITE_CHMOD DenyAll /Limit # Bar use of RETR (download) since this is not a public file drop Limit RETR DenyAll # Do not authenticate against real (system) users AuthPAM off # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxydb galaxyftp 1234 SQLAuthTypesSHA1 SQLAuthenticate users # An empty directory in case chroot fails SQLDefaultHomedir /var/opt/local/proftpd # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. # Do not authenticate against real (system) users AuthPAM off # Set up mod_sql_password - Galaxy passwords are stored as hex-encoded SHA1 SQLPasswordEngine on SQLPasswordEncoding hex # Set up mod_sql to authenticate against the Galaxy database SQLEngine on SQLBackend postgres SQLConnectInfo galaxydb galaxyftp 1234 SQLAuthTypesSHA1 SQLAuthenticate users # An empty directory in case chroot fails SQLDefaultHomedir /var/opt/local/proftpd # Define a custom query for lookup that returns a passwd-like entry. UID and GID should match your Galaxy user. SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT email,password,'512','512','/home/galaxy/galaxy-dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U' SQLUserInfo custom:/LookupGalaxyUser SQLNamedQuery LookupGalaxyUser SELECT email,password,'512','512','/home/galaxy/galaxy-dist/database/ftp/%U','/bin/bash' FROM galaxy_user WHERE email='%U' /Limit ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
