Re: [galaxy-dev] [galaxy-user] How to create a tool from a script that loads additional source code files?
On Mon, Feb 25, 2013 at 8:02 PM, Samuel Lampa samuel.la...@gmail.com wrote: Hi, thanks for the quick reply! On 02/25/2013 08:39 PM, Peter Cock wrote: I realize that when the master script (run.R in my case) is run, it will not find the other files, since it is itself moved away to a temporary working directory (something like .../galaxy/database/job_working_directory/000/165). Is there any standard way of dealing with this, any special toolconfig syntax etc? ... or do I have to just hard code the absolute paths to the extra files in my script? :| Presumably all the child R scripts will be in the same folder as your master R script (or you know the relative path if not). So all you need to do is workout the path of the master R script, and from that compute the path to the child R scripts. I guess you can do this in R using commandArgs(). True. And yes, I do keep the other scripts in the same folder. But since the main script is moved away to the job_working_directory (but not the other scripts, as far as I can tell), they will not be in the same folder anymore, ... Are you just calling a pre-existing static R script, or doing something more complicated like generating the R script in the tool XML file? For calling pre-existing scripts or binaries, I thought it was just the current working directory which was changed to something like job_working_directory. Regards, Peter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] error installing BWA from toolshed
Hi Greg, Works. Thanks. Is it possible to create a branch call 'latest_stable', which always points to the latest stable release so then on our end, we can always do 'hg update latest_stable' in one of our scripts to update rather using 'release_2013.02.08'? Thanks, On Mon, Feb 25, 2013 at 8:52 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Quang, It looks like you are attempting to install from the main Galaxy tool shed into a Galaxy instance that is older than the December 20, 2012 Galaxy release. You have to be using that release or newer (up to the current stable branch) to use the main Galaxy tool shed. To get the latest stable branch, you can do this: hg update release_2013.02.08 Then to get all updates to the stable branch, you can: hg pull -b stable https://bitbucket.org/galaxy/galaxy-central followed by: hg update Greg Von Kuster On Feb 25, 2013, at 8:35 AM, Quang Trinh wrote: Hi dev, I am trying to install bwa from toolshed on my Galaxy instance on Amazon just now and get this error ( please see below). Installing bwa worked for me before. URL: http://xxx.xxx.xxx.xxx/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=25bdd3c7f0d57b06changeset_revisions=ffa8aaa14f7c Module paste.exceptions.errormiddleware:143 in __call__ app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ environ, self.app) Module paste.wsgilib:539 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ body = method( trans, **kwargs ) Module galaxy.web.framework:206 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1199 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Thanks, Q ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] error installing BWA from toolshed
On Feb 26, 2013, at 8:23 AM, Quang Trinh wrote: Hi Greg, Works. Thanks. Is it possible to create a branch call 'latest_stable', which always points to the latest stable release so then on our end, we can always do 'hg update latest_stable' in one of our scripts to update rather using 'release_2013.02.08'? Hi Quang, Once your repository is on the stable branch (you can check using `hg branch`, then `hg pull` followed by `hg update` will always get you the latest stable changes. This may include some changes committed after the release, however. --nate Thanks, On Mon, Feb 25, 2013 at 8:52 AM, Greg Von Kuster g...@bx.psu.edu wrote: Hello Quang, It looks like you are attempting to install from the main Galaxy tool shed into a Galaxy instance that is older than the December 20, 2012 Galaxy release. You have to be using that release or newer (up to the current stable branch) to use the main Galaxy tool shed. To get the latest stable branch, you can do this: hg update release_2013.02.08 Then to get all updates to the stable branch, you can: hg pull -b stable https://bitbucket.org/galaxy/galaxy-central followed by: hg update Greg Von Kuster On Feb 25, 2013, at 8:35 AM, Quang Trinh wrote: Hi dev, I am trying to install bwa from toolshed on my Galaxy instance on Amazon just now and get this error ( please see below). Installing bwa worked for me before. URL: http://xxx.xxx.xxx.xxx/admin_toolshed/prepare_for_install?tool_shed_url=http://toolshed.g2.bx.psu.edu/repository_ids=25bdd3c7f0d57b06changeset_revisions=ffa8aaa14f7c Module paste.exceptions.errormiddleware:143 in __call__ app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ environ, self.app) Module paste.wsgilib:539 in intercept_output app_iter = application(environ, replacement_start_response) Module paste.recursive:80 in __call__ return self.application(environ, start_response) Module paste.httpexceptions:632 in __call__ return self.application(environ, start_response) Module galaxy.web.framework.base:160 in __call__ body = method( trans, **kwargs ) Module galaxy.web.framework:206 in decorator return func( self, trans, *args, **kwargs ) Module galaxy.webapps.galaxy.controllers.admin_toolshed:1199 in prepare_for_install description, repository_clone_url, changeset_revision, ctx_rev, repository_owner, tool_dependencies = repo_info_tuple ValueError: too many values to unpack Thanks, Q ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Macs14 - Invalid Literal for int error
Hi guys, (Sorry for showing up on this list so much, hopefully I'll get everything running soon!) On our local galaxy install when I try to run MACS14 like this: http://snag.gy/RYBBN.jpg we get this error: Dataset generation errors Dataset 74: MACS14 on data 29 and data 24 (peaks: bed) Tool execution generated the following error message: Traceback (most recent call last): File /misc/local/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/ryo-tas/macs14/cdd9791c0afa/macs14/macs14_wrapper.py, line 132, in module if __name__ == __main__: main() File /misc/local/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/ryo-tas/macs14/cdd9791c0afa/macs14/macs14_wrapper.py, line 94, in main xls_to_interval( create_peak_xls_file, options['xls_to_interval']['peaks_file'], header = 'peaks file' ) File /misc/local/galaxy/shed_tools/toolshed.g2.bx.psu.edu/repos/ryo-tas/macs14/cdd9791c0afa/macs14/macs14_wrapper.py, line 40, in xls_to_interval fields[1] = str( int( fields[1] ) - 1 ) ValueError: invalid literal for int() with base 10: 'start' ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Downloading datasets without username/password when login is required
Great, thanks John! Ilya On 2/26/13 12:18 AM, John Chilton chil...@msi.umn.edu wrote: I have also encountered the need for this and believe it is important. I have issued a pull request implementing this. https://bitbucket.org/galaxy/galaxy-central/pull-request/131/implement-equ ivalent-of-dataset-display/diff Thanks, -John On Thu, Dec 13, 2012 at 2:20 PM, Sytchev, Ilya isytc...@hsph.harvard.edu wrote: So, it turns out that it's currently impossible to download datasets programmatically without using username/password (when require_login is set to True) because Galaxy API does not have functionality to download datasets. Are there any plans to implement this feature? Thanks, Ilya On 12/11/12 2:41 PM, Sytchev, Ilya isytc...@hsph.harvard.edu wrote: Hi, Is it possible to download a dataset when require_login=True in universe_wsgi.ini using the API key? Thanks, Ilya ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ On Thu, Dec 13, 2012 at 2:20 PM, Sytchev, Ilya isytc...@hsph.harvard.edu wrote: So, it turns out that it's currently impossible to download datasets programmatically without using username/password (when require_login is set to True) because Galaxy API does not have functionality to download datasets. Are there any plans to implement this feature? Thanks, Ilya On 12/11/12 2:41 PM, Sytchev, Ilya isytc...@hsph.harvard.edu wrote: Hi, Is it possible to download a dataset when require_login=True in universe_wsgi.ini using the API key? Thanks, Ilya ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] How to test DRMAA configuration?
Thanks for your response. That seemed to work, to a point. I redid the same FASTQ-FASTA conversion test and could see using command-line utilities that the job started on the grid (SGE is behind it.) About 10 minutes have gone by though since the job stopped running on the grid and the interface is still spinning as if it's running. I checked the Galaxy log for that job ID and I see these entries: galaxy.jobs.runners.drmaa INFO 2013-02-26 10:51:15,670 (103) queued as 5224332 galaxy.jobs.runners.drmaa DEBUG 2013-02-26 10:51:16,390 (103/5224332) state change: job is queued and active galaxy.jobs.runners.drmaa DEBUG 2013-02-26 10:51:30,422 (103/5224332) state change: job is running galaxy.jobs.runners.drmaa DEBUG 2013-02-26 10:52:32,623 (103/5224332) is still in running state, adding to the DRM queue galaxy.jobs.runners.drmaa DEBUG 2013-02-26 10:54:31,961 (103/5224332) job left DRM queue with following message: code 18: The job specified by the 'jobid' does not exist. The SGE qacct utility for that job shows me that it took 3.5 minutes to run and that the return status was 0, so it executed successfully. It seems that Galaxy is missing the completion event for the job? Joshua On Tue, Feb 26, 2013 at 12:17 AM, Ross ross.laza...@gmail.com wrote: See if adding the default queue name to the job runner path - eg: default_cluster_job_runner = drmaa:///default works any better? Galaxy will default to the local runner if it can't find the nominated drmaa path AFAIK and I don't think 'default' is the default :) On Tue, Feb 26, 2013 at 4:51 PM, Joshua Orvis jor...@gmail.com wrote: I have a working local Galaxy instance and wanted to enable DRMAA support to utilize our SGE (or LSF) grid. Following the guide herehttp://wiki.galaxyproject.org/Admin/Config/Performance/Cluster I set what I appeared to need to make this work. From the DRMAA_LIBRARY_PATH env variable to all the configuration settings in universe_wsgi.ini, reconfiguring the server hosting Galaxy as a submit host, etc. Some specific config file changes made: new_file_path = /seq/gscidA/www/gscid_devel/htdocs/galaxy-dist/database/tmp start_job_runners = drmaa default_cluster_job_runner = drmaa:/// set_metadata_externally = True outputs_to_working_directory = True I then killed and restarted the Galaxy instance and tried a simple FASTQ - FASTA test execution, but it ran locally. I couldn't find any sort of errors or messages related to DRMAA in the server log, and the job ran to completion. I commented out the local tool runner overrides. What can I do to test my DRMAA configuration and where should I look for errors? Thanks - Joshua ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] bowtie-wrapper
Hello, I am using the bowtie-wrapper on my locally installed galaxy server to align reads. However I missing the stats read-out. Is there any possibility to include statistics about unaligned reads? If I use bowtie vi comman line I get following output: bowtie ~/dm3 -v 2 -k 5 --best --strata -S -t reads.fastq reads.sam End-to-end 2/3-mismatch full-index search: 01:00:21 # reads processed: 12084153 # reads with at least one reported alignment: 9391748 (77.72%) # reads that failed to align: 2692405 (22.28%) Reported 30293838 alignments to 1 output stream(s) The output should be normally saved in the stderr but unfortunatly the stderr is somehow deleted after the alignment job is done in bowtie under galaxy. Any idea how I can still access the stats? Thanks, Alex -- Alexander Kurze, DPhil University of Oxford Department of Biochemistry South Parks Road Oxford, OX1 3QU United Kingdom Tel: +44 1865 613 230 Fax:+44 1865 613 341 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] GALAXY LOCAL INSTALLATION: Tools and Data
Hi Rafa, The .loc files in 'tool-data' directory link the index files with the actual paths so you need to configure each of the .loc files to match where the index files are placed. With that, you can put the index files anywhere you'd like on the file system (preferably outside Galaxy's home directory). As a reference for a working setup, you can rsync (portion) of the data from GalaxyMain (http://wiki.galaxyproject.org/Admin/Data%20Integration) and see how that's setup. Also, you can fire up a cloud instance and see how it's setup. Hope this helps, Enis On Thu, Feb 21, 2013 at 12:14 AM, Rafael Hernández rafah...@gmail.comwrote: Hi, I've installed Galaxy in local and following the Tool dependencieshttp://wiki.galaxyproject.org/Admin/Tools/Tool%20Dependencies manual I installed all necessary tools. However for some of then such as BWA, Bowtie,... it's usually needed to create indexes. Where should I store those indexes? In the galaxy directory or in each tool directory? How can I tell galaxy where the indexes are? I find a manual's page talking about it / http://wiki.galaxyproject.org/Admin/NGS%20Local%20Setup), suggesting to organize the various index files for each build, but I don't know if these folders should be stored into Galaxy directory (eg. in tool-data directory) or what. Could you please explain this a little? Thank you and regards, Rafa Hernández de Diego Genomics of Gene Expression Lab. Bioinformatics and Genomics Department Centro de Investigación Príncipe Felipe (CIPF) C/ Eduardo Primo Yúfera 3 46012 Valencia, Spain ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-dev] Resizing EBS on Amazon
Used cloud man to create a new cluster on Feb 22 and picked 500GB as the initial size of the data drive. Working with TCGA exome DNA seq data didn't take long to fill that up. Used the cloud man admin interface to resize from 500GB to 1TB and the resize operation took 15 hours. Not sure if that is expected so wanted to give some heads up in case that is an area for optimization. Since I now have a local storage problem as I need to work with more than 1TB of data I tried to go the route of setting a S3 bucket using Fuse. Ran into a problem where the first s3fs software I tried to install had a version issue with Ubuntu 10. I remember something in a support email that better support for Amazon S3 was in the works. Can you provide any guidance or thoughts on how to work with more than 1TB of data using cost effective S3 versus expensive EBS? The same applies for storing results at S3. With s3fs the file system can hide many of the complexities of moving files back and forth with caching where working with 30GB+ files isn't going to be fun. Thanks Scooter ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
