Re: [galaxy-user] biomart plugin
Hello Andrea, Make a copy of the ~/tools/data_source/biomart.xml for your local biomart install, and change the action in the following tag to point to it. inputs action=http://www.biomart.org/biomart/martview; check_values=false method=get target=_top Then add your new biomart tool wrapper to your tool_conf.xml file and start up your Galaxy instance. For details about adding a new tool, see https://bitbucket.org/galaxy/galaxy-central/wiki/AddToolTutorial. Greg Von Kuster On Mar 26, 2011, at 10:19 AM, Andrea Edwards wrote: Hello I have looked at the biomart plugin for Galaxy and this seems to allow access to marts on the biomart central server. Is there anyway to use this plugin to access a biomart on my server if I can't make my biomart available on the biomart central server. If not, would it be possible to achieve this with galaxy tools. I've heard of galaxy tools but never made one so I don;t know what is involved. Failing that would i be able to use the biomart plugin to access my server if I had a local installation of galaxy thanks ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Convert SOLiD data
Lishiyong, You should not convert colorspace to base space prior to aligning reads. The reason for this is that if there is an error in one of the color calls, it will effect all the downstream color calls. Instead, you should use an aligner that will do the assembly in color-space instead. I know there are a few out there, but don't know them off the top of my head. Ryan On 3/27/11 11:35 PM, lishiyong wrote: Hello! I convert SOLiD csfasta- and qual-files to fastq-files ,I want to use the fastq-files to do denovo with *SOAPdenovo*.because the SOLiD de novo accessory tools Require large memory .But ,I find that there're some question for the converting . for example: T0202322110210103200200203001123212113333311200 —— AGAGTGGCCAGCACATGAAGAAGATAACCGTGCGCCTTTTTCCGAA But I think it should to be TCCTAGACAAGTTGGCTTTCCCTTAAACAGCTGACATAGAGATATGTCCC Who knows the reason about this. 2011-03-28 lishiyong ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information. attachment: golharam.vcf___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Sort index SAM-files automatically
Jo, Use the SAM Tools SAM-TO-BAM tool in Galaxy to convert your SAM file to a BAM file. Ryan On 3/25/11 10:35 AM, Jochen Seggewiß wrote: Hi! Thank you for your reply. So that means, I should convert the csfasta qual to fastq, map it with Bowtie, get an SAM, convert it to BAM and then index that BAM? How can I index BAM using GALAXY? I haven´t found a way to get a BAM directly from GALAXY using csfasta qual as input files. Best regards Jo Am 3/25/2011 3:13 PM, schrieb Jim Robinson: Hi Jo, To short-circuit confusion I'll jump in here. I'm the developer of IGV and igvtools, the sorting and indexing for SAM files was added long ago, even before indexed BAM files were possible from Java programs. The recommendation now is to convert to BAM and index that, although SAM files still work. If the galaxy community would like the SAM option I'm happy to have igvtools wrapped as a module, and will help with that. BTW, I also have some code (xml) to wrap IGV itself as a Galaxy visualizer, contributed to me by a user. As I don't have a private Galaxy installation I'm unable to test it myself, but can make it available if anyone is interested. Jim Hello! I convert SOLiD csfasta- and qual-files to fastq-files and map those against Hg19 (Bowtie). I would like to use the resulting sam-files in the IGV browser (Broad Institute). Therefore, the sam-file need to be sorted an indexed. This could be done using the “igvtools”. However, it would be nicer if this sorting and indexing could be done automatically using GALAXY. I guess that it is certainly possible – but I do not know how. Could anybody let me know how it works and what function I have to use, respectively? How can I sort the sam-file the correct way? Thank you in advance. Best regards Jo ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- CONFIDENTIALITY NOTICE: This email communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this email in error, please notify the sender immediately by email and permanently delete all copies of this email including all attachments without reading them. If you are the intended recipient, secure the contents in a manner that conforms to all applicable state and/or federal requirements related to privacy and confidentiality of such information. attachment: golharam.vcf___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Convert SOLiD data
Hello collegues, I have two questions which I could not get answered. I have Illumina single end sequences files, and want to use them for ChIP-Seq analysis. My first question is: In Galaxy screencasts (ChIP-Seq analysis for TAF1-protein...) he does not tell how he has generated the txt. format of the file used for demonstration of ChIP-Seq analysis. I would like to know how I can generate that file from my Illumina sequence files to proceed with analysis. My second question is, 2) How can I convert SAM/BAM formated files (generated by Bowtei mapping tool at Galaxy) to Wiggle or Bigwig formats. I would be thankful for the answers and comments. Falak From: galaxy-user-boun...@lists.bx.psu.edu [galaxy-user-boun...@lists.bx.psu.edu] On Behalf Of Jennifer Jackson [j...@bx.psu.edu] Sent: Monday, March 28, 2011 9:57 AM To: lishiyong Cc: galaxy-user Subject: Re: [galaxy-user] Convert SOLiD data Hello Lishiyong, Just to confirm, the conversion was performed at Galaxy main using NGS: QC and manipulation - Convert SOLiD output to fastq? With the option double encode = yes? If so, the output appears to be correct. quote from tool help: Double encode? - converts color calls (0123.) to pseudo-nucleotides (ACGTN). Not necessary for bowtie. Required for BWA. Please let us know if we can help more, Best, Jen Galaxy team On 3/27/11 8:35 PM, lishiyong wrote: Hello! I convert SOLiD csfasta- and qual-files to fastq-files ,I want to use the fastq-files to do denovo with *SOAPdenovo*.because the SOLiD de novo accessory tools Require large memory .But ,I find that there're some question for the converting . for example: T0202322110210103200200203001123212113333311200 —— AGAGTGGCCAGCACATGAAGAAGATAACCGTGCGCCTTTTTCCGAA But I think it should to be TCCTAGACAAGTTGGCTTTCCCTTAAACAGCTGACATAGAGATATGTCCC Who knows the reason about this. 2011-03-28 lishiyong ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] galaxy-user Digest, Vol 57, Issue 26
closed On 3/28/11 7:38 AM, Brian Lam wrote: galaxy-user-requ...@lists.bx.psu.edu wrote: Send galaxy-user mailing list submissions to galaxy-user@lists.bx.psu.edu To subscribe or unsubscribe via the World Wide Web, visit http://lists.bx.psu.edu/listinfo/galaxy-user or, via email, send a message with subject or body 'help' to galaxy-user-requ...@lists.bx.psu.edu You can reach the person managing the list at galaxy-user-ow...@lists.bx.psu.edu When replying, please edit your Subject line so it is more specific than Re: Contents of galaxy-user digest... HEY! This is important! If you reply to a thread in a digest, please 1. Change the subject of your response from Galaxy-user Digest Vol ... to the original subject for the thread. 2. Strip out everything else in the digest that is not part of the thread you are responding to. Why? 1. This will keep the subject meaningful. People will have some idea from the subject line if they should read it or not. 2. Not doing this greatly increases the number of emails that match search queries, but that aren't actually informative. Today's Topics: 1. Convert SOLiD data (lishiyong) 2. Re: biomart plugin (Greg Von Kuster) 3. Re: Convert SOLiD data (Ryan Golhar) 4. Re: Convert SOLiD data (Jennifer Jackson) 5. Re: Sort index SAM-files automatically (Ryan Golhar) -- Message: 1 Date: Mon, 28 Mar 2011 11:35:15 +0800 From: lishiyonglishiy...@genomics.org.cn To: galaxy-usergalaxy-user@lists.bx.psu.edu Subject: [galaxy-user] Convert SOLiD data Message-ID:201103281135102031...@genomics.org.cn Content-Type: text/plain; charset=us-ascii Hello! I convert SOLiD csfasta- and qual-files to fastq-files ,I want to use the fastq-files to do denovo with SOAPdenovo.because the SOLiD de novo accessory tools Require large memory .But ,I find that there're some question for the converting . for example: T0202322110210103200200203001123212113333311200 �� AGAGTGGCCAGCACATGAAGAAGATAACCGTGCGCCTTTTTCCGAA But I think it should to be TCCTAGACAAGTTGGCTTTCCCTTAAACAGCTGACATAGAGATATGTCCC Who knows the reason about this. 2011-03-28 lishiyong -- next part -- An HTML attachment was scrubbed... URL:http://lists.bx.psu.edu/pipermail/galaxy-user/attachments/20110328/43b70c8a/attachment-0001.html -- Message: 2 Date: Mon, 28 Mar 2011 08:57:49 -0400 From: Greg Von Kusterg...@bx.psu.edu To: Andrea Edwardsedwar...@cs.man.ac.uk Cc: galaxy-user@lists.bx.psu.edu Subject: Re: [galaxy-user] biomart plugin Message-ID:a3348bc1-7c00-4b8a-9313-de8444f44...@bx.psu.edu Content-Type: text/plain; charset=us-ascii Hello Andrea, Make a copy of the ~/tools/data_source/biomart.xml for your local biomart install, and change the action in the following tag to point to it. inputs action=http://www.biomart.org/biomart/martview; check_values=false method=get target=_top Then add your new biomart tool wrapper to your tool_conf.xml file and start up your Galaxy instance. For details about adding a new tool, see https://bitbucket.org/galaxy/galaxy-central/wiki/AddToolTutorial. Greg Von Kuster On Mar 26, 2011, at 10:19 AM, Andrea Edwards wrote: Hello I have looked at the biomart plugin for Galaxy and this seems to allow access to marts on the biomart central server. Is there anyway to use this plugin to access a biomart on my server if I can't make my biomart available on the biomart central server. If not, would it be possible to achieve this with galaxy tools. I've heard of galaxy tools but never made one so I don;t know what is involved. Failing that would i be able to use the biomart plugin to access my server if I had a local installation of galaxy thanks ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ Greg Von Kuster Galaxy Development Team g...@bx.psu.edu -- next part -- An HTML attachment was scrubbed... URL:http://lists.bx.psu.edu/pipermail/galaxy-user/attachments/20110328/3ebdda80/attachment-0001.html -- Message: 3 Date: Mon, 28 Mar 2011 09:53:56 -0400 From: Ryan Golhargolha...@umdnj.edu To: lishiyonglishiy...@genomics.org.cn Cc: galaxy-usergalaxy-user@lists.bx.psu.edu Subject: Re: [galaxy-user] Convert SOLiD data Message-ID:4d9092f4.9040...@umdnj.edu Content-Type: text/plain; charset=windows-1252; Format=flowed Lishiyong, You should not convert
Re: [galaxy-user] Problems using aaChanges tool
Hi Evan, The update to include zebrafish (and a few other genomes) should roll out to Galaxy main in the next few days, if not sooner. Feel free to check back Friday for an update if you do not see it there by then. Thank you for your patience! Best, Jen Galaxy team On 3/28/11 8:01 AM, Evan Schwab wrote: Hi Jen, The aaChanges tool is still not supporting the zebrafish Zv9 build. Can you please fix this so that I am able to use the aaChanges tool. It's very important. Thanks Evan -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] server error trying to DL data from galaxy on cloud
Hello, I'm trying to download the library files I processed on my galaxy cloud instance, but I'm getting an error. At the top (on the right panel) it says Server Error and then lists the URL where the data should be and then lists: Module paste.exceptions.errormiddleware:143 in __call__ try: __traceback_supplement__ = Supplement, self, environ app_iter = self.application(environ, start_response) return self.make_catching_iter(app_iter, environ) except: app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ try: status, headers, body = wsgilib.intercept_output( environ, self.app) if status is None: # Some error occurred environ, self.app) Module paste.wsgilib:544 in intercept_output try: for item in app_iter: output.write(item) finally: if hasattr(app_iter, 'close'): output.write(item) MemoryError: out of memory Is there some easy fix to this? I'd really like to get that data off of the cloud instance and be able to terminate it. thanks, karl ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] A genbank to gtf converter
Hi Jen, Many thanks for the reply. Sadly my programming is not up to anything like a gbk to gtf converter! The main reason I want one is that as a virologist this would be very useful since many viruses do not have a gtf file but do have genbank submissions. I know of a site that has some viruses listed together with GFF files but alas I cannot find a GFF to GTF converter - nightmare!! I'll keep looking for one and if I find it I'll let you know. Cheers David On 23 Mar 2011, at 18:02, Jennifer Jackson wrote: Hello David, This is a great idea that the team has been considering adding, but nothing immediate is planned. There are some external teams that are working on outside development, and this is on their list, to. If interested in what that project is doing, please see this thread: http://lists.bx.psu.edu/pipermail/galaxy-dev/2011-March/004692.html For now, if the data resides in a track at UCSC (many are, especially for vertebrate genomes and it is updated daily), using the Table browser can allow you to export the data in GTF and push to Galaxy with the Get Data tool. Since some of the data can be large, using BX Main (our local UCSC mirror) may be the best source. To do this, navigate to the target genome and track (RefSeq under Gene Predictions, others under Mrna EST), and choose output format GTF - gene transfer format. Please note that the gene_id attribute in the 9th field will not be populated with the gene name (will be same as transcript_id). This is just how UCSC does it right now (on their list to get the full GTF output set up in the TB, as far as we know). But, to get that info now, go back in and reexport the same table data again as all fields from selected table into Galaxy and the gene name will be in the data field named name2. The text manipulation tools can help to format the data. A workflow would be a good option once you have the tool path worked out, so that it can be reused without having to do it all again, for future similar genbank datasets. You may even want to publish the workflow for others to use, as it is very popular request, maybe add published page to explain how to use/prep data for input. Apologies for the current inconvenience, but hopefully this can get you going until a more direct method is implemented directly in Galaxy main. Great idea that many other users are also very interested in. Any contributions (page, workflow) would be most welcomed. A tool that does the extraction directly from Genbank would also be welcomed in the Tool Shed, if you want to contribute. http://community.g2.bx.psu.edu/ Best, Jen Galaxy team On 3/14/11 1:15 PM, David Matthews wrote: Hi again, Does anyone know of a genbank to gtf converter? I have heard such things exist but never found one... Cheers David ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] server error trying to DL data from galaxy on cloud
Hi Karl, Hmm, not having Galaxy accessible is definitely not a step in the right direction. Being signed into command line is not an issue; something else must have gone wrong. To start, please take a look at the (bottom of) galaxy log file (and email the relevant part if you don't see how to fix it immediately); the file is saved as /mnt/galaxyTools/galaxy-central/paster.log As far as the location of the files in data libraries, they should be stored in the same location as history datasets, namely /mnt/galaxyData/files/000/dataset_ID.dat Because all of the datasets are named simply based on the database ID, it won't necessarily be obvious which file to get without doing some (python) coding or doing some guess work. If you know how large your files is, you can easily narrow your choices down by listing the contents of the given directory and sorting it by size (using command ls -lS), then pulling out the file(s) that you want. If several files are of approx. the same size, open them up and see which one you want. Good luck and let us know if you have any more trouble, Enis On Mon, Mar 28, 2011 at 3:35 PM, karlerh...@berkeley.edu wrote: Hello Enis, Thanks for the quick response and suggestions. I actually did have a job running while I tried to download a file the first time, that's the first time it gave the error message. But the jobs have long since finished and it's still giving the error message. I've been able to edit the universe_wsgi.ini file to debug = False, but now I'm getting an Internal server error when I try to reload the galaxy instance. Should I be signed out at the command-line to reload galaxy from a browser? Forgive my simplicity, I'm really not at all command-line savvy. Also, another extremely basic problem I have is I just don't know where the data library that holds my files is located. Any help would be greatly appreciated. best, karl Hi Karl, As you see from the error message, you seem to be getting this error because the machine is running out of memory. This can in part be caused by a configuration option that might be set in Galaxy's universe_wsgi.ini file (see below). Did you have any jobs running while trying to download the file? Waiting until those finish might free up some memory. A thing to try is to connect to the instance, edit Galaxy's universe_wsgi.ini file to se debug = False, restart Galaxy and try again. Are you familiar with that at all? The basic steps are as follows: [local]$ ssh -i path to your AWS private key file ubuntu@instance public IP [ec2]$ sudo su galaxy [ec2]$ cd /mnt/galaxyTools/galaxy-central [ec2]$ vi universe_wsgi.ini -- edit file (around line 226) to set: debug = False [ec2]$ sh run.sh --stop-daemon [ec2]$ sh run.sh --daemon Yet another option is to connect the instance in the same way, look through the data library on the file system and manually copy the file out of the instance. You can use the following command to copy the file to your local instance: [local]$ scp -i path to your AWS private key file ubuntu@instance public IP:/mnt/galaxyData/files/000/dataset_ID.dat . Let us know if none of this works, Enis On Mon, Mar 28, 2011 at 12:02 PM, karlerh...@berkeley.edu wrote: Hello, I'm trying to download the library files I processed on my galaxy cloud instance, but I'm getting an error. At the top (on the right panel) it says Server Error and then lists the URL where the data should be and then lists: Module paste.exceptions.errormiddleware:143 in __call__ try: __traceback_supplement__ = Supplement, self, environ app_iter = self.application(environ, start_response) return self.make_catching_iter(app_iter, environ) except: app_iter = self.application(environ, start_response) Module paste.debug.prints:98 in __call__ try: status, headers, body = wsgilib.intercept_output( environ, self.app) if status is None: # Some error occurred environ, self.app) Module paste.wsgilib:544 in intercept_output try: for item in app_iter: output.write(item) finally: if hasattr(app_iter, 'close'): output.write(item) MemoryError: out of memory Is there some easy fix to this? I'd really like to get that data off of the cloud instance and be able to terminate it. thanks, karl ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list:
Re: [galaxy-user] Pseudo Autosomal regions in Chrs X and Y
Listed on the hg19 gateway page at the UCSC genome browser. - Original Message - From: David Matthews d.a.matth...@bristol.ac.uk To: Jennifer Jackson j...@bx.psu.edu Cc: galaxy-u...@bx.psu.edu Sent: Monday, March 28, 2011 2:04:02 PM Subject: Re: [galaxy-user] Pseudo Autosomal regions in Chrs X and Y Hi, Again, thanks for the feedback. I made my own female hg19 by deleting chrY from my copy of hg19 so thats OK. It still leaves the problem of how to analyse male transcriptomes since maps to PAR1 and 2 genes get reported as multimap reads which can end up being filtered out depending on how you analyse your transcriptome. If I knew with certainty where PAR1 and 2 are on chrY of hg19 I was planning to replace the nucleotides with N's on chrY so that they would no longer show up as a multimap problem - do you (or anyone else) happen to know the co-ordinates on hg19? Cheers David On 23 Mar 2011, at 14:19, Jennifer Jackson wrote: Hi David, Right now we don't have anything built-in to filter out this type of duplication automatically. As a potential option, did you know that we offer a Canonical Female build for certain genomes? This may help with some of the duplication issues, if the loss of novel Y is OK for your project. Please see: https://bitbucket.org/galaxy/galaxy-central/wiki/GenomeData Thanks for bringing up a good point! Best, Jen On 3/10/11 8:44 AM, David Matthews wrote: Hi All again, A separate point about the analysis of cufflinks data is the subject of the Pseudo Autosomal Regions in X and Y - this will make a mess of gene expression analysis in some cases especially because tophat will assign a read to both places which therefore makes it a multihit read (which you might then filter out) or it may double the true levels of reported expression. Anyone had experience/thoughts on this? Best Wishes, David. __ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 d.a.matth...@bristol.ac.uk mailto:d.a.matth...@bristol.ac.uk ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] server error trying to DL data from galaxy on cloud
You're almost there, the command should be executed from your local machine (home directory is fine) and it should look as follows: scp -i path to keyfile ubuntu@publicDNS:/mnt/galaxyData/files/000/dataset_11.dat . (note the 'ubuntu@' before the public DNS and a trailing dot (.) - the dot means your current directory on your current machine, i.e., your home dir if that's where you are executing the command from) I apologize for the trouble in getting the this data out and hope it does not keep you from using Galaxy Cloud in the future (we're looking into why the browser-based data copy didn't work and should have a fix shortly for the main app). Enis On Mon, Mar 28, 2011 at 6:16 PM, karlerh...@berkeley.edu wrote: Hi Enis I'm getting the following error at the bottom of the galaxy log: RuntimeError: Content returned before start_response called I have no idea how to fix this, but I'm trying to focus now on actually getting the files, as I don't really need this galaxy instance anymore. I have been able to locate where my files are and which are the ones I want. I've tried the following command: scp -i path to keyfile publicDNS:/mnt/galaxyData/files/000/dataset_11.dat But I just get the scp usage statement coming back. Is there something else I'm missing here? I was executing this command in my home directory, do I need to be somewhere else? I feel like I'm so close!!! Thanks so much for your help so far, I'd be lost otherwise. karl Hi Karl, Hmm, not having Galaxy accessible is definitely not a step in the right direction. Being signed into command line is not an issue; something else must have gone wrong. To start, please take a look at the (bottom of) galaxy log file (and email the relevant part if you don't see how to fix it immediately); the file is saved as /mnt/galaxyTools/galaxy-central/paster.log As far as the location of the files in data libraries, they should be stored in the same location as history datasets, namely /mnt/galaxyData/files/000/dataset_ID.dat Because all of the datasets are named simply based on the database ID, it won't necessarily be obvious which file to get without doing some (python) coding or doing some guess work. If you know how large your files is, you can easily narrow your choices down by listing the contents of the given directory and sorting it by size (using command ls -lS), then pulling out the file(s) that you want. If several files are of approx. the same size, open them up and see which one you want. Good luck and let us know if you have any more trouble, Enis On Mon, Mar 28, 2011 at 3:35 PM, karlerh...@berkeley.edu wrote: Hello Enis, Thanks for the quick response and suggestions. I actually did have a job running while I tried to download a file the first time, that's the first time it gave the error message. But the jobs have long since finished and it's still giving the error message. I've been able to edit the universe_wsgi.ini file to debug = False, but now I'm getting an Internal server error when I try to reload the galaxy instance. Should I be signed out at the command-line to reload galaxy from a browser? Forgive my simplicity, I'm really not at all command-line savvy. Also, another extremely basic problem I have is I just don't know where the data library that holds my files is located. Any help would be greatly appreciated. best, karl Hi Karl, As you see from the error message, you seem to be getting this error because the machine is running out of memory. This can in part be caused by a configuration option that might be set in Galaxy's universe_wsgi.ini file (see below). Did you have any jobs running while trying to download the file? Waiting until those finish might free up some memory. A thing to try is to connect to the instance, edit Galaxy's universe_wsgi.ini file to se debug = False, restart Galaxy and try again. Are you familiar with that at all? The basic steps are as follows: [local]$ ssh -i path to your AWS private key file ubuntu@instance public IP [ec2]$ sudo su galaxy [ec2]$ cd /mnt/galaxyTools/galaxy-central [ec2]$ vi universe_wsgi.ini -- edit file (around line 226) to set: debug = False [ec2]$ sh run.sh --stop-daemon [ec2]$ sh run.sh --daemon Yet another option is to connect the instance in the same way, look through the data library on the file system and manually copy the file out of the instance. You can use the following command to copy the file to your local instance: [local]$ scp -i path to your AWS private key file ubuntu@instance public IP:/mnt/galaxyData/files/000/dataset_ID.dat . Let us know if none of this works, Enis On Mon, Mar 28, 2011 at 12:02 PM, karlerh...@berkeley.edu wrote: Hello, I'm trying to download the library files I
Re: [galaxy-user] Pseudo Autosomal regions in Chrs X and Y
Hi David, The PAR regions are documented at UCSC on the hg19 genome gateway page (and for some other recent genomes). Start at the main page, click into Genomes, select hg19, then scroll down to credits: http://genome.ucsc.edu/ quote: The Y chromosome in this assembly contains two pseudoautosomal regions (PARs) that were taken from the corresponding regions in the X chromosome and are exact duplicates: chrY:10001-2649520 and chrY:59034050-59363566 chrX:60001-2699520 and chrX:154931044-155260560 Hopefully this helps! Jen On 3/28/11 2:04 PM, David Matthews wrote: Hi, Again, thanks for the feedback. I made my own female hg19 by deleting chrY from my copy of hg19 so thats OK. It still leaves the problem of how to analyse male transcriptomes since maps to PAR1 and 2 genes get reported as multimap reads which can end up being filtered out depending on how you analyse your transcriptome. If I knew with certainty where PAR1 and 2 are on chrY of hg19 I was planning to replace the nucleotides with N's on chrY so that they would no longer show up as a multimap problem - do you (or anyone else) happen to know the co-ordinates on hg19? Cheers David On 23 Mar 2011, at 14:19, Jennifer Jackson wrote: Hi David, Right now we don't have anything built-in to filter out this type of duplication automatically. As a potential option, did you know that we offer a Canonical Female build for certain genomes? This may help with some of the duplication issues, if the loss of novel Y is OK for your project. Please see: https://bitbucket.org/galaxy/galaxy-central/wiki/GenomeData Thanks for bringing up a good point! Best, Jen On 3/10/11 8:44 AM, David Matthews wrote: Hi All again, A separate point about the analysis of cufflinks data is the subject of the Pseudo Autosomal Regions in X and Y - this will make a mess of gene expression analysis in some cases especially because tophat will assign a read to both places which therefore makes it a multihit read (which you might then filter out) or it may double the true levels of reported expression. Anyone had experience/thoughts on this? Best Wishes, David. __ Dr David A. Matthews Senior Lecturer in Virology Room E49 Department of Cellular and Molecular Medicine, School of Medical Sciences University Walk, University of Bristol Bristol. BS8 1TD U.K. Tel. +44 117 3312058 Fax. +44 117 3312091 d.a.matth...@bristol.ac.ukmailto:d.a.matth...@bristol.ac.uk ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] server error trying to DL data from galaxy on cloud
Excellent, it was the trailing dot that I was missing! Thanks so much for the help, I will most certainly be using Galaxy again, it's been very useful so far. karl You're almost there, the command should be executed from your local machine (home directory is fine) and it should look as follows: scp -i path to keyfile ubuntu@publicDNS:/mnt/galaxyData/files/000/dataset_11.dat . (note the 'ubuntu@' before the public DNS and a trailing dot (.) - the dot means your current directory on your current machine, i.e., your home dir if that's where you are executing the command from) I apologize for the trouble in getting the this data out and hope it does not keep you from using Galaxy Cloud in the future (we're looking into why the browser-based data copy didn't work and should have a fix shortly for the main app). Enis On Mon, Mar 28, 2011 at 6:16 PM, karlerh...@berkeley.edu wrote: Hi Enis I'm getting the following error at the bottom of the galaxy log: RuntimeError: Content returned before start_response called I have no idea how to fix this, but I'm trying to focus now on actually getting the files, as I don't really need this galaxy instance anymore. I have been able to locate where my files are and which are the ones I want. I've tried the following command: scp -i path to keyfile publicDNS:/mnt/galaxyData/files/000/dataset_11.dat But I just get the scp usage statement coming back. Is there something else I'm missing here? I was executing this command in my home directory, do I need to be somewhere else? I feel like I'm so close!!! Thanks so much for your help so far, I'd be lost otherwise. karl Hi Karl, Hmm, not having Galaxy accessible is definitely not a step in the right direction. Being signed into command line is not an issue; something else must have gone wrong. To start, please take a look at the (bottom of) galaxy log file (and email the relevant part if you don't see how to fix it immediately); the file is saved as /mnt/galaxyTools/galaxy-central/paster.log As far as the location of the files in data libraries, they should be stored in the same location as history datasets, namely /mnt/galaxyData/files/000/dataset_ID.dat Because all of the datasets are named simply based on the database ID, it won't necessarily be obvious which file to get without doing some (python) coding or doing some guess work. If you know how large your files is, you can easily narrow your choices down by listing the contents of the given directory and sorting it by size (using command ls -lS), then pulling out the file(s) that you want. If several files are of approx. the same size, open them up and see which one you want. Good luck and let us know if you have any more trouble, Enis On Mon, Mar 28, 2011 at 3:35 PM, karlerh...@berkeley.edu wrote: Hello Enis, Thanks for the quick response and suggestions. I actually did have a job running while I tried to download a file the first time, that's the first time it gave the error message. But the jobs have long since finished and it's still giving the error message. I've been able to edit the universe_wsgi.ini file to debug = False, but now I'm getting an Internal server error when I try to reload the galaxy instance. Should I be signed out at the command-line to reload galaxy from a browser? Forgive my simplicity, I'm really not at all command-line savvy. Also, another extremely basic problem I have is I just don't know where the data library that holds my files is located. Any help would be greatly appreciated. best, karl Hi Karl, As you see from the error message, you seem to be getting this error because the machine is running out of memory. This can in part be caused by a configuration option that might be set in Galaxy's universe_wsgi.ini file (see below). Did you have any jobs running while trying to download the file? Waiting until those finish might free up some memory. A thing to try is to connect to the instance, edit Galaxy's universe_wsgi.ini file to se debug = False, restart Galaxy and try again. Are you familiar with that at all? The basic steps are as follows: [local]$ ssh -i path to your AWS private key file ubuntu@instance public IP [ec2]$ sudo su galaxy [ec2]$ cd /mnt/galaxyTools/galaxy-central [ec2]$ vi universe_wsgi.ini -- edit file (around line 226) to set: debug = False [ec2]$ sh run.sh --stop-daemon [ec2]$ sh run.sh --daemon Yet another option is to connect the instance in the same way, look through the data library on the file system and manually copy the file out of the instance. You can use the following command to copy the file to your local instance: [local]$ scp -i path to your AWS private key file ubuntu@instance public