Re: [galaxy-user] Problem with Depth of Coverage on BAM files (GATK tools)
May I join to the question of Carlos? what is exactly hg_g1k_v37? and how can I get the intervals of specific genes in this format? Thanks, Lilach 2012/6/27 Lilach Friedman lilac...@gmail.com Hi Jennifer, Is there a way to directly upload my files from the public Galaxy to my cloud Galaxy instance (in AWS)? Or should I download them first to my computer, and then to upload them? (It takes a lot of time because of the low uploading speed). Thanks, Lilach 2012/6/26 Jennifer Jackson j...@bx.psu.edu Hello Lilach, Currently, the human reference genome indexed for the GATK-beta tools is 'hg_g1k_v37'. The GATK-beta tools are under active revision by our team, so we expect there to be little to no change to the beta version on the main public instance until this is completed. Attempting to convert data between different builds is not recommended. These tools are very sensitive to exact inputs, which extends to naming conventions, etc. The best practice path is to start and continue an analysis project with the same exact genome build throughout. If you want to use the hg19 indexes provided by the GATK project, a cloud instance is the current option (using a hg19 genome as a 'custom genome' will exceed the processing limits available on the public Galaxy instance). Following the links on the GATK tools can provide more information about sources, including links on the GATK web site which will note the exact contents of the both of these genome versions, downloads, and other resources. Hopefully this helps to clear up any confusion, Best, Jen Galaxy team On 6/21/12 7:50 AM, Lilach Friedman wrote: Hi Jennifer, Thank you for this reply. I made a new BWA file, this time using the hg19(full) genome. However, when I am trying to use DepthOfCoverage, the reference genomr is stucked on the hg_g1k_v37 (this is the only option to select), and I cannot change it to hg19(full). Most probably, because I selected hg_g1k_v37 in the previous time I tried to use DepthOfCoverage. It seems as a bug? How can I change it? Thanks, Lilach 2012/6/18 Jennifer Jackson j...@bx.psu.edu Hi Lilach, The problem with this analysis probably has to do with a mismatch between the genomes: the intervals obtained from UCSC (hg19) and the BAM from your BWA (hg_g1k_v37) run. UCSC does not contain the genome 'hg_g1k_v37' - the genome available from UCSC is 'hg19'. Even though these are technically the same human release, on a practical level, they have a different arrangement for some of the chromosomes. You can compare NBCI GRCh37http://www.ncbi.nlm.nih.gov/genome/assembly/2758/ with UCSC hg19 http://genome.ucsc.edu for an explanation. Reference genomes must be *exact* in order to be used with tools - base for base. When they are exact, the identifier will be exact between Galaxy and the source (UCSC, Ensembl) or the full Build name will provide enough information to make a connection to NCBI or other. Sometimes genomes are similar enough that a dataset sourced from one can be used with another, if the database attribute is changed and the data from the regions that differ is removed. This may be possible in your case, only trying will let you know how difficult it actually is with your analysis. The GATK pipeline is very sensitive to exact inputs. You will need to be careful with genome database assignments, etc. Following the links on the tool forms to the GATK help pages can provide some more detail about expected inputs, if this is something that you are going to try. Good luck with the re-run! Jen Galaxy team On 6/18/12 4:42 AM, Lilach Friedman wrote: Hi, I am trying to used Depth of Coverage to see the coverages is specific intervals. The intervals were taken from UCSC (exons of 2 genes), loaded to Galaxy and the file type was changed to intervals. I gave to Depth of Coverage two BAM files (resulted from BWA, selection of only raws with the Matching pattern: XT:A:U, and then SAM-to-BAM) and the intervals file (in advanced GATK options). The consensus genome is hg_g1k_v37. I got the following error message: An error occurred running this job: *Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/space/g2main # ERROR -- # ERROR A USER ERROR has occurred (version 1.4-18-g80a4ce0): # ERROR The invalid argume *Is it a bug, or did I do anything wrong? I will be grateful for any help. Thanks! Lilach* * ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy
Re: [galaxy-user] Problem with Depth of Coverage on BAM files (GATK tools)
Hi Carlos, Thank you very much for this explanation. The format of my intervals file is: chr133289059732890664NM_59_cds_1_0_chr13_32890598_f0+chr1332893213 32893462NM_59_cds_2_0_chr13_32893214_f0+chr133289921232899321 NM_59_cds_3_0_chr13_32899213_f0+chr133290023732900287 NM_59_cds_4_0_chr13_32900238_f0+etc... Can you please explain me how to change this format so I will be able to give it as an input to DepthOfCoverage Thanks, Lilach 2012/6/21 Carlos Borroto carlos.borr...@gmail.com On Thu, Jun 21, 2012 at 10:50 AM, Lilach Friedman lilac...@gmail.com wrote: Hi Jennifer, Thank you for this reply. I made a new BWA file, this time using the hg19(full) genome. However, when I am trying to use DepthOfCoverage, the reference genomr is stucked on the hg_g1k_v37 (this is the only option to select), and I cannot change it to hg19(full). Most probably, because I selected hg_g1k_v37 in the previous time I tried to use DepthOfCoverage. It seems as a bug? How can I change it? Hi Lilach, I have been dealing with these issues for some time now. The only genome you can use with Picard and GATK tools in Galaxy is hg_g1k_v37. I think this is why. From GATK Wiki[1]: If you are using human data, your reads must be aligned to one of the official b3x (e.g. b36, b37) or hg1x (e.g. hg18, hg19) references. The contig ordering in the reference you used must exactly match that of one of the official references canonical orderings. These are defined by historical karotyping of largest to smallest chromosomes, followed by the X, Y, and MT. The order is thus 1, 2, 3, ..., 10, 11, 12, ... 20, 21, 22, X, Y, MT. The GATK will detect misordered contigs (for example, lexicographically sorted) and throw an error. This draconian approach, though unnecessary technically, ensures that all supplementary data provided with the GATK works correctly. You can use ReorderSam to fix a BAM file aligned to a missorted reference sequence. [1] http://www.broadinstitute.org/gsa/wiki/index.php/Input_files_for_the_GATK So far what I have done when presented with a BAM file produced with reference with lexicographical chromosomes ordering, is to use Picard's ReorderSam tool, also in Galaxy, selecting hg_g1k_v37 as reference. You might not be able to this, as if a recall correctly hg19 also use chr1, chr2... instead of 1, 2, ... In that case more work needs to be done and at that point is almost easier to just remap with the correct reference for use with GATK. In your case it seems you already have it. What you might need to do is resort your intervals file and probably change the chromosomes identifiers, this I think can be done inside Galaxy. I would love to hear comments about this approach, as sometime I do worry like Hiram's comment hints to, that hg19 and hg_g1k_v37 might not be completely identical beside the chromosome ordering. In that case my resorted BAM or intervals files might be incorrect. Hope it helps, Carlos Thanks, Lilach 2012/6/18 Jennifer Jackson j...@bx.psu.edu Hi Lilach, The problem with this analysis probably has to do with a mismatch between the genomes: the intervals obtained from UCSC (hg19) and the BAM from your BWA (hg_g1k_v37) run. UCSC does not contain the genome 'hg_g1k_v37' - the genome available from UCSC is 'hg19'. Even though these are technically the same human release, on a practical level, they have a different arrangement for some of the chromosomes. You can compare NBCI GRCh37 with UCSC hg19 for an explanation. Reference genomes must be exact in order to be used with tools - base for base. When they are exact, the identifier will be exact between Galaxy and the source (UCSC, Ensembl) or the full Build name will provide enough information to make a connection to NCBI or other. Sometimes genomes are similar enough that a dataset sourced from one can be used with another, if the database attribute is changed and the data from the regions that differ is removed. This may be possible in your case, only trying will let you know how difficult it actually is with your analysis. The GATK pipeline is very sensitive to exact inputs. You will need to be careful with genome database assignments, etc. Following the links on the tool forms to the GATK help pages can provide some more detail about expected inputs, if this is something that you are going to try. Good luck with the re-run! Jen Galaxy team On 6/18/12 4:42 AM, Lilach Friedman wrote: Hi, I am trying to used Depth of Coverage to see the coverages is specific intervals. The intervals were taken from UCSC (exons of 2 genes), loaded to Galaxy and the file type was changed to intervals. I gave to Depth of Coverage two BAM files (resulted from BWA, selection of only raws with the Matching pattern: XT:A:U, and then SAM-to-BAM) and the intervals file (in advanced GATK options
Re: [galaxy-user] Problem with Depth of Coverage on BAM files (GATK tools)
Hi Jennifer, Thank you for this reply. I made a new BWA file, this time using the hg19(full) genome. However, when I am trying to use DepthOfCoverage, the reference genomr is stucked on the hg_g1k_v37 (this is the only option to select), and I cannot change it to hg19(full). Most probably, because I selected hg_g1k_v37 in the previous time I tried to use DepthOfCoverage. It seems as a bug? How can I change it? Thanks, Lilach 2012/6/18 Jennifer Jackson j...@bx.psu.edu Hi Lilach, The problem with this analysis probably has to do with a mismatch between the genomes: the intervals obtained from UCSC (hg19) and the BAM from your BWA (hg_g1k_v37) run. UCSC does not contain the genome 'hg_g1k_v37' - the genome available from UCSC is 'hg19'. Even though these are technically the same human release, on a practical level, they have a different arrangement for some of the chromosomes. You can compare NBCI GRCh37http://www.ncbi.nlm.nih.gov/genome/assembly/2758/ with UCSC hg19 http://genome.ucsc.edu for an explanation. Reference genomes must be *exact* in order to be used with tools - base for base. When they are exact, the identifier will be exact between Galaxy and the source (UCSC, Ensembl) or the full Build name will provide enough information to make a connection to NCBI or other. Sometimes genomes are similar enough that a dataset sourced from one can be used with another, if the database attribute is changed and the data from the regions that differ is removed. This may be possible in your case, only trying will let you know how difficult it actually is with your analysis. The GATK pipeline is very sensitive to exact inputs. You will need to be careful with genome database assignments, etc. Following the links on the tool forms to the GATK help pages can provide some more detail about expected inputs, if this is something that you are going to try. Good luck with the re-run! Jen Galaxy team On 6/18/12 4:42 AM, Lilach Friedman wrote: Hi, I am trying to used Depth of Coverage to see the coverages is specific intervals. The intervals were taken from UCSC (exons of 2 genes), loaded to Galaxy and the file type was changed to intervals. I gave to Depth of Coverage two BAM files (resulted from BWA, selection of only raws with the Matching pattern: XT:A:U, and then SAM-to-BAM) and the intervals file (in advanced GATK options). The consensus genome is hg_g1k_v37. I got the following error message: An error occurred running this job: *Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/space/g2main # ERROR -- # ERROR A USER ERROR has occurred (version 1.4-18-g80a4ce0): # ERROR The invalid argume *Is it a bug, or did I do anything wrong? I will be grateful for any help. Thanks! Lilach* * ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jacksonhttp://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] generate pileup not working?
Hi Jen, Thank you! It worked on the original BWA, only when it is aligned to hg19 (failed when the alignment was to hg_g1k_v37. However, it doen't work on BAM or SAM files after sorting with Select Matching pattern: XT:A:U or after filtering the BWA results with Filter SAM on bitwise flag values. Is there any solution? Thanks, Lilach 2012/6/18 Jennifer Jackson j...@bx.psu.edu Hello Lilach, Please try running pileup on the original BWA output - (SAM is an OK input with this tool) and let us know if you continue to have problems. Hopefully this helps, Jen Galaxy team On 6/14/12 1:36 PM, Lilach Friedman wrote: Hi, I am trying to do variants call with generate pileup. My steps where: 1. BWA 2. select only lines with the pattern Matching pattern: XT:A:U 3. SAM-to-BAM 4. then I tried to use Generate pileup from BAM dataset However, it does not work, and I get the error message: 114: Generate pileup on data 97: converted pileup 0 bytes An error occurred running this job: Samtools Version: 0.1.16 (r963:234) Error running Samtools pileup tool Floating point exception Did I do anything wrong, or is it a bug? The parameters are copied below. Thanks, Lilach The parameters are: Tool: Generate pileup Name: Generate pileup on data 97: converted pileup Created: Jun 14, 2012 Filesize: 0 bytes Dbkey: hg_g1k_v37 Format: tabular Tool Version: Tool Standard Output: stdout Tool Standard Error: stderr Input Parameter Value Conditional (refOrHistory) 0 Select the BAM file to generate the pileup file for 97: SAM-to-BAM on data 94: converted BAM Whether or not to print the mapping quality as the last column Do not print the mapping quality as the last column Whether or not to print only output pileup lines containing indels Print all lines Where to cap mapping quality 60 Conditional (c) 1 Theta parameter (error dependency coefficient) in the MAQ consensus calling model 0.85 Number of haplotypes in the sample 2 Expected fraction of differences between a pair of haplotypes 0.001 Phred probability of an indel in sequencing/prep 40 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://galaxyproject.org ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] Problem with Depth of Coverage on BAM files (GATK tools)
Hi, I am trying to used Depth of Coverage to see the coverages is specific intervals. The intervals were taken from UCSC (exons of 2 genes), loaded to Galaxy and the file type was changed to intervals. I gave to Depth of Coverage two BAM files (resulted from BWA, selection of only raws with the Matching pattern: XT:A:U, and then SAM-to-BAM) and the intervals file (in advanced GATK options). The consensus genome is hg_g1k_v37. I got the following error message: An error occurred running this job: *Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/space/g2main # ERROR -- # ERROR A USER ERROR has occurred (version 1.4-18-g80a4ce0): # ERROR The invalid argume *Is it a bug, or did I do anything wrong? I will be grateful for any help. Thanks! Lilach* * ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] generate pileup not working?
Hi, I am trying to do variants call with generate pileup. My steps where: 1. BWA 2. select only lines with the pattern Matching pattern: XT:A:U 3. SAM-to-BAM 4. then I tried to use Generate pileuphttps://main.g2.bx.psu.edu/tool_runner?tool_id=sam_pileupfrom BAM dataset However, it does not work, and I get the error message: 114: Generate pileup on data 97: converted pileup 0 bytes An error occurred running this job: *Samtools Version: 0.1.16 (r963:234) Error running Samtools pileup tool Floating point exception* Did I do anything wrong, or is it a bug? The parameters are copied below. Thanks, Lilach The parameters are: Tool: Generate pileup Name:Generate pileup on data 97: converted pileup Created:Jun 14, 2012 Filesize:0 bytes Dbkey:hg_g1k_v37 Format:tabular Tool Version: Tool Standard Output:stdouthttps://main.g2.bx.psu.edu/datasets/d038161e3fe9072e/stdout Tool Standard Error:stderrhttps://main.g2.bx.psu.edu/datasets/d038161e3fe9072e/stderr Input Parameter Value Conditional (refOrHistory) 0 Select the BAM file to generate the pileup file for 97: SAM-to-BAM on data 94: converted BAM Whether or not to print the mapping quality as the last column Do not print the mapping quality as the last column Whether or not to print only output pileup lines containing indels Print all lines Where to cap mapping quality 60 Conditional (c) 1 Theta parameter (error dependency coefficient) in the MAQ consensus calling model 0.85 Number of haplotypes in the sample 2 Expected fraction of differences between a pair of haplotypes 0.001 Phred probability of an indel in sequencing/prep 40 ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] Galaxy in AWS AMI
and I terminated a created from the beginning new instanced for the last several hours (at least 5 different instances). All don't work. Lilach 2012/6/14 Lilach Friedman lilac...@gmail.com Hi, Can somebody please help me? I'm trying for hours to connect to Galaxy on Amazon EC2. I did everything according the instructions, but did not succeed. The Access Galaxy button in the instance remains grey out. I copied the screen print and the log to the attached file. Can anyone help me please? and another questions: I want to use it to analyze MiSeq results (to align to the human genome). 100 Gb EBS and a large machine are enough? too much? Many thanks, Lilach. ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
[galaxy-user] USing GATK DepthOfCoverage with all exon enriched data
Hi, I have NGS results of DNA enriched for exons with an AllExon kit (Agilent). I have a bed file with the list of targeted sequences. I want to use GATK DepthOfCoverage to compare the results to the bed file and to get all the targets that were covered by n reads. How can I do that with Galaxy on the web? or in Amazon? My question has 2 parts: 1. How can I specify the target intervals in Galaxy on the web? (the -L command in Unix commandline) 2. How can I ask the coverage for single bases instead of statistics? Thanks, Lilach ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
