Re: [galaxy-user] wig to bigwig error
Jennifer, Thanks for your help. I ran the filter and sort tool as advised, and then ran the wig to bigwig on the new history item generated by the filter. This time I got a different error: 84: Wig-to-bigWig on data 83 0 bytes An error occurred running this job:stdin is empty of data Error running wigToBigWig. 83: Select on data 49 1 line, 1 comments format: wig, database: mm8 Info: Matching pattern: track Again, I'm sure I left off something obvious. Could you tell me what I did wrong? Thanks, Mike On Apr 13, 2012, at 1:27 PM, Jennifer Jackson wrote: Hi Michael, This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool Filter and Sort - Select by matching the pattern track. Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined. Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised. Hopefully this helps! Best, Jen Galaxy team On 4/13/12 6:19 AM, Michael Sikes wrote: Hi, I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344 The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record: 49: GSM560344_03112009_313D2AAXX_B7.wi https://main.g2.bx.psu.edu/history ~960,000 lines format: wig, database: mm8 Info: uploaded wig file https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/display?to_ext=wig https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/show_paramshttps://main.g2.bx.psu.edu/tool_runner/rerun?id=4746000 https://main.g2.bx.psu.edu/history https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111eitem_class=HistoryDatasetAssociation https://main.g2.bx.psu.edu/dataset/annotate?id=47465bc44dbd111e display at UCSC main https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25sdisplay_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at The details for this upload are as follows: Tool: Upload File Name: GSM560344_03112009_313D2AAXX_B7.wi Created:Dec 08, 2011 Filesize: 12.1 Mb Dbkey: mm8 Format: wig Tool Version: Tool Standard Output: stdout https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stdout Tool Standard Error:stderr https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stderr Input Parameter Value File Format auto Genome Conditional (files_metadata)32 Inheritance Chain GSM560344_03112009_313D2AAXX_B7.wi The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 https://main.g2.bx.psu.edu/history 0 bytes An error occurred running this job:/line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252/ https://main.g2.bx.psu.edu/dataset/errors?id=6041022https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/show_params https://main.g2.bx.psu.edu/tool_runner/rerun?id=6041022 https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/display/?preview=True https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/edithttps://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/delete?show_deleted_on_refresh=False The details for this operation are as follows: Tool: Wig-to-bigWig Name: Wig-to-bigWig on data 49 Created:Mar 21, 2012 Filesize: 0 bytes Dbkey: mm8 Format: bigwig Tool Version: Tool Standard Output: stdout https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stdout Tool Standard Error:stderr https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stderr Input Parameter Value Convert 49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings) 1 Items to bundle in r-tree 256 Data points bundled at lowest level 1024 Clip chromosome positions True Do not use compression True Inheritance Chain Wig-to-bigWig on data 49 I gather that the chromosome ends are not being snipped off, even though I
Re: [galaxy-user] wig to bigwig error
Jen, A couple of uninformed questions. I gather from your response that the author lab submitted a multiple track group .wig file instead of a single track group .wig file, and that I need to generate a single track group file before the bigwig conversion will work. So, with regard to the instructions below, I am to run the text manipulation on the original author submitted .wig file. Then run filter and sort-- Select lines that match an expression on the newly created file that: Matching the pattern: track. This generates yet another file that has the following info: 88: Select on data 87 1 line, 1 comments format: wig, database: mm8 Info: Matching pattern: track track type=wiggle_0 visibility=full name=Smc3_mES autoScale=on color=100,0,100 1 track visibility=dense name=Smc3_mES enriched regions - 1e-09 color=100,0,100 892178 Is the number 892173 the number of track lines? If so, do I then do the Remove beginning of a file using 892178 on the original author .wig file? Mike On Apr 16, 2012, at 10:35 AM, Jennifer Jackson wrote: Hi Mike, I apologize if I wasn't clear, but the 'Select' was to show you how to identify the multi-track group wig files. I wanted to give you a way to screen similar files going forward. The wig-to-bigWig program in Galaxy comes from UCSC. It accepts .wig files with a single track group as input: http://genome.ucsc.edu/goldenPath/help/bigWig.html (see step #1) The data author lab can either submit the data as single track group .wig files, or, if you are confident that the multiple track group .wig format is expected and OK from this source, split the file. There are no specific tools in Galaxy to do this, but something like this would work: - Text Manipulation - Add column, 1, Iterate? = yes - Select, track - note the line number of track lines - Remove beginning of a file, using line numbers, and the - original- .wig file, to break up into individual .wig files. Good luck! Jen Galaxy team On 4/16/12 6:57 AM, Michael Sikes wrote: Jennifer, Thanks for your help. I ran the filter and sort tool as advised, and then ran the wig to bigwig on the new history item generated by the filter. This time I got a different error: 84: Wig-to-bigWig on data 83 https://main.g2.bx.psu.edu/history 0 bytes An error occurred running this job:/stdin is empty of data Error running wigToBigWig. / https://main.g2.bx.psu.edu/dataset/errors?id=6818347https://main.g2.bx.psu.edu/datasets/0f70746579b165e2/show_params https://main.g2.bx.psu.edu/tool_runner/rerun?id=6818347 https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/display/?preview=True https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/edithttps://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/delete?show_deleted_on_refresh=False 83: Select on data 49 https://main.g2.bx.psu.edu/history 1 line, 1 comments format: wig, database: mm8 Info: Matching pattern: track https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/display?to_ext=wig https://main.g2.bx.psu.edu/datasets/b4fb2e8c767b4258/ show_paramshttps://main.g2.bx.psu.edu/tool_runner/rerun? id=6818275https://main.g2.bx.psu.edu/history https://main.g2.bx.psu.edu/tag/retag?item_id=b4fb2e8c767b4258item_class=HistoryDatasetAssociation https://main.g2.bx.psu.edu/dataset/annotate?id=b4fb2e8c767b4258 Again, I'm sure I left off something obvious. Could you tell me what I did wrong? Thanks, Mike On Apr 13, 2012, at 1:27 PM, Jennifer Jackson wrote: Hi Michael, This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool Filter and Sort - Select by matching the pattern track. Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined. Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised. Hopefully this helps! Best, Jen Galaxy team Michael Sikes, Ph.D. Associate Professor of Immunology North Carolina State University Microbiology Department 4524A Gardner Hall Campus Box 7615 Raleigh, NC 27695 Ph: 919-513-0528 Fax: 919-515-7867 email: mlsi...@ncsu.edu ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-user] wig to bigwig error
Hi Michael, This particular .wig file has a data format problem that is the root cause of the conversion error. Specifically, there is an extra track line in the file. This can be found using unix tools with a grep or in Galaxy with the tool Filter and Sort - Select by matching the pattern track. Ideally this would be corrected and resubmitted by the data author before use, since how/why this was inserted and what impact it has would need to be examined. Since you noticed problems with other GEO files (conversion problems), verifying the .wig format and making any necessary corrections would also be advised. Hopefully this helps! Best, Jen Galaxy team On 4/13/12 6:19 AM, Michael Sikes wrote: Hi, I have hit a brick wall when trying to convert wig files from the GEO to bigwig files. Each time I try (and I have tried many times since October), I get the same error. For example, here is a downloaded wig file, that I assigned to the mouse mm8 genome, and the error I got when I tried to convert it to a bigwig file. The dataset came from Bing Ren's lab, and its GEO record is here: http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM560344 The wig file was uploaded to Galaxy Dec. 8, 2011, and I assigned mm8 rather than mm9 based on the GEO record: 49: GSM560344_03112009_313D2AAXX_B7.wi https://main.g2.bx.psu.edu/history ~960,000 lines format: wig, database: mm8 Info: uploaded wig file https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/display?to_ext=wighttps://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/show_paramshttps://main.g2.bx.psu.edu/tool_runner/rerun?id=4746000https://main.g2.bx.psu.edu/history https://main.g2.bx.psu.edu/tag/retag?item_id=47465bc44dbd111eitem_class=HistoryDatasetAssociationhttps://main.g2.bx.psu.edu/dataset/annotate?id=47465bc44dbd111e display at UCSC main https://main.g2.bx.psu.edu/datasets/4746000/display_at/ucsc_main?redirect_url=http%3A%2F%2Fgenome.ucsc.edu%2Fcgi-bin%2FhgTracks%3Fdb%3Dmm8%26position%3Dchr11%3A3000251-3023451%26hgt.customText%3D%25sdisplay_url=https%3A%2F%2Fmain.g2.bx.psu.edu%2Froot%2Fdisplay_as%3Fid%3D4746000%26display_app%3Ducsc%26authz_method%3Ddisplay_at The details for this upload are as follows: Tool: Upload File Name: GSM560344_03112009_313D2AAXX_B7.wi Created:Dec 08, 2011 Filesize: 12.1 Mb Dbkey: mm8 Format: wig Tool Version: Tool Standard Output: stdout https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stdout Tool Standard Error:stderr https://main.g2.bx.psu.edu/datasets/47465bc44dbd111e/stderr Input Parameter Value File Format auto Genome Conditional (files_metadata)32 Inheritance Chain GSM560344_03112009_313D2AAXX_B7.wi The wig-to-bigWig conversion on data 49 (using the wig to bigwig conversion tool in the convert formats toolbox) was run on March 21, 2012 and gave the following error: 77: Wig-to-bigWig on data 49 https://main.g2.bx.psu.edu/history 0 bytes An error occurred running this job:/line 152351 of stdin: chromosome chr13 has 120614378 bases, but item ends at 120614600 line 298005 of stdin: chromosome chr17 has 95177420 bases, but item ends at 95177625 line 325066 of stdin: chromosome chr16 has 98252459 bases, but item ends at 9825252/ https://main.g2.bx.psu.edu/dataset/errors?id=6041022https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/show_paramshttps://main.g2.bx.psu.edu/tool_runner/rerun?id=6041022 https://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/display/?preview=Truehttps://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/edithttps://main.g2.bx.psu.edu/datasets/48bd55b0d18aebad/delete?show_deleted_on_refresh=False The details for this operation are as follows: Tool: Wig-to-bigWig Name: Wig-to-bigWig on data 49 Created:Mar 21, 2012 Filesize: 0 bytes Dbkey: mm8 Format: bigwig Tool Version: Tool Standard Output: stdout https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stdout Tool Standard Error:stderr https://main.g2.bx.psu.edu/datasets/a2b45d5d5a106890/stderr Input Parameter Value Convert 49: GSM560344_03112009_313D2AAXX_B7.wi Conditional (settings) 1 Items to bundle in r-tree 256 Data points bundled at lowest level 1024 Clip chromosome positions True Do not use compression True Inheritance Chain Wig-to-bigWig on data 49 I gather that the chromosome ends are not being snipped off, even though I toggle this option on the galaxy conversion tool. And I know it's doing something, because if I toggle that option off, I get an error that includes broken pipe and simply aborts. I apologize for knowing so little about the bioinformatics involved here. And I'm sure I've overlooked something that is likely obvious to others and/or failed to provide some critical bit of info in this email. But any help would be greatly appreciated. Thanks, Mike ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public
