Hi Dave:

Maybe some one knows a better way…

I would probably muck around in the database directly if there were a lot of 
files.

One could probably move them all into a history, groom them, then pull the data 
back into your libraries, deleting the old ones.


Brad
On Apr 25, 2013, at 4:58 AM, Brad Langhorst 
<langho...@neb.com<mailto:langho...@neb.com>> wrote:

Hi dave:

The fastq groomer tool will convert your fastq files (of unknown base quality 
scale) to fastqsanger.  Are you sure these files are not already sanger scaled? 
Modern illumina pipelines produce fastqsanger files.

If you do know the scale, just import the files explictly as fastqsanger (not 
fastq).

Galaxy team - maybe the sniffer should be smarter about guessing the file 
format…

If you need to convert a large number you could set up a galaxy workflow with a 
single fastq groomer step.  That would allow you to start the job on many fastq 
files at once.



Brad
On Apr 25, 2013, at 12:14 AM, Dave Lin 
<d...@verdematics.com<mailto:d...@verdematics.com>> wrote:

Hi Brad, Galaxy Support-

Besides changing them individually, is there a way to modify the settings for a 
large number files from fastq to fastqsanger in batch? Either via the bioblend 
or manually via the UI?

Thanks,
Dave


On Fri, Dec 7, 2012 at 3:53 AM, Langhorst, Brad 
<langho...@neb.com<mailto:langho...@neb.com>> wrote:
You can just change the format of the data from fastq to fastqsanger if you're 
sure about the error format (use the pencil, then datatype tab) note: 
fastqsanger !=fastqcsanger

Brad

On Dec 7, 2012, at 3:20 AM, 泽 蔡 
<caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn>> wrote:

Hi Alex

Now is an another problem. I now deal with two fastq files, there are Illumina 
enconding 1.8 and pair-end, so I don't need to groom. But the fact is, I need 
to use the "filter by quality" and "Fastq interlacer" and without groom thses 
two tools can not regonize the files. Any idea to solve this problem?

发件人: "Bossers, Alex" <alex.boss...@wur.nl<mailto:alex.boss...@wur.nl>>
收件人: 泽 蔡 <caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn>>
抄送: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" 
<galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>>
发送日期: 2012年12月5日, 星期三, 4:02 上午
主题: RE: 回复: [galaxy-dev] 回复: Speed up the galaxy

It seems old Illumina encoding 1.5

So yes some tools requiring the fastqsanger would need grooming.

If you are up to programming you can seriously speed this up by using a 
precalculated transfer or hash table.

That way you do not have to do any calculation but just translate each quality 
line using generic regexp/grep/sed like tools or $seq =~ 
t/STARTSCORES/SANGERSCORES/g in perl.

The wiki table could be an alternative option. You still have to set the upload 
type to fastsanger likely since it will probably sniff the header and see its 
fastq but not 1.8+ encoding.



Good luck!

Alex



________________________________
Van: 泽 蔡 [caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn>]
Verzonden: dinsdag 4 december 2012 16:41
To: Bossers, Alex
Cc: galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>
Onderwerp: 回复: [galaxy-dev] 回复: Speed up the galaxy

Hi Alex

I look the page of wikipedia, but I have a little confused. We sequenced with 
solexa. I paste a read of my data, can you tell me whether I need run Groom?
The read is like this:@HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1
AGAAGTACATCGCGATGCCGTTNCCNNCGAAGGCGATAGNNNACAAGNCCAAATGNTTCTNCATCNNNCNCGAGNNGNCGAGGNCGCCGTGCGACCCTGC
+HWUSI-EAS1734_0003_FC620JEAAXX:8:1:1174:9013#0/1
Ya^a`edddeddc\c`a`dc]\Ba^BBZ]ZZ`ZZZ]a]]BBB^[`\UB_V[V\`ZBSZX^BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB

发件人: "Bossers, Alex" <alex.boss...@wur.nl<mailto:alex.boss...@wur.nl>>
收件人: 泽 蔡 <caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn>>
抄送: "galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>" 
<galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>>; Nate Coraor 
<n...@bx.psu.edu<mailto:n...@bx.psu.edu>>
发送日期: 2012年12月4日, 星期二, 下午 11:21
主题: RE: [galaxy-dev] 回复: Speed up the galaxy

Hi

I presume the best way to optimise your current problem is to evaluate whether 
you really need to groom your data!? If its old data presumably yes, but if it 
is recent data in Illumina 1.8+ 
encoding(http://en.wikipedia.org/wiki/FASTQ_format) it is not necessary.... 
speedup 100% :-)
Groom would take long at our servers as well but due to the new Illumina format 
we didn’t bother to optimise it further by parallelisation for instance...

Alex


Van: 
galaxy-dev-boun...@lists.bx.psu.edu<mailto:galaxy-dev-boun...@lists.bx.psu.edu> 
[mailto:galaxy-dev-boun...@lists.bx.psu.edu<mailto:galaxy-dev-boun...@lists.bx.psu.edu>]
 Namens ? ?
Verzonden: dinsdag 4 december 2012 16:08
Aan: Nate Coraor
CC: galaxy-dev@lists.bx.psu.edu<mailto:galaxy-dev@lists.bx.psu.edu>
Onderwerp: [galaxy-dev] 回复: Speed up the galaxy

Hi,

I read the page of the document. But I don't think there is anything I need. 
Now, I installed a local instance of galaxy, and I just run it on a PC. I need 
to deal with large data, if I just use the default configure of galaxy, every 
task would take long time. For example I ran the FASTQ Groomer with a large 
file, the precedure is so slow, and galaxy would not use the potential of my 
machine. So I want to know how can I get galaxy ran faster. I already can 
upload files quickly and I just need to know how to ran tools quickly.
发件人: Nate Coraor 
<n...@bx.psu.edu<mailto:n...@bx.psu.edu><mailto:n...@bx.psu.edu<mailto:n...@bx.psu.edu>>>
收件人: 泽 蔡 
<caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn><mailto:caizexi...@yahoo.com.cn<mailto:caizexi...@yahoo.com.cn>>>
抄送: 
"galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu><mailto:galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>>"
 
<galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu><mailto:galaxy-...@bx.psu.edu<mailto:galaxy-...@bx.psu.edu>>>
发送日期: 2012年12月4日, 星期二, 下午 9:38
主题: Re: [galaxy-dev] Speed up the galaxy

On Dec 4, 2012, at 4:36 AM, 泽 蔡 wrote:

> Hi all,
>
> How can I speed up the galaxy? Like how to use more cores and memeries.

Hi,

This is a pretty broad question.  However, I would recommend that you start at:

    http://usegalaxy.org/production

--nate

> ___________________________________________________________
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/






___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

--
Brad Langhorst
langho...@neb.com<mailto:langho...@neb.com>






___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


--
Brad Langhorst
langho...@neb.com<mailto:langho...@neb.com>






--
Brad Langhorst
langho...@neb.com<mailto:langho...@neb.com>





___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to