Re: [Bioc-devel] Proof-of-concept parallel preloading FastqStreamer

2013-10-02 Thread Martin Morgan
On 10/02/2013 11:58 AM, Gregoire Pau wrote: Hello Ryan, You may be interested in the function sclapply(...) located in the HTSeqGenie package. sclapply is a multicore dispatcher that accepts 3 main arguments (inext, fun, max.parallel.jobs). The data produced by the function inext, executed in th

Re: [Bioc-devel] Proof-of-concept parallel preloading FastqStreamer

2013-10-02 Thread Gregoire Pau
Hello Ryan, You may be interested in the function sclapply(...) located in the HTSeqGenie package. sclapply is a multicore dispatcher that accepts 3 main arguments (inext, fun, max.parallel.jobs). The data produced by the function inext, executed in the main thread, is dispatched to fun(), execute

[Bioc-devel] Proof-of-concept parallel preloading FastqStreamer

2013-09-30 Thread Ryan
Hi all, I have previously written an Rscript to read, filter, and write large fastq files using FastqSteamer to read. Through some complicated tricks, I was able to get the input to happen in parallel with the processing and output (using parallel::mcparallel and friends). In other words, whi