On Tue, Feb 18, 2014 at 7:30 PM, Damian Plichta <damian.plic...@gmail.com> wrote: > Thanks, that helped a lot. It took me less than 3 hours to perform the > background correction. > > Now I'm wondering if for the next step, quantile normalization, I could do a > similar trick. Is there a way to precompute the target empirical > distribution based on all arrays and then do the normalization on chunks of > data (thus in an independent manner)? I can see the option > targetDistribution under QuantileNormalization.
# Calculate the target distribution based on *all* arrays [not parallalized] qn <- QuantileNormalization(dsC, typesToUpdate="pm") target <- getTargetDistribution(qn, verbose=verbose) # Normalize array by array toward the same target distribution [in chucks] dsCs <- extract(dsC, 1:100) qn <- QuantileNormalization(dsCs, typesToUpdate="pm", targetDistribution=target) csNs <- process(qn, verbose=verbose) Hope this helps /Henrik > > Kind regards, > > Damian Plichta > > On Monday, February 17, 2014 4:03:54 PM UTC-5, Henrik Bengtsson wrote: >> >> Hi. >> >> On Sun, Feb 16, 2014 at 6:53 PM, Damian Plichta >> <damian....@gmail.com> wrote: >> > Hi, >> > >> > I'm processing around 5500 affymetrix exon arrays. The >> > RmaBackgroundCorrection() is pretty slow, 1-2 minutes/array. I played >> > with >> > setOption(aromaSettings, "memory/ram", X) and increased X up to 100 but >> > it >> > didn't have any effect on this stage of analysis. >> >> If you don't notice any difference in processing time by changing >> "memory/ram" from the default (1.0) to 100, then the memory is not >> your bottleneck. >> > >> > Any way to speed the process up? >> >> If you haven't already, make sure to read "How to: Improve processing >> time": >> >> http://aroma-project.org/howtos/ImproveProcessingTime >> >> If you have access to multiple machines on the same file system, you >> can do poor mans parallel processing for the *background correction*, >> because each array is corrected independently of the others. You can >> do this by processing a subset of arrays per computer, e.g. >> >> dsR <- AffymetrixCelSet$byName("MyDataSet", chipType="HuEx-1_0-st-v2") >> dsR <- extract(dsR, 1:100) >> bg <- RmaBackgroundCorrection(dsS) >> dsC <- process(bg, verbose=verbose) >> >> Repeat on another machine with 101:200, and so on. >> >> When all arrays have been background corrected, you can move back to >> your original script - all arrays background corrected are already >> saved to file and will therefore not be redone. >> >> /Henrik >> >> > >> > Kind regards, >> > >> > Damian Plichta >> > >> > -- >> > -- >> > When reporting problems on aroma.affymetrix, make sure 1) to run the >> > latest >> > version of the package, 2) to report the output of sessionInfo() and >> > traceback(), and 3) to post a complete code example. >> > >> > >> > You received this message because you are subscribed to the Google >> > Groups >> > "aroma.affymetrix" group with website http://www.aroma-project.org/. >> > To post to this group, send email to aroma-af...@googlegroups.com >> > To unsubscribe and other options, go to >> > http://www.aroma-project.org/forum/ >> > >> > --- >> > You received this message because you are subscribed to the Google >> > Groups >> > "aroma.affymetrix" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an >> > email to aroma-affymetr...@googlegroups.com. >> > For more options, visit https://groups.google.com/groups/opt_out. > > -- > -- > When reporting problems on aroma.affymetrix, make sure 1) to run the latest > version of the package, 2) to report the output of sessionInfo() and > traceback(), and 3) to post a complete code example. > > > You received this message because you are subscribed to the Google Groups > "aroma.affymetrix" group with website http://www.aroma-project.org/. > To post to this group, send email to aroma-affymetrix@googlegroups.com > To unsubscribe and other options, go to http://www.aroma-project.org/forum/ > > --- > You received this message because you are subscribed to the Google Groups > "aroma.affymetrix" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to aroma-affymetrix+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. -- -- When reporting problems on aroma.affymetrix, make sure 1) to run the latest version of the package, 2) to report the output of sessionInfo() and traceback(), and 3) to post a complete code example. You received this message because you are subscribed to the Google Groups "aroma.affymetrix" group with website http://www.aroma-project.org/. To post to this group, send email to aroma-affymetrix@googlegroups.com To unsubscribe and other options, go to http://www.aroma-project.org/forum/ --- You received this message because you are subscribed to the Google Groups "aroma.affymetrix" group. To unsubscribe from this group and stop receiving emails from it, send an email to aroma-affymetrix+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
