Hi, thanks for spotting/reporting on this. In the definition of the local extractSignals() function, the line:
tcn$y <- 2*tcn$y; has to be replaced by: setSignals(tcn, 2*getSignals(tcn)); I've updated the online vignette [http://aroma-project.org/vignettes/CalMaTe] accordingly and verified that it now works. Sorry about the troubles /Henrik On Fri, Jun 15, 2012 at 8:20 AM, MH <[email protected]> wrote: > That worked and I got up and running again. However, I just hit another > error at extract signal. > I have included the command before it also. > >> print(dsCNList); > $total > AromaUnitTotalCnBinarySet: > Name: CLLtest > Tags: ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences > Full name: CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences > > Number of files: 6 > Names: 1_S7_CD3_SNP6, 2_S7_CD15_SNP6, 3_S7_CD19_SNP6, ..., 6_S29_CD19_SNP6 > [6] > Path (to the first file): > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences/GenomeWideSNP_6 > > Total file size: 43.06 MB > RAM: 0.01MB > > $fracB > AromaUnitFracBCnBinarySet: > Name: CLLtest > Tags: ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences > Full name: CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences > > Number of files: 6 > Names: 1_S7_CD3_SNP6, 2_S7_CD15_SNP6, 3_S7_CD19_SNP6, ..., 6_S29_CD19_SNP6 > [6] > Path (to the first file): > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY,CMTN,v2,normalReferences/GenomeWideSNP_6 > > Total file size: 43.06 MB > RAM: 0.00MB > >> extractSignals <- function(dsList, sampleName, reference=c("none", >> "median"), refIdxs=NULL, ..., verbose=FALSE) { > + reference <- match.arg(reference); > + idx <- indexOf(dsList$total, sampleName); > + dfT <- getFile(dsList$total, idx); > + dfB <- getFile(dsList$fracB, idx); > + tcn <- extractRawCopyNumbers(dfT, logBase=NULL, ..., > verbose=verbose); > + baf <- extractRawAlleleBFractions(dfB, ..., verbose=verbose); > + if (reference == "median") { > + if (!is.null(refIdxs)) { > + dsR <- extract(dsList$total, refIdxs); > + } else { > + dsR <- dsList$total; > + } > + dfTR <- getAverageFile(dsR, verbose=verbose); > + tcnR <- extractRawCopyNumbers(dfTR, logBase=NULL, ..., > verbose=verbose); > + tcn <- divideBy(tcn, tcnR); > + tcn$y <- 2*tcn$y; > + } > + list(tcn=tcn, baf=baf); > + } >> pch <- 19; >> cex <- 0.8; >> >> snT <- sampleName; >> chr <- 8; >> >> for (normalRefs in c(TRUE, FALSE)) { > + if (normalRefs) { > + figName <- sprintf("%s,Chr%02d,CalMaTe,normalReferences", snT, chr); > + dataT <- extractSignals(dsList, sampleName=snT, chromosome=chr, > reference="median", refIdxs=idxsN, verbose=verbose); > + dataTC <- extractSignals(dsCNList, sampleName=snT, chromosome=chr, > verbose=verbose); > + } else { > + figName <- sprintf("%s,Chr%02d,CalMaTe", snT, chr); > + dataT <- extractSignals(dsList, sampleName=snT, chromosome=chr, > reference="median", verbose=verbose); > + dataTC <- extractSignals(dsCList, sampleName=snT, chromosome=chr, > verbose=verbose); > + } > + > + > + toPNG(figName, width=1200, { # (save to figures/) > + subplots(4, ncol=1); > + par(mar=c(2,5,1,1)+0.1, cex=cex, cex.lab=2.4, cex.axis=2.2); > + > + plot(dataT$tcn, ylim=c(0,4), pch=pch); > + plot(dataT$baf, pch=pch); > + plot(dataTC$tcn, ylim=c(0,4), pch=pch); > + plot(dataTC$baf, pch=pch); > + }); > + } > Extracting RawCopyNumbers object... > Name: 2_S7_CD15_SNP6,total > Chromosome: 8 > Identifying units on chromosome... > AromaUgpFile: > Name: GenomeWideSNP_6 > Tags: Full,na31,hg19,HB20110328 > Full name: GenomeWideSNP_6,Full,na31,hg19,HB20110328 > Pathname: > annotationData/chipTypes/GenomeWideSNP_6/GenomeWideSNP_6,Full,na31,hg19,HB20110328.ugp > File size: 8.97 MB (9407867 bytes) > RAM: 0.00 MB > Number of data rows: 1881415 > File format: v1 > Dimensions: 1881415x2 > Column classes: integer, integer > Number of bytes per column: 1, 4 > Footer: <createdOn>20110328 19:16:46 > PDT</createdOn><platform>Affymetrix</platform><chipType>GenomeWideSNP_6,Full</chipType><createdBy><fullname>Henrik > Bengtsson</fullname><email>[email protected]</email></createdBy><srcFiles><srcFile1><filename>GenomeWideSNP_6,Full.CDF</filename><filesize>493291745</filesize><checksum>3fbe0f6e7c8a346105238a3f3d10d4ec</checksum></srcFile1><srcFile2><filename>GenomeWideSNP_6.na31.annot.csv</filename><filesize>1501563495</filesize><checksum>3f59efe386ffc1af656d01be0ab035e9</checksum></srcFile2><srcFile3><filename>GenomeWideSNP_6.cn.na31.annot.csv</filename><filesize>879257480</filesize><checksum>6a21fef62c71fb56bc5dda4499b70a7f</checksum></srcFile3></srcFiles> > Chip type: GenomeWideSNP_6,Full > Platform: Affymetrix > Units: > int [1:98308] 55161 55162 55163 55164 55165 55166 55167 55168 55169 55170 > ... > Genomic positions: > int [1:98308] 190568 237498 239189 239324 281424 281572 282252 282384 > 415401 446359 ... > Identifying units on chromosome...done > Extracting data... > num [1:98308] 2748 5059 5300 2567 1878 ... > Extracting data...done > Extracting RawCopyNumbers object...done > Extracting RawAlleleBFractions object... > Name: 2_S7_CD15_SNP6,fracB > Chromosome: 8 > Identifying units on chromosome... > AromaUgpFile: > Name: GenomeWideSNP_6 > Tags: Full,na31,hg19,HB20110328 > Full name: GenomeWideSNP_6,Full,na31,hg19,HB20110328 > Pathname: > annotationData/chipTypes/GenomeWideSNP_6/GenomeWideSNP_6,Full,na31,hg19,HB20110328.ugp > File size: 8.97 MB (9407867 bytes) > RAM: 0.00 MB > Number of data rows: 1881415 > File format: v1 > Dimensions: 1881415x2 > Column classes: integer, integer > Number of bytes per column: 1, 4 > Footer: <createdOn>20110328 19:16:46 > PDT</createdOn><platform>Affymetrix</platform><chipType>GenomeWideSNP_6,Full</chipType><createdBy><fullname>Henrik > Bengtsson</fullname><email>[email protected]</email></createdBy><srcFiles><srcFile1><filename>GenomeWideSNP_6,Full.CDF</filename><filesize>493291745</filesize><checksum>3fbe0f6e7c8a346105238a3f3d10d4ec</checksum></srcFile1><srcFile2><filename>GenomeWideSNP_6.na31.annot.csv</filename><filesize>1501563495</filesize><checksum>3f59efe386ffc1af656d01be0ab035e9</checksum></srcFile2><srcFile3><filename>GenomeWideSNP_6.cn.na31.annot.csv</filename><filesize>879257480</filesize><checksum>6a21fef62c71fb56bc5dda4499b70a7f</checksum></srcFile3></srcFiles> > Chip type: GenomeWideSNP_6,Full > Platform: Affymetrix > Units: > int [1:98308] 55161 55162 55163 55164 55165 55166 55167 55168 55169 55170 > ... > Genomic positions: > int [1:98308] 190568 237498 239189 239324 281424 281572 282252 282384 > 415401 446359 ... > Identifying units on chromosome...done > Extracting data... > num [1:98308] 1 0.226 0.182 0.118 0.571 ... > Extracting data...done > Extracting RawAlleleBFractions object...done > Retrieving average unit signals across 3 arrays... > Searching for an existing file... > Paths: > [1] > "totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6" > Filename: .average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > Searching path #1 of 1... > Path: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6 > Pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > Searching path #1 of 1...done > Located pathname: > Searching for an existing file...done > Allocating empty data file to store average signals... > Path: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6 > Filename: .average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > Adding temporary suffix from file... > Pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > Suffix: .tmp > Rename existing file?: FALSE > Temporary pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb.tmp > Adding temporary suffix from file...done > Dropping temporary suffix from file... > Temporary pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb.tmp > Suffix: .tmp > Regular expression for suffix: \.tmp$ > Pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > Renaming existing file... > Result: TRUE > Renaming existing file...done > Dropping temporary suffix from file...done > Allocating empty data file to store average signals...done > AromaUnitTotalCnBinaryFile: > Name: .average-signals-median-mad > Tags: 65e3f541c6c39dfbe63b1428b7c795f5 > Full name: .average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5 > Pathname: > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6/.average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5.asb > File size: 7.18 MB (7525849 bytes) > RAM: 0.00 MB > Number of data rows: 1881415 > File format: v1 > Dimensions: 1881415x1 > Column classes: double > Number of bytes per column: 4 > Footer: <createdOn>20120615 11:06:00 > EDT</createdOn><platform>Affymetrix</platform><chipType>GenomeWideSNP_6,Full</chipType> > Platform: Affymetrix > Chip type: GenomeWideSNP_6,Full > Number of units to be updated: 1881415 > Processing chunk... > chr "Indices in chunk:" > int [1:1881415] 1 2 3 4 5 6 7 8 9 10 ... > Reading data... > Reading data...done > Estimating averages and standard deviations... > Estimating averages and standard deviations...done > Writing estimates... > Writing estimates...done > Processing chunk...done > Retrieving average unit signals across 3 arrays...done > Extracting RawCopyNumbers object... > Name: .average-signals-median-mad,65e3f541c6c39dfbe63b1428b7c795f5 > Chromosome: 8 > Identifying units on chromosome... > AromaUgpFile: > Name: GenomeWideSNP_6 > Tags: Full,na31,hg19,HB20110328 > Full name: GenomeWideSNP_6,Full,na31,hg19,HB20110328 > Pathname: > annotationData/chipTypes/GenomeWideSNP_6/GenomeWideSNP_6,Full,na31,hg19,HB20110328.ugp > File size: 8.97 MB (9407867 bytes) > RAM: 0.00 MB > Number of data rows: 1881415 > File format: v1 > Dimensions: 1881415x2 > Column classes: integer, integer > Number of bytes per column: 1, 4 > Footer: <createdOn>20110328 19:16:46 > PDT</createdOn><platform>Affymetrix</platform><chipType>GenomeWideSNP_6,Full</chipType><createdBy><fullname>Henrik > Bengtsson</fullname><email>[email protected]</email></createdBy><srcFiles><srcFile1><filename>GenomeWideSNP_6,Full.CDF</filename><filesize>493291745</filesize><checksum>3fbe0f6e7c8a346105238a3f3d10d4ec</checksum></srcFile1><srcFile2><filename>GenomeWideSNP_6.na31.annot.csv</filename><filesize>1501563495</filesize><checksum>3f59efe386ffc1af656d01be0ab035e9</checksum></srcFile2><srcFile3><filename>GenomeWideSNP_6.cn.na31.annot.csv</filename><filesize>879257480</filesize><checksum>6a21fef62c71fb56bc5dda4499b70a7f</checksum></srcFile3></srcFiles> > Chip type: GenomeWideSNP_6,Full > Platform: Affymetrix > Units: > int [1:98308] 55161 55162 55163 55164 55165 55166 55167 55168 55169 55170 > ... > Genomic positions: > int [1:98308] 190568 237498 239189 239324 281424 281572 282252 282384 > 415401 446359 ... > Identifying units on chromosome...done > Extracting data... > num [1:98308] 2924 3757 5788 2567 2177 ... > Extracting data...done > Extracting RawCopyNumbers object...done > Error in value[[3L]](cond) : > [2012-06-15 11:06:24] Exception: Failed assign column 'y', because: > replacement has 0 rows, data has 98308 > > at #09. value[[3L]](cond) > - value[[3L]]() is local of the calling function > > at #08. tryCatchOne(expr, names, parentenv, handlers[[1L]]) > - tryCatchOne() is local of the calling function > > at #07. tryCatchList(expr, classes, parentenv, handlers) > - tryCatchList() is local of the calling function > > at #06. tryCatch({ > this <- NextMethod("[[<-", this) > }, error = function(ex) { > msg <- sprintf("Failed assign column '%s', because: %s", > name, ex$message) > throw(msg) > }) > - tryCatch() is in environment 'base' > > at #05. `[[<-.RichDataFrame`(`*tmp*`, name, value = numeric(0)) > - `[[<-.RichDataFrame`() is in environment 'aroma.core' > > at #04. `[[<-`(`*tmp*`, name, value = numeric(0)) > - `[[<-`() is local of the calling function > > at #03. `$ > > > > > On Thursday, June 14, 2012 4:54:43 PM UTC-4, Henrik Bengtsson wrote: >> >> >> > 1) I have been working from your page. >> > http://aroma-project.org/vignettes/CalMaTe - over the course of the last >> > couple of days. I have been following that carefully.. >> >> > On Thu, Jun 14, 2012 at 12:59 PM, MH <[email protected]> wrote: >> >> More or less the complete script follows.. >> >> >> >> dsList <- doASCRMAv2(csR, plm="RmaCnPlm", verbose=verbose); >> >> print(dsList); >> >> >> >> cmt <- CalMaTeCalibration(dsList); >> >> print(cmt); >> >> >> >> >> >> sampleName <- "2_S7_CD15_SNP6"; >> >> >> >> >> >> patientIDs <- c(1,2,4); >> >> sampleTypes <- c("tumor", "normal"); >> >> >> >> pids <- rep(patientIDs, each=length(sampleTypes)); >> >> types <- rep(sampleTypes, times=length(patientIDs)); >> >> >> >> mat <- cbind(names, pids, types); >> >> str(mat); >> >> >> >> idxsN <- which(mat[, "types"] == "normal"); >> >> >> >> >> >> cmtN <- CalMaTeCalibration(dsList, tags=c("*", "normalReferences"), >> >> references=idxsN); >> >> > >> > 2) > print(cmtN); >> > CalMaTeCalibration: >> > Data sets (2): >> > <Total>: >> > AromaUnitTotalCnBinarySet: >> > Name: CLLtest >> > Tags: ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY >> > Full name: CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY >> > Number of files: 6 >> > Names: 1_S7_CD3_SNP6, 2_S7_CD15_SNP6, 3_S7_CD19_SNP6, ..., >> > 6_S29_CD19_SNP6 >> > [6] >> > Path (to the first file): >> > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6 >> > Total file size: 43.06 MB >> > RAM: 0.01MB >> > <FracB>: >> > AromaUnitFracBCnBinarySet: >> > Name: CLLtest >> > Tags: ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY >> > Full name: CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY >> > Number of files: 6 >> > Names: 1_S7_CD3_SNP6, 2_S7_CD15_SNP6, 3_S7_CD19_SNP6, ..., >> > 6_S29_CD19_SNP6 >> > [6] >> > Path (to the first file): >> > totalAndFracBData/CLLtest,ACC,ra,-XY,BPN,-XY,RMA,FLN,-XY/GenomeWideSNP_6 >> > Total file size: 43.06 MB >> > RAM: 0.00MB >> > Number of arrays: 6 >> > Number of references: 3 (50.00%) >> > Additional parameters: [2] references: int [1:3] 2 4 6, flavor: chr "v2" >> > >> > 3) > sessionInfo() >> > R version 2.14.2 (2012-02-29) >> > Platform: i386-pc-mingw32/i386 (32-bit) >> > >> > locale: >> > [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United >> > States.1252 >> > [3] LC_MONETARY=English_United States.1252 >> > LC_NUMERIC=C >> > [5] LC_TIME=English_United States.1252 >> > >> > attached base packages: >> > [1] stats graphics grDevices utils datasets methods base >> > >> > other attached packages: >> > [1] preprocessCore_1.16.0 BiocInstaller_1.2.1 sfit_0.1.9 >> > calmate_0.9.2 >> > [5] MASS_7.3-17 aroma.affymetrix_2.5.0 affxparser_1.26.4 >> > aroma.apd_0.2.2 >> > [9] R.huge_0.3.0 aroma.core_2.5.0 aroma.light_1.24.0 >> > matrixStats_0.5.0 >> > [13] R.rsp_0.7.5 R.cache_0.6.2 R.filesets_1.1.5 >> > digest_0.5.2 >> > [17] R.utils_1.12.1 R.oo_1.9.3 R.methodsS3_1.2.2 >> > >> > loaded via a namespace (and not attached): >> > [1] splines_2.14.2 tools_2.14.2 >> >> All of the above looks good. FYI, all methods of the Aroma framework >> are designed to use a constant amount of memory regardless of chip >> type and number of samples, so it's a bit weird that you "run out of >> memory" (or more likely, the memory allocated for/by R is so >> fragmented that it cannot allocate a continuous stretch of 95.4 >> Mbytes). Also, the above script is run part of the regular system >> tests and there we don't run out of memory, but I'll check the memory >> usage on my Windows 7 machine when I have time. Regardless, I don't >> think it is possible troubleshoot this further. >> >> The good thing though, you can just restart with a fresh R session, >> rerun your script and things should go well. Since aroma stores >> intermediate and final results to the file system, any steps that have >> already been processed will already be there so it should be really >> quick to rerun it. Let us know if that works. >> >> Best, >> >> Henrik >> >> > >> > thanks, M >> > >> > >> > On Thursday, June 14, 2012 2:31:02 PM UTC-4, Henrik Bengtsson wrote: >> >> >> >> >> >> > Hi All, >> >> > I have my test run almost completed and at the step >> >> > dsCNList <- process(cmtN, verbose=verbose); >> >> > I get the following memory error >> >> > Error: cannot allocate vector of size 95.4 Mb >> >> > >> >> > I am on Windows7, 32bit and have many Gs of storage. >> >> > >> >> > Any ideas for how to deal with this? >> >> >> >> My best guess now is that you have done lots of other things in the >> >> same R session before doing your aroma analysis, but it is impossible >> >> to tell based on that single line of code. So, please post (1) your >> >> complete script (at least up to the point where you get the error), >> >> (2) the output of print(cmtN), (3) the output of sessionInfo() after >> >> you get the error. >> >> >> >> /Henrik >> >> >> >> > thanks, M >> >> > >> >> > -- >> >> > When reporting problems on aroma.affymetrix, make sure 1) to run the >> >> > latest >> >> > version of the package, 2) to report the output of sessionInfo() and >> >> > traceback(), and 3) to post a complete code example. >> >> > >> >> > >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "aroma.affymetrix" group with website http://www.aroma-project.org/. >> >> > To post to this group, send email to >> >> > [email protected] >> >> > To unsubscribe and other options, go to >> >> > http://www.aroma-project.org/forum/ >> > >> > -- >> > When reporting problems on aroma.affymetrix, make sure 1) to run the >> > latest >> > version of the package, 2) to report the output of sessionInfo() and >> > traceback(), and 3) to post a complete code example. >> > >> > >> > You received this message because you are subscribed to the Google >> > Groups >> > "aroma.affymetrix" group with website http://www.aroma-project.org/. >> > To post to this group, send email to [email protected] >> > To unsubscribe and other options, go to >> > http://www.aroma-project.org/forum/ > > -- > When reporting problems on aroma.affymetrix, make sure 1) to run the latest > version of the package, 2) to report the output of sessionInfo() and > traceback(), and 3) to post a complete code example. > > > You received this message because you are subscribed to the Google Groups > "aroma.affymetrix" group with website http://www.aroma-project.org/. > To post to this group, send email to [email protected] > To unsubscribe and other options, go to http://www.aroma-project.org/forum/ -- When reporting problems on aroma.affymetrix, make sure 1) to run the latest version of the package, 2) to report the output of sessionInfo() and traceback(), and 3) to post a complete code example. You received this message because you are subscribed to the Google Groups "aroma.affymetrix" group with website http://www.aroma-project.org/. To post to this group, send email to [email protected] To unsubscribe and other options, go to http://www.aroma-project.org/forum/
