I fail to reproduce this with R 3.0.2 and affxparser 1.34.4 and
aroma.affymetrix_2.12.8 and the same data file (just like you).
However, from code inspection it could be that you have some cached
results from an earlier run that used a CDF with the exact same
filename but with an actually different content - could that be the
case? (I'll try to close that pitfall in the next release). If this
is the problem, then try to clear your cache. The content in
~/.Rcache/ is non-critical and can be deleted at any time. You can
erase the cache like this:
> library("R.cache")
> clearCache()
Are you really sure you want to delete the 184 files and 16
directories in '/home/JohnDoe/.Rcache'? [y/N]: y
The only thing you'll notice is that some steps will be slow when you
run the first time after clearing the cache. This is the Aroma
Framework recalculating/regenerated part of the cache.
Let me know if this works. If not, other ideas are:
* You seem to have other packages loaded/attached too. Retry in a
fresh R session, e.g. R --vanilla.
* Your R version and therefore your affxparser versions are outdated.
I would retry with a recent version of R, i.e. R >= 3.1.2.
/Henrik
On Mon, Dec 29, 2014 at 5:20 PM, Amon <aisyah...@gmail.com> wrote:
> Below is the output (for the binary, not text, CEL file) :
>
>> cf <- cs[[1]]
>> print(cf) # Should be the 'EA_BT549' CEL file
> AffymetrixCelFile:
> Name: EA_BT549
> Tags:
> Full name: EA_BT549
> Pathname: rawData/myData/HuEx-1_0-st-v2/EA_BT549.CEL
> File size: 63.16 MB (66226783 bytes)
> RAM: 0.01 MB
> File format: v4 (binary; XDA)
> Platform: Affymetrix
> Chip type: HuEx-1_0-st-v2,fullR3,A20071112,EP
> Timestamp: 2007-08-30 20:27:52
>> str(getHeader(cf))
> List of 14
> $ filename : chr "rawData/myData/HuEx-1_0-st-v2/EA_BT549.CEL"
> $ version : int 4
> $ cols : int 2560
> $ rows : int 2560
> $ total : int 6553600
> $ algorithm : chr "Percentile"
> $ parameters : chr
> "Percentile:75;CellMargin:4;OutlierHigh:1.500;OutlierLow:1.004;AlgVersion:6.0;FixedCellSize:TRUE;FullFeatureWidth:7;FullFeatureH"|
> __truncated__
> $ chiptype : chr "HuEx-1_0-st-v2"
> $ header : chr
> "Cols=2560\nRows=2560\nTotalX=2560\nTotalY=2560\nOffsetX=0\nOffsetY=0\nGridCornerUL=617
> 453\nGridCornerUR=18881 579\nGridCornerL"| __truncated__
> $ datheader : chr "[0..65534] 0293_BT549:CLS=19341RWS=19341XIN=0
> YIN=0 VE=30 2.0 08/30/07 20:27:52 54716690 M10 \024 \024
> HuEx-1_0-s"| __truncated__
> $ librarypackage: chr ""
> $ cellmargin : int 4
> $ noutliers : int 172409
> $ nmasked : int 0
>
>
> If it helps for you to visually inspect the file yourself, I downloaded this
> file from a publicly available dataset here
> http://www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-181/samples/ BT549 is
> the 8th row from the top I think.
>
> Best
>
> Aisyah
>
> On Tuesday, December 30, 2014 9:11:26 AM UTC+8, Henrik Bengtsson wrote:
>>
>> It turns out this error is triggered inside affxparser::updateCel()
>> and it looks like it can happen for two reasons, where one is due to a
>> corrupt/corruptly read CEL header. What does
>>
>> > cdf <- AffymetrixCdfFile$byChipType("HuEx-1_0-st-v2",
>> > tags="fullR3,A20071112,EP")
>> > cs <- AffymetrixCelSet$byName("myData", cdf=cdf)
>> > cf <- cs[[1]]
>> > print(cf) # Should be the 'EA_BT549' CEL file
>> > str(getHeader(cf))
>>
>> output?
>>
>> /Henrik
>>
>>
>> On Mon, Dec 29, 2014 at 4:39 PM, Amon <aisy...@gmail.com> wrote:
>> > Hi Henrik, thanks for your very prompt response. Below is the error
>> > message
>> > I got when I tried to run the same code on the unconverted (i.e.,
>> > original)
>> > v4 CEL file. I've reddened the area that alerted me to the possibility
>> > that
>> > it's the format of the file that caused it not to work (the binary
>> > format
>> > possibly causes NA values, creating the "missing value where TRUE/FALSE
>> > needed" error), because as I said I've run this preprocessing step on my
>> > v3
>> > data, using the same CDF, many times with no problems before:
>> >
>> > 20141229 05:17:51|Background correcting data set...
>> > 20141229 05:17:51| Number of arrays: 1
>> > 20141229 05:17:51| Array #1 ('EA_BT549') of 1...
>> > 20141229 05:17:52| Adjusting PM signals only
>> > 20141229 05:17:52| Obtaining signals...
>> > 20141229 05:17:53| Obtaining signals...done
>> > 20141229 05:17:53| Applying normal+exponential signal model...
>> > 20141229 05:17:55| Applying normal+exponential signal model...done
>> > 20141229 05:17:55| Writing adjusted probe signals...
>> > 20141229 05:17:55| Adding temporary suffix from file...
>> > 20141229 05:17:55| Pathname:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL
>> > 20141229 05:17:55| Suffix: .tmp
>> > 20141229 05:17:55| Rename existing file?: FALSE
>> > 20141229 05:17:55| Temporary pathname:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> > 20141229 05:17:55| Adding temporary suffix from file...done
>> > 20141229 05:17:55| Creating CEL file for results, if missing...
>> > 20141229 05:17:55| Creating CEL file...
>> > 20141229 05:17:55| Chip type: HuEx-1_0-st-v2,fullR3,A20071112,EP
>> > 20141229 05:17:55| Pathname:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> > 20141229 05:17:55| Adding temporary suffix from file...
>> > 20141229 05:17:55| Pathname:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> > 20141229 05:17:55| Suffix: .tmp
>> > 20141229 05:17:55| Rename existing file?: FALSE
>> > 20141229 05:17:55| Temporary pathname:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp.tmp
>> > 20141229 05:17:55| Adding temporary suffix from file...done
>> > 20141229 05:17:55| Method 'copy'...
>> > 20141229 05:17:55| Copying file safely...
>> > 20141229 05:17:55| Source:
>> > rawData/myData/HuEx-1_0-st-v2/EA_BT549.CEL
>> > 20141229 05:17:55| Destination:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp.tmp
>> > 20141229 05:17:55| Copying to temporary file using file.copy()...
>> > 20141229 05:17:57| Copying to temporary file using
>> > file.copy()...done
>> > 20141229 05:17:57| Renaming temporary file to destination name...
>> > 20141229 05:17:57| Renaming temporary file to destination
>> > name...done
>> > 20141229 05:17:57| Validating destination file...
>> > 20141229 05:17:57| Validating destination file...done
>> > 20141229 05:17:57| Copying file safely...done
>> > 20141229 05:17:57| Renaming AffymetrixCelFile pathname...
>> > 20141229 05:17:57| Source:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp.tmp
>> > 20141229 05:17:57| Destination:
>> > probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> > 20141229 05:17:57| Renaming file...
>> > 20141229 05:17:57| Renaming file...done
>> > 20141229 05:17:57| Renaming AffymetrixCelFile pathname...done
>> > 20141229 05:17:57| Method 'copy'...done
>> > AffymetrixCelFile:
>> > Name: EA_BT549.CEL.tmp
>> > Tags:
>> > Full name: EA_BT549.CEL.tmp
>> > Pathname: probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> > File size: 63.16 MB (66226783 bytes)
>> > RAM: 0.00 MB
>> > File format: v4 (binary; XDA)
>> > Platform: Affymetrix
>> > Chip type: HuEx-1_0-st-v2,fullR3,A20071112,EP
>> > Timestamp: 2007-08-30 20:27:52
>> > 20141229 05:17:58| Creating CEL file...done
>> > 20141229 05:17:58| Creating CEL file for results, if missing...done
>> > 20141229 05:17:58| Writing adjusted intensities...
>> > Error in if (r[1] < 1 || r[2] > nbrOfCells) { :
>> > missing value where TRUE/FALSE needed
>> > 20141229 05:17:59| Writing adjusted intensities...done
>> > 20141229 05:17:59| Writing adjusted probe signals...done
>> > 20141229 05:17:59| Array #1 ('EA_BT549') of 1...done
>> > 20141229 05:17:59|Background correcting data set...done
>> >
>> >
>> > As a result, no csBC object is created.
>> >
>> > Aisyah
>> >
>> >
>> > On Tuesday, December 30, 2014 6:55:59 AM UTC+8, Henrik Bengtsson wrote:
>> >>
>> >> On Mon, Dec 29, 2014 at 1:37 PM, Amon <aisy...@gmail.com> wrote:
>> >> > Hello,
>> >> >
>> >> > I'd like to perform background correction on a HuEx-1_0-st-v2 CEL
>> >> > file
>> >> > that
>> >> > I've previously converted from xda binary format (Affymetrix v4) to
>> >> > ASCII
>> >> > text format (Affymetrix v3). I did the file format conversion because
>> >> > aroma
>> >> > does not work xda-format CEL files,
>> >>
>> >> This is not correct. Where did you read this (because if it's written
>> >> somewhere it should be corrected)?
>> >>
>> >> The aroma.affymetrix package works with the binary/XDA CEL file format
>> >> too. From the front page of http://aroma-project.org/:
>> >>
>> >> "File formats: Works directly with CEL and CDF files (all versions;
>> >> text/ASCII, binary/XDA, binary/Calvin)."
>> >>
>> >> It's unnecessary to convert back to ASCII and it would also slow things
>> >> down.
>> >>
>> >> I would retry with the binary/XDA CEL files and see if you still get
>> >> the error. If so, I'll look into it. If the error only occurs with
>> >> ASCII CEL files, I'll add it to the to do list to troubleshoot.
>> >>
>> >> Hope this helps
>> >>
>> >> Henrik
>> >>
>> >> > and I used apt-cel-convert provided by
>> >> > Affymetrix Power Tools for this conversion. There are several error
>> >> > messages
>> >> > that come up when I use RmaBackgroundCorrection() on the file, some
>> >> > of
>> >> > which
>> >> > are:
>> >> >
>> >> > 20141229 20:59:04| Cannot create CEL file of version 4
>> >> > (probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp). Template CEL
>> >> > file is
>> >> > of version 3: rawData/myData/HuEx-1_0-st-v2/EA_BT549.CEL
>> >> > 20141229 21:00:50| Method 'create'...done
>> >> > AffymetrixCelFile:
>> >> > Name: EA_BT549.CEL.tmp
>> >> > Tags:
>> >> > Full name: EA_BT549.CEL.tmp
>> >> > Pathname: probeData/myData,RBC/HuEx-1_0-st-v2/EA_BT549.CEL.tmp
>> >> > File size: 62.50 MB (65536700 bytes)
>> >> > RAM: 0.00 MB
>> >> > File format: v4 (binary; XDA)
>> >> > Platform: Affymetrix
>> >> > Chip type: HuEx-1_0-st-v2,fullR3,A20071112,EP
>> >> > Timestamp: 2014-12-29 20:59:04
>> >> > 20141229 21:00:51| Creating CEL file...done
>> >> > 20141229 21:00:51| Creating CEL file for results, if missing...done
>> >> > 20141229 21:00:51| Writing adjusted intensities...
>> >> > Error in if (r[1] < 1 || r[2] > nbrOfCells) { :
>> >> > missing value where TRUE/FALSE needed
>> >> > 20141229 21:00:51| Writing adjusted intensities...done
>> >> > 20141229 21:00:51| Writing adjusted probe signals...done
>> >> > 20141229 21:00:51| Array #1 ('EA_BT549') of 1...done
>> >> > 20141229 21:00:51|Background correcting data set...done
>> >> >
>> >> > I've provided the full report of the process in the attachment. From
>> >> > the
>> >> > error message, it seems that the information from the original v4
>> >> > file
>> >> > has
>> >> > been retained somehow. Is there an option I can use in the
>> >> > RmaBackgroundCorrection() function to fix this? Steps I used prior to
>> >> > this
>> >> > were those found in the aroma page (additionally, I've used this on
>> >> > my
>> >> > v3
>> >> > data many times before with no problems):
>> >> >
>> >> > chipType <- "HuEx-1_0-st-v2"
>> >> > cdf <- AffymetrixCdfFile$byChipType(chipType, tags =
>> >> > "fullR3,A20071112,EP")
>> >> > verbose <- Arguments$getVerbose(-8, timestamp=TRUE)
>> >> > cs <- AffymetrixCelSet$byName("myData", cdf=cdf)
>> >> > bc <- RmaBackgroundCorrection(cs)
>> >> > csBC <- process(bc, verbose = verbose)
>> >> >
>> >> > -----------------------------------------------------
>> >> >
>> >> >> sessionInfo()
>> >> > R version 3.0.2 (2013-09-25)
>> >> > Platform: x86_64-redhat-linux-gnu (64-bit)
>> >> >
>> >> > locale:
>> >> > [1] C
>> >> >
>> >> > attached base packages:
>> >> > [1] splines grid parallel stats graphics grDevices
>> >> > utils
>> >> > datasets methods base
>> >> >
>> >> > other attached packages:
>> >> > [1] Hmisc_3.14-6 Formula_1.1-2 survival_2.37-7
>> >> > lattice_0.20-29 aroma.light_2.0.0
>> >> > [6] matrixStats_0.10.0 aroma.affymetrix_2.12.8
>> >> > aroma.core_2.12.8
>> >> > R.devices_2.12.0 R.filesets_2.6.0
>> >> > [11] affy_1.40.0 Biobase_2.22.0
>> >> > BiocGenerics_0.8.0
>> >> > affxparser_1.34.2 R.utils_1.34.0
>> >> > [16] R.oo_1.18.2 R.methodsS3_1.6.2
>> >> >
>> >> > loaded via a namespace (and not attached):
>> >> > [1] BiocInstaller_1.12.1 DNAcopy_1.36.0 PSCBS_0.43.0
>> >> > R.cache_0.11.0 R.huge_0.8.0 R.rsp_0.19.6
>> >> > [7] RColorBrewer_1.1-2 acepack_1.3-3.3 affyio_1.30.0
>> >> > aroma.apd_0.5.0 base64enc_0.1-2 cluster_1.15.3
>> >> > [13] digest_0.6.7 foreign_0.8-61 latticeExtra_0.6-26
>> >> > nnet_7.3-8 preprocessCore_1.24.0 rpart_4.1-8
>> >> > [19] tools_3.0.2 zlibbioc_1.8.0
>> >> >
>> >> >
>> >> >
>> >> > Many thanks in advance.
>> >> >
>> >> > Aish
>> >> >
>> >> > --
>> >> > --
>> >> > When reporting problems on aroma.affymetrix, make sure 1) to run the
>> >> > latest
>> >> > version of the package, 2) to report the output of sessionInfo() and
>> >> > traceback(), and 3) to post a complete code example.
>> >> >
>> >> >
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "aroma.affymetrix" group with website http://www.aroma-project.org/.
>> >> > To post to this group, send email to aroma-af...@googlegroups.com
>> >> > To unsubscribe and other options, go to
>> >> > http://www.aroma-project.org/forum/
>> >> >
>> >> > ---
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "aroma.affymetrix" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an
>> >> > email to aroma-affymetr...@googlegroups.com.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > --
>> > When reporting problems on aroma.affymetrix, make sure 1) to run the
>> > latest
>> > version of the package, 2) to report the output of sessionInfo() and
>> > traceback(), and 3) to post a complete code example.
>> >
>> >
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "aroma.affymetrix" group with website http://www.aroma-project.org/.
>> > To post to this group, send email to aroma-af...@googlegroups.com
>> > To unsubscribe and other options, go to
>> > http://www.aroma-project.org/forum/
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "aroma.affymetrix" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to aroma-affymetr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> When reporting problems on aroma.affymetrix, make sure 1) to run the latest
> version of the package, 2) to report the output of sessionInfo() and
> traceback(), and 3) to post a complete code example.
>
>
> You received this message because you are subscribed to the Google Groups
> "aroma.affymetrix" group with website http://www.aroma-project.org/.
> To post to this group, send email to aroma-affymetrix@googlegroups.com
> To unsubscribe and other options, go to http://www.aroma-project.org/forum/
>
> ---
> You received this message because you are subscribed to the Google Groups
> "aroma.affymetrix" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to aroma-affymetrix+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
--
When reporting problems on aroma.affymetrix, make sure 1) to run the latest
version of the package, 2) to report the output of sessionInfo() and
traceback(), and 3) to post a complete code example.
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group with website http://www.aroma-project.org/.
To post to this group, send email to aroma-affymetrix@googlegroups.com
To unsubscribe and other options, go to http://www.aroma-project.org/forum/
---
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to aroma-affymetrix+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
