I've updated aroma.affymetrix v2.11.8 and friends. Update by:
source("http://callr.org/install#aroma.affymetrix";)
I'd be keen to hear whether this solves your problem or not.
/Henrik
On Thu, Sep 4, 2014 at 1:47 PM, Henrik Bengtsson <h...@biostat.ucsf.edu> wrote:
> Hi,
>
> good that it works for you with this workaround. The fix in aroma is
> fairly simple(*). I'll let you know when an updated version is
> available.
>
> (*) DETAILS: This has to do with base::file.copy() and its argument
> 'copy.mode' which was introduced in R 2.13.0 (April 2011). The thing
> is that they set the default to TRUE (whereas the previous behavior
> was equivalent to FALSE), meaning any files copied will inherit the
> file permissions from the source. Various steps in the aroma pipeline
> *copies* the source CEL file and uses that as a template to store
> updates signals. This is what hits you. I'll updating to make use of
> base::file.copy(..., copy.mode=FALSE) such that one can work with also
> read-only CEL files.
>
> Thanks for reporting on this (not too unlikely) use case
>
> Henrik
>
> On Thu, Sep 4, 2014 at 1:35 PM, Taylor Raborn <taylor.rab...@gmail.com> wrote:
>> Hi Henrik:
>>
>> Thank you for the prompt reply. You are right- the .CEL files are all read
>> only (see their permissions, below)
>>
>> rtraborn@Mason:
>> /N/dc2/scratch/rtraborn/ML_Project/NIK_2014/rawData/NIK_2014/HuEx-1_0-st-v2>
>> ls -lkhtra
>>
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709338_HSB194-VFC-R.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709337_HSB194-VFC-L.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709336_HSB194-V1C-R.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709335_HSB194-V1C-L.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709334_HSB194-STR-L.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709333_HSB194-STC-R.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709332_HSB194-STC-L.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709331_HSB194-S1C-R.CEL
>>
>> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709330_HSB194-S1C-L.CEL
>>
>>
>> After changing the permissions, the job performed without issue (I'm showing
>> you only the results of 1/9 for the sake of brevity):
>>
>>> csBC <- process(bc,verbose=verbose)
>>
>> Background correcting data set...
>>
>> Number of arrays: 9
>>
>> Array #1 ('GSM709330_HSB194-S1C-L') of 9...
>>
>> Adjusting PM signals only
>>
>> Obtaining signals...
>>
>> Obtaining signals...done
>>
>> Applying normal+exponential signal model...
>>
>> Applying normal+exponential signal model...done
>>
>> Writing adjusted probe signals...
>>
>> Adding temporary suffix from file...
>>
>> Pathname:
>> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>
>> Suffix: .tmp
>>
>> Rename existing file?: FALSE
>>
>> Temporary pathname:
>> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>>
>> Adding temporary suffix from file...done
>>
>> Creating CEL file for results, if missing...
>>
>> Creating CEL file for results, if missing...done
>>
>> Writing adjusted intensities...
>>
>> Writing adjusted intensities...done
>>
>> Dropping temporary suffix from file...
>>
>> Temporary pathname:
>> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>>
>> Suffix: .tmp
>>
>> Regular expression for suffix: \.tmp$
>>
>> Pathname:
>> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>
>> Renaming existing file...
>>
>> Result: TRUE
>>
>> Renaming existing file...done
>>
>> Dropping temporary suffix from file...done
>>
>> Writing adjusted probe signals...done
>>
>> used (Mb) gc trigger (Mb) max used (Mb)
>>
>> Ncells 709078 37.9 1166886 62.4 1166886 62.4
>>
>> Vcells 31586985 241.0 47054688 359.0 46991036 358.6
>>
>> AffymetrixCelFile:
>>
>> Name: GSM709330_HSB194-S1C-L
>>
>> Tags:
>>
>> Full name: GSM709330_HSB194-S1C-L
>>
>> Pathname:
>> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>
>> File size: 62.73 MB (65775403 bytes)
>>
>> RAM: 0.00 MB
>>
>> File format: v4 (binary; XDA)
>>
>> Platform: Affymetrix
>>
>> Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>>
>> Timestamp: 2011-01-25 12:15:03
>>
>> Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
>>
>>
>> -----
>>
>>
>> There must be a file permissions transfer behavior that takes place on a
>> multi-user cluster environment that doesn't take place on other types of
>> machines, because I was unable to reproduce this error elsewhere. I didn't
>> bother changing the raw .CEL files' permissions because I never experienced
>> a problem of this nature with .CEL files.
>>
>>
>> Let me know if you patch this up so I can pull the latest code from the
>> aroma repo.
>>
>>
>> Thanks for your help with this and for taking a look at the issue so
>> quickly.
>>
>>
>> Best regards,
>>
>>
>> Taylor
>>
>>
>>
>> On Thursday, September 4, 2014 1:37:36 PM UTC-4, Henrik Bengtsson wrote:
>>>
>>> Hi,
>>>
>>> interesting. A quick guess is that the input file, i.e.
>>>
>>> rawData/NIK_2014/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>>
>>> is write protected (e.g. file owned by someone else). This could be
>>> the reason, because internally that file is copied and used as a
>>> template and the copy may inherit the file permissions. What does ls
>>> -l say about the above file?
>>>
>>> If this is the case, I need to update the copying such that the new
>>> file has proper permissions.
>>>
>>> Thanks for the report
>>>
>>> Henrik
>>>
>>>
>>> On Thu, Sep 4, 2014 at 10:15 AM, Taylor Raborn <taylor...@gmail.com>
>>> wrote:
>>> > Hi Henrik:
>>> >
>>> > I just come across an unusual error when using aroma.affymetrix on our
>>> > high-memory HPC machine. It may or may not be platform-specific, and
>>> > I'll
>>> > give you all the information I can in the hopes of isolating and
>>> > identifying
>>> > the error.
>>> >
>>> > Traceback:
>>> >
>>> >> traceback()
>>> >
>>> > 18: stop(cond)
>>> >
>>> > 17: throw.Exception(Exception(...))
>>> >
>>> > 16: throw(Exception(...))
>>> >
>>> > 15: throw.default("No permission to modify existing file: ", pathname)
>>> >
>>> > 14: throw("No permission to modify existing file: ", pathname)
>>> >
>>> > 13: getWritablePathname.Arguments(static, ...)
>>> >
>>> > 12: getWritablePathname(static, ...) at <text>#1
>>> >
>>> > 11: Arguments$getWritablePathname(pathname, mustExist = TRUE)
>>> >
>>> > 10: renameFile.default(srcPathname, pathname, ...)
>>> >
>>> > 9: renameFile(srcPathname, pathname, ...)
>>> >
>>> > 8: renameTo.GenericDataFile(res, filename = pathname, verbose =
>>> > less(verbose))
>>> >
>>> > 7: renameTo(res, filename = pathname, verbose = less(verbose))
>>> >
>>> > 6: createFrom.AffymetrixCelFile(this, filename = pathnameT, path = NULL,
>>> >
>>> > verbose = less(verbose))
>>> >
>>> > 5: createFrom(this, filename = pathnameT, path = NULL, verbose =
>>> > less(verbose))
>>> >
>>> > 4: bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly = pmonly,
>>> >
>>> > addJitter = addJitter, jitterSd = jitterSd, overwrite = force,
>>> >
>>> > verbose = verbose, .deprecated = FALSE)
>>> >
>>> > 3: bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter =
>>> > addJitter,
>>> >
>>> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
>>> >
>>> > .deprecated = FALSE)
>>> >
>>> > 2: process.RmaBackgroundCorrection(bc, verbose = verbose)
>>> >
>>> > 1: process(bc, verbose = verbose)
>>> >
>>> >
>>> > ----
>>> >
>>> > sessionInfo:
>>> >
>>> >> sessionInfo()
>>> >
>>> > R version 3.0.1 (2013-05-16)
>>> >
>>> > Platform: x86_64-unknown-linux-gnu (64-bit)
>>> >
>>> >
>>> > locale:
>>> >
>>> > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>>> >
>>> > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>>> >
>>> > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>>> >
>>> > [7] LC_PAPER=C LC_NAME=C
>>> >
>>> > [9] LC_ADDRESS=C LC_TELEPHONE=C
>>> >
>>> > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>> >
>>> >
>>> > attached base packages:
>>> >
>>> > [1] tools parallel stats graphics grDevices utils datasets
>>> >
>>> > [8] methods base
>>> >
>>> >
>>> > other attached packages:
>>> >
>>> > [1] R.cache_0.10.0 base64enc_0.1-2 aroma.apd_0.5.0
>>> >
>>> > [4] preprocessCore_1.24.0 affyio_1.30.0 Biobase_2.20.1
>>> >
>>> > [7] BiocGenerics_0.6.0 aroma.light_1.32.0 matrixStats_0.10.0
>>> >
>>> > [10] aroma.affymetrix_2.12.0 aroma.core_2.12.1 R.devices_2.11.0
>>> >
>>> > [13] R.filesets_2.6.0 R.utils_1.33.0 R.oo_1.18.0
>>> >
>>> > [16] affxparser_1.34.2 affy_1.40.0 R.methodsS3_1.6.1
>>> >
>>> > [19] PSCBS_0.43.0 DNAcopy_1.36.0
>>> > BiocInstaller_1.12.1
>>> >
>>> >
>>> > loaded via a namespace (and not attached):
>>> >
>>> > [1] digest_0.6.4 R.huge_0.8.0 R.rsp_0.19.0 zlibbioc_1.11.1
>>> >
>>> >
>>> >
>>> > Here's how I produced the error. Note that my directory structure
>>> > matches
>>> > that on the aroma-project.org site, shown here:
>>> >
>>> >> chipType <- "HuEx-1_0-st-v2"
>>> >
>>> >
>>> >> cdf <- AffymetrixCdfFile$byChipType(chipType,
>>> >> tags="coreR3,A20071112,EP")
>>> >
>>> >
>>> >> print(cdf)
>>> >
>>> > AffymetrixCdfFile:
>>> >
>>> > Path: annotationData/chipTypes/HuEx-1_0-st-v2
>>> >
>>> > Filename: HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf
>>> >
>>> > File size: 38.25 MB (40108891 bytes)
>>> >
>>> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>>> >
>>> > RAM: 0.00MB
>>> >
>>> > File format: v4 (binary; XDA)
>>> >
>>> > Dimension: 2560x2560
>>> >
>>> > Number of cells: 6553600
>>> >
>>> > Number of units: 18708
>>> >
>>> > Cells per unit: 350.31
>>> >
>>> > Number of QC units: 1
>>> >
>>> >
>>> >> cs <- AffymetrixCelSet$byName("NIK_2014", cdf=cdf)
>>> >
>>> >
>>> >> print(cs)
>>> >
>>> > AffymetrixCelSet:
>>> >
>>> > Name: NIK_2014
>>> >
>>> > Tags:
>>> >
>>> > Path: rawData/NIK_2014/HuEx-1_0-st-v2
>>> >
>>> > Platform: Affymetrix
>>> >
>>> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>>> >
>>> > Number of arrays: 9
>>> >
>>> > Names: GSM709330_HSB194-S1C-L, GSM709331_HSB194-S1C-R,
>>> > GSM709332_HSB194-STC-L, ..., GSM709338_HSB194-VFC-R [9]
>>> >
>>> > Time period: 2011-01-25 12:15:03 -- 2011-01-25 13:48:02
>>> >
>>> > Total file size: 564.87MB
>>> >
>>> > RAM: 0.02MB
>>> >
>>> >
>>> >> csBC <- process(bc,verbose=verbose)
>>> >
>>> > Background correcting data set...
>>> >
>>> > Number of arrays: 9
>>> >
>>> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...
>>> >
>>> > Adjusting PM signals only
>>> >
>>> > Obtaining signals...
>>> >
>>> > Obtaining signals...done
>>> >
>>> > Applying normal+exponential signal model...
>>> >
>>> > Applying normal+exponential signal model...done
>>> >
>>> > Writing adjusted probe signals...
>>> >
>>> > Adding temporary suffix from file...
>>> >
>>> > Pathname:
>>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>> >
>>> > Suffix: .tmp
>>> >
>>> > Rename existing file?: FALSE
>>> >
>>> > Temporary pathname:
>>> >
>>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>>> >
>>> > Adding temporary suffix from file...done
>>> >
>>> > Creating CEL file for results, if missing...
>>> >
>>> > [2014-09-04 12:38:54] Exception: No permission to modify existing file:
>>> >
>>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>>> >
>>> >
>>> > at #13. getWritablePathname.Arguments(static, ...)
>>> >
>>> > - getWritablePathname.Arguments() is in environment 'R.utils'
>>> >
>>> >
>>> > at #12. getWritablePathname(static, ...)
>>> >
>>> > - getWritablePathname() is in environment 'R.utils'
>>> >
>>> > - originating from '<text>'
>>> >
>>> >
>>> > at #11. Arguments$getWritablePathname(pathname, mustExist = TRUE)
>>> >
>>> > - Arguments$getWritablePathname() is local of the calling
>>> > function
>>> >
>>> >
>>> > at #10. renameFile.default(srcPathname, pathname, ...)
>>> >
>>> > - renameFile.default() is in environment 'R.utils'
>>> >
>>> >
>>> > at #09. renameFile(srcPathname, pathname, ...)
>>> >
>>> > - renameFile() is in environment 'R.utils'
>>> >
>>> >
>>> > at #08. renameTo.GenericDataFile(res, filename = pathname, verbose =
>>> > less(verbose))
>>> >
>>> > - renameTo.GenericDataFile() is in environment 'R.filesets'
>>> >
>>> >
>>> > at #07. renameTo(res, filename = pathname, verbose = less(verbose))
>>> >
>>> > - renameTo() is in environment 'R.filesets'
>>> >
>>> >
>>> > at #06. createFrom.AffymetrixCelFile(this, filename = pathnameT, path
>>> > =
>>> > NULL,
>>> >
>>> > verbose = less(verbose))
>>> >
>>> > - createFrom.AffymetrixCelFile() is in environment
>>> > 'aroma.affymetrix'
>>> >
>>> >
>>> > at #05. createFrom(this, filename = pathnameT, path = NULL, verbose =
>>> > less(verbose))
>>> >
>>> > - createFrom() is in environment 'aroma.affymetrix'
>>> >
>>> >
>>> > at #04. bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly =
>>> > pmonly,
>>> >
>>> > addJitter = addJitter, jitterSd = jitterSd, overwrite =
>>> > force,
>>> >
>>> > verbose = verbose, .deprecated = FALSE)
>>> >
>>> > - bgAdjustRma.AffymetrixCelFile() is in environment
>>> > 'aroma.affymetrix'
>>> >
>>> >
>>> > at #03. bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter
>>> > =
>>> > addJitter,
>>> >
>>> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
>>> >
>>> > .deprecated = FALSE)
>>> >
>>> > - bgAdjustRma() is in environment 'aroma.affymetrix'
>>> >
>>> >
>>> > at #02. process.RmaBackgroundCorrection(bc, verbose = verbose)
>>> >
>>> > - process.RmaBackgroundCorrection() is in environment
>>> > 'aroma.affymetrix'
>>> >
>>> >
>>> > at #01. process(bc, verbose = verbose)
>>> >
>>> > - process() is in environment 'aroma.core'
>>> >
>>> >
>>> > Error: No permission to modify existing file:
>>> >
>>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>>> >
>>> > Creating CEL file for results, if missing...done
>>> >
>>> > Writing adjusted probe signals...done
>>> >
>>> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
>>> >
>>> > Background correcting data set...done
>>> >
>>> > --------------------
>>> >
>>> >
>>> > I tried to find a solution from the group archives and other places to
>>> > no
>>> > avail. It's worth mentioning that I get this problem even if I open up
>>> > permissions to the entire directory. The *CEL.tmp.tmp file appears to be
>>> > written with write-protections, which is baffling to me. The machine is
>>> > a
>>> > large, high-memory cluster from my institution, and the error may be
>>> > associated with this.
>>> >
>>> >
>>> > Thanks in advance for any insight you can provide with this.
>>> >
>>> >
>>> > Best regards,
>>> >
>>> >
>>> > Taylor
>>> >
>>> >
>>> > --
>>> > --
>>> > When reporting problems on aroma.affymetrix, make sure 1) to run the
>>> > latest
>>> > version of the package, 2) to report the output of sessionInfo() and
>>> > traceback(), and 3) to post a complete code example.
>>> >
>>> >
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "aroma.affymetrix" group with website http://www.aroma-project.org/.
>>> > To post to this group, send email to aroma-af...@googlegroups.com
>>> > To unsubscribe and other options, go to
>>> > http://www.aroma-project.org/forum/
>>> >
>>> > ---
>>> > You received this message because you are subscribed to the Google
>>> > Groups
>>> > "aroma.affymetrix" group.
>>> > To unsubscribe from this group and stop receiving emails from it, send
>>> > an
>>> > email to aroma-affymetr...@googlegroups.com.
>>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> --
>> When reporting problems on aroma.affymetrix, make sure 1) to run the latest
>> version of the package, 2) to report the output of sessionInfo() and
>> traceback(), and 3) to post a complete code example.
>>
>>
>> You received this message because you are subscribed to the Google Groups
>> "aroma.affymetrix" group with website http://www.aroma-project.org/.
>> To post to this group, send email to aroma-affymetrix@googlegroups.com
>> To unsubscribe and other options, go to http://www.aroma-project.org/forum/
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "aroma.affymetrix" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to aroma-affymetrix+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
--
--
When reporting problems on aroma.affymetrix, make sure 1) to run the latest
version of the package, 2) to report the output of sessionInfo() and
traceback(), and 3) to post a complete code example.
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group with website http://www.aroma-project.org/.
To post to this group, send email to aroma-affymetrix@googlegroups.com
To unsubscribe and other options, go to http://www.aroma-project.org/forum/
---
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to aroma-affymetrix+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
