Hi,
good that it works for you with this workaround. The fix in aroma is
fairly simple(*). I'll let you know when an updated version is
available.
(*) DETAILS: This has to do with base::file.copy() and its argument
'copy.mode' which was introduced in R 2.13.0 (April 2011). The thing
is that they set the default to TRUE (whereas the previous behavior
was equivalent to FALSE), meaning any files copied will inherit the
file permissions from the source. Various steps in the aroma pipeline
*copies* the source CEL file and uses that as a template to store
updates signals. This is what hits you. I'll updating to make use of
base::file.copy(..., copy.mode=FALSE) such that one can work with also
read-only CEL files.
Thanks for reporting on this (not too unlikely) use case
Henrik
On Thu, Sep 4, 2014 at 1:35 PM, Taylor Raborn <taylor.rab...@gmail.com> wrote:
> Hi Henrik:
>
> Thank you for the prompt reply. You are right- the .CEL files are all read
> only (see their permissions, below)
>
> rtraborn@Mason:
> /N/dc2/scratch/rtraborn/ML_Project/NIK_2014/rawData/NIK_2014/HuEx-1_0-st-v2>
> ls -lkhtra
>
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709338_HSB194-VFC-R.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709337_HSB194-VFC-L.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709336_HSB194-V1C-R.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709335_HSB194-V1C-L.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709334_HSB194-STR-L.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709333_HSB194-STC-R.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709332_HSB194-STC-L.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709331_HSB194-S1C-R.CEL
>
> -r--r--r-- 1 ... 63M Jan 25 2011 GSM709330_HSB194-S1C-L.CEL
>
>
> After changing the permissions, the job performed without issue (I'm showing
> you only the results of 1/9 for the sake of brevity):
>
>> csBC <- process(bc,verbose=verbose)
>
> Background correcting data set...
>
> Number of arrays: 9
>
> Array #1 ('GSM709330_HSB194-S1C-L') of 9...
>
> Adjusting PM signals only
>
> Obtaining signals...
>
> Obtaining signals...done
>
> Applying normal+exponential signal model...
>
> Applying normal+exponential signal model...done
>
> Writing adjusted probe signals...
>
> Adding temporary suffix from file...
>
> Pathname:
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>
> Suffix: .tmp
>
> Rename existing file?: FALSE
>
> Temporary pathname:
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>
> Adding temporary suffix from file...done
>
> Creating CEL file for results, if missing...
>
> Creating CEL file for results, if missing...done
>
> Writing adjusted intensities...
>
> Writing adjusted intensities...done
>
> Dropping temporary suffix from file...
>
> Temporary pathname:
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>
> Suffix: .tmp
>
> Regular expression for suffix: \.tmp$
>
> Pathname:
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>
> Renaming existing file...
>
> Result: TRUE
>
> Renaming existing file...done
>
> Dropping temporary suffix from file...done
>
> Writing adjusted probe signals...done
>
> used (Mb) gc trigger (Mb) max used (Mb)
>
> Ncells 709078 37.9 1166886 62.4 1166886 62.4
>
> Vcells 31586985 241.0 47054688 359.0 46991036 358.6
>
> AffymetrixCelFile:
>
> Name: GSM709330_HSB194-S1C-L
>
> Tags:
>
> Full name: GSM709330_HSB194-S1C-L
>
> Pathname:
> probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>
> File size: 62.73 MB (65775403 bytes)
>
> RAM: 0.00 MB
>
> File format: v4 (binary; XDA)
>
> Platform: Affymetrix
>
> Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>
> Timestamp: 2011-01-25 12:15:03
>
> Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
>
>
> -----
>
>
> There must be a file permissions transfer behavior that takes place on a
> multi-user cluster environment that doesn't take place on other types of
> machines, because I was unable to reproduce this error elsewhere. I didn't
> bother changing the raw .CEL files' permissions because I never experienced
> a problem of this nature with .CEL files.
>
>
> Let me know if you patch this up so I can pull the latest code from the
> aroma repo.
>
>
> Thanks for your help with this and for taking a look at the issue so
> quickly.
>
>
> Best regards,
>
>
> Taylor
>
>
>
> On Thursday, September 4, 2014 1:37:36 PM UTC-4, Henrik Bengtsson wrote:
>>
>> Hi,
>>
>> interesting. A quick guess is that the input file, i.e.
>>
>> rawData/NIK_2014/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>>
>> is write protected (e.g. file owned by someone else). This could be
>> the reason, because internally that file is copied and used as a
>> template and the copy may inherit the file permissions. What does ls
>> -l say about the above file?
>>
>> If this is the case, I need to update the copying such that the new
>> file has proper permissions.
>>
>> Thanks for the report
>>
>> Henrik
>>
>>
>> On Thu, Sep 4, 2014 at 10:15 AM, Taylor Raborn <taylor...@gmail.com>
>> wrote:
>> > Hi Henrik:
>> >
>> > I just come across an unusual error when using aroma.affymetrix on our
>> > high-memory HPC machine. It may or may not be platform-specific, and
>> > I'll
>> > give you all the information I can in the hopes of isolating and
>> > identifying
>> > the error.
>> >
>> > Traceback:
>> >
>> >> traceback()
>> >
>> > 18: stop(cond)
>> >
>> > 17: throw.Exception(Exception(...))
>> >
>> > 16: throw(Exception(...))
>> >
>> > 15: throw.default("No permission to modify existing file: ", pathname)
>> >
>> > 14: throw("No permission to modify existing file: ", pathname)
>> >
>> > 13: getWritablePathname.Arguments(static, ...)
>> >
>> > 12: getWritablePathname(static, ...) at <text>#1
>> >
>> > 11: Arguments$getWritablePathname(pathname, mustExist = TRUE)
>> >
>> > 10: renameFile.default(srcPathname, pathname, ...)
>> >
>> > 9: renameFile(srcPathname, pathname, ...)
>> >
>> > 8: renameTo.GenericDataFile(res, filename = pathname, verbose =
>> > less(verbose))
>> >
>> > 7: renameTo(res, filename = pathname, verbose = less(verbose))
>> >
>> > 6: createFrom.AffymetrixCelFile(this, filename = pathnameT, path = NULL,
>> >
>> > verbose = less(verbose))
>> >
>> > 5: createFrom(this, filename = pathnameT, path = NULL, verbose =
>> > less(verbose))
>> >
>> > 4: bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly = pmonly,
>> >
>> > addJitter = addJitter, jitterSd = jitterSd, overwrite = force,
>> >
>> > verbose = verbose, .deprecated = FALSE)
>> >
>> > 3: bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter =
>> > addJitter,
>> >
>> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
>> >
>> > .deprecated = FALSE)
>> >
>> > 2: process.RmaBackgroundCorrection(bc, verbose = verbose)
>> >
>> > 1: process(bc, verbose = verbose)
>> >
>> >
>> > ----
>> >
>> > sessionInfo:
>> >
>> >> sessionInfo()
>> >
>> > R version 3.0.1 (2013-05-16)
>> >
>> > Platform: x86_64-unknown-linux-gnu (64-bit)
>> >
>> >
>> > locale:
>> >
>> > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
>> >
>> > [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
>> >
>> > [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
>> >
>> > [7] LC_PAPER=C LC_NAME=C
>> >
>> > [9] LC_ADDRESS=C LC_TELEPHONE=C
>> >
>> > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>> >
>> >
>> > attached base packages:
>> >
>> > [1] tools parallel stats graphics grDevices utils datasets
>> >
>> > [8] methods base
>> >
>> >
>> > other attached packages:
>> >
>> > [1] R.cache_0.10.0 base64enc_0.1-2 aroma.apd_0.5.0
>> >
>> > [4] preprocessCore_1.24.0 affyio_1.30.0 Biobase_2.20.1
>> >
>> > [7] BiocGenerics_0.6.0 aroma.light_1.32.0 matrixStats_0.10.0
>> >
>> > [10] aroma.affymetrix_2.12.0 aroma.core_2.12.1 R.devices_2.11.0
>> >
>> > [13] R.filesets_2.6.0 R.utils_1.33.0 R.oo_1.18.0
>> >
>> > [16] affxparser_1.34.2 affy_1.40.0 R.methodsS3_1.6.1
>> >
>> > [19] PSCBS_0.43.0 DNAcopy_1.36.0
>> > BiocInstaller_1.12.1
>> >
>> >
>> > loaded via a namespace (and not attached):
>> >
>> > [1] digest_0.6.4 R.huge_0.8.0 R.rsp_0.19.0 zlibbioc_1.11.1
>> >
>> >
>> >
>> > Here's how I produced the error. Note that my directory structure
>> > matches
>> > that on the aroma-project.org site, shown here:
>> >
>> >> chipType <- "HuEx-1_0-st-v2"
>> >
>> >
>> >> cdf <- AffymetrixCdfFile$byChipType(chipType,
>> >> tags="coreR3,A20071112,EP")
>> >
>> >
>> >> print(cdf)
>> >
>> > AffymetrixCdfFile:
>> >
>> > Path: annotationData/chipTypes/HuEx-1_0-st-v2
>> >
>> > Filename: HuEx-1_0-st-v2,coreR3,A20071112,EP.cdf
>> >
>> > File size: 38.25 MB (40108891 bytes)
>> >
>> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>> >
>> > RAM: 0.00MB
>> >
>> > File format: v4 (binary; XDA)
>> >
>> > Dimension: 2560x2560
>> >
>> > Number of cells: 6553600
>> >
>> > Number of units: 18708
>> >
>> > Cells per unit: 350.31
>> >
>> > Number of QC units: 1
>> >
>> >
>> >> cs <- AffymetrixCelSet$byName("NIK_2014", cdf=cdf)
>> >
>> >
>> >> print(cs)
>> >
>> > AffymetrixCelSet:
>> >
>> > Name: NIK_2014
>> >
>> > Tags:
>> >
>> > Path: rawData/NIK_2014/HuEx-1_0-st-v2
>> >
>> > Platform: Affymetrix
>> >
>> > Chip type: HuEx-1_0-st-v2,coreR3,A20071112,EP
>> >
>> > Number of arrays: 9
>> >
>> > Names: GSM709330_HSB194-S1C-L, GSM709331_HSB194-S1C-R,
>> > GSM709332_HSB194-STC-L, ..., GSM709338_HSB194-VFC-R [9]
>> >
>> > Time period: 2011-01-25 12:15:03 -- 2011-01-25 13:48:02
>> >
>> > Total file size: 564.87MB
>> >
>> > RAM: 0.02MB
>> >
>> >
>> >> csBC <- process(bc,verbose=verbose)
>> >
>> > Background correcting data set...
>> >
>> > Number of arrays: 9
>> >
>> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...
>> >
>> > Adjusting PM signals only
>> >
>> > Obtaining signals...
>> >
>> > Obtaining signals...done
>> >
>> > Applying normal+exponential signal model...
>> >
>> > Applying normal+exponential signal model...done
>> >
>> > Writing adjusted probe signals...
>> >
>> > Adding temporary suffix from file...
>> >
>> > Pathname:
>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL
>> >
>> > Suffix: .tmp
>> >
>> > Rename existing file?: FALSE
>> >
>> > Temporary pathname:
>> >
>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp
>> >
>> > Adding temporary suffix from file...done
>> >
>> > Creating CEL file for results, if missing...
>> >
>> > [2014-09-04 12:38:54] Exception: No permission to modify existing file:
>> >
>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>> >
>> >
>> > at #13. getWritablePathname.Arguments(static, ...)
>> >
>> > - getWritablePathname.Arguments() is in environment 'R.utils'
>> >
>> >
>> > at #12. getWritablePathname(static, ...)
>> >
>> > - getWritablePathname() is in environment 'R.utils'
>> >
>> > - originating from '<text>'
>> >
>> >
>> > at #11. Arguments$getWritablePathname(pathname, mustExist = TRUE)
>> >
>> > - Arguments$getWritablePathname() is local of the calling
>> > function
>> >
>> >
>> > at #10. renameFile.default(srcPathname, pathname, ...)
>> >
>> > - renameFile.default() is in environment 'R.utils'
>> >
>> >
>> > at #09. renameFile(srcPathname, pathname, ...)
>> >
>> > - renameFile() is in environment 'R.utils'
>> >
>> >
>> > at #08. renameTo.GenericDataFile(res, filename = pathname, verbose =
>> > less(verbose))
>> >
>> > - renameTo.GenericDataFile() is in environment 'R.filesets'
>> >
>> >
>> > at #07. renameTo(res, filename = pathname, verbose = less(verbose))
>> >
>> > - renameTo() is in environment 'R.filesets'
>> >
>> >
>> > at #06. createFrom.AffymetrixCelFile(this, filename = pathnameT, path
>> > =
>> > NULL,
>> >
>> > verbose = less(verbose))
>> >
>> > - createFrom.AffymetrixCelFile() is in environment
>> > 'aroma.affymetrix'
>> >
>> >
>> > at #05. createFrom(this, filename = pathnameT, path = NULL, verbose =
>> > less(verbose))
>> >
>> > - createFrom() is in environment 'aroma.affymetrix'
>> >
>> >
>> > at #04. bgAdjustRma.AffymetrixCelFile(df, path = outputPath, pmonly =
>> > pmonly,
>> >
>> > addJitter = addJitter, jitterSd = jitterSd, overwrite =
>> > force,
>> >
>> > verbose = verbose, .deprecated = FALSE)
>> >
>> > - bgAdjustRma.AffymetrixCelFile() is in environment
>> > 'aroma.affymetrix'
>> >
>> >
>> > at #03. bgAdjustRma(df, path = outputPath, pmonly = pmonly, addJitter
>> > =
>> > addJitter,
>> >
>> > jitterSd = jitterSd, overwrite = force, verbose = verbose,
>> >
>> > .deprecated = FALSE)
>> >
>> > - bgAdjustRma() is in environment 'aroma.affymetrix'
>> >
>> >
>> > at #02. process.RmaBackgroundCorrection(bc, verbose = verbose)
>> >
>> > - process.RmaBackgroundCorrection() is in environment
>> > 'aroma.affymetrix'
>> >
>> >
>> > at #01. process(bc, verbose = verbose)
>> >
>> > - process() is in environment 'aroma.core'
>> >
>> >
>> > Error: No permission to modify existing file:
>> >
>> > probeData/NIK_2014,RBC,coreR3/HuEx-1_0-st-v2/GSM709330_HSB194-S1C-L.CEL.tmp.tmp
>> >
>> > Creating CEL file for results, if missing...done
>> >
>> > Writing adjusted probe signals...done
>> >
>> > Array #1 ('GSM709330_HSB194-S1C-L') of 9...done
>> >
>> > Background correcting data set...done
>> >
>> > --------------------
>> >
>> >
>> > I tried to find a solution from the group archives and other places to
>> > no
>> > avail. It's worth mentioning that I get this problem even if I open up
>> > permissions to the entire directory. The *CEL.tmp.tmp file appears to be
>> > written with write-protections, which is baffling to me. The machine is
>> > a
>> > large, high-memory cluster from my institution, and the error may be
>> > associated with this.
>> >
>> >
>> > Thanks in advance for any insight you can provide with this.
>> >
>> >
>> > Best regards,
>> >
>> >
>> > Taylor
>> >
>> >
>> > --
>> > --
>> > When reporting problems on aroma.affymetrix, make sure 1) to run the
>> > latest
>> > version of the package, 2) to report the output of sessionInfo() and
>> > traceback(), and 3) to post a complete code example.
>> >
>> >
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "aroma.affymetrix" group with website http://www.aroma-project.org/.
>> > To post to this group, send email to aroma-af...@googlegroups.com
>> > To unsubscribe and other options, go to
>> > http://www.aroma-project.org/forum/
>> >
>> > ---
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "aroma.affymetrix" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to aroma-affymetr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> When reporting problems on aroma.affymetrix, make sure 1) to run the latest
> version of the package, 2) to report the output of sessionInfo() and
> traceback(), and 3) to post a complete code example.
>
>
> You received this message because you are subscribed to the Google Groups
> "aroma.affymetrix" group with website http://www.aroma-project.org/.
> To post to this group, send email to aroma-affymetrix@googlegroups.com
> To unsubscribe and other options, go to http://www.aroma-project.org/forum/
>
> ---
> You received this message because you are subscribed to the Google Groups
> "aroma.affymetrix" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to aroma-affymetrix+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
--
--
When reporting problems on aroma.affymetrix, make sure 1) to run the latest
version of the package, 2) to report the output of sessionInfo() and
traceback(), and 3) to post a complete code example.
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group with website http://www.aroma-project.org/.
To post to this group, send email to aroma-affymetrix@googlegroups.com
To unsubscribe and other options, go to http://www.aroma-project.org/forum/
---
You received this message because you are subscribed to the Google Groups
"aroma.affymetrix" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to aroma-affymetrix+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
