Re: [Rd] Compression (really about LazyDate)
Thank you Brian. I had not quite grasped how the process works, now the descriptions and usage make sense. Terry On 2/19/21 4:28 AM, Prof Brian Ripley wrote: > On 18/02/2021 18:30, Therneau, Terry M., Ph.D. via R-devel wrote: >> This is a CRAN question: >> >> I have taken care to compress files in the data directory using "xz" (and >> checked that it >> is the best). Is there then any impact or use for the LazyDataCompression >> option in the >> DESCRIPTION file? >> > > I have difficulty comprehending that, so I will try to answer my guess at > what you meant > to ask. > > What LazyDataCompression does is completely separate from the contents of the > data > directory. As the manual say > > > Some packages using ‘LazyData’ will benefit from using a form of compression > other than > gzip in the installed lazy-loading database. This can be selected by the > --data-compress > option to R CMD INSTALL or by using the ‘LazyDataCompression’ field in the > DESCRIPTION > file. Useful values are bzip2, xz and the default, gzip. The only way to > discover which > is best is to try them all and look at the size of the pkgname/data/Rdata.rdb > file. > > > When a package is installed with LazyData (and you neglected to tell us if > that is the > case), the datasets in the data directory are loaded (and hence > decompressed), and > stored in a database. For a LazyData package the compression used in the data > directory > only affects the source package size (I guess your criterion for 'best') and > how fast it > is installed (rarely a consideration but there have been LazyData packages > where > installing the data takes most of the time). At run-time only the > compression specified > by LazyDataCompression is relevant. > [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Compression (really about LazyDate)
On 18/02/2021 18:30, Therneau, Terry M., Ph.D. via R-devel wrote: This is a CRAN question: I have taken care to compress files in the data directory using "xz" (and checked that it is the best). Is there then any impact or use for the LazyDataCompression option in the DESCRIPTION file? I have difficulty comprehending that, so I will try to answer my guess at what you meant to ask. What LazyDataCompression does is completely separate from the contents of the data directory. As the manual say Some packages using ‘LazyData’ will benefit from using a form of compression other than gzip in the installed lazy-loading database. This can be selected by the --data-compress option to R CMD INSTALL or by using the ‘LazyDataCompression’ field in the DESCRIPTION file. Useful values are bzip2, xz and the default, gzip. The only way to discover which is best is to try them all and look at the size of the pkgname/data/Rdata.rdb file. When a package is installed with LazyData (and you neglected to tell us if that is the case), the datasets in the data directory are loaded (and hence decompressed), and stored in a database. For a LazyData package the compression used in the data directory only affects the source package size (I guess your criterion for 'best') and how fast it is installed (rarely a consideration but there have been LazyData packages where installing the data takes most of the time). At run-time only the compression specified by LazyDataCompression is relevant. -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Compression
This is a CRAN question: I have taken care to compress files in the data directory using "xz" (and checked that it is the best). Is there then any impact or use for the LazyDataCompression option in the DESCRIPTION file? -- Terry M Therneau, PhD Department of Health Science Research Mayo Clinic thern...@mayo.edu "TERR-ree THUR-noh" [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Compression of largish expression array files in the DAAGbio/inst/doc directory?
As far as I can see read.maimages is built on top of R's own file-reading facilties, and they all read compressed (but not zipped) files as from R 2.10.0. So simply use gzip -9 coral55?.spot and rename the files back to *.spot. If you need more compression, use xz -9e. (You can also do this in R: readLines() on the file, writeLines() using gzfile or xzfile.) You will need to make the package 'Depends: R (= 2.10)'. On Sat, 9 Apr 2011, John Maindonald wrote: The inst/doc directory of the DAAG package has 6 files coral551.spot, ... that are around 0.85 MB each. It would be useful to be able to zip then, but that as matters stand interferes with the use of the Sweave file that uses them to demonstrate input of expression array data that is in the spot format. They do not automatically get unzipped when required. I have checked that read.maimages (in limma) does not, unless I have missed something, have an option for reading zipped files. Is there any way to get around this without substantially complicating the exposition in marray-notes.pdf (also in the inst/doc subdirectory)? John Maindonald email: john.maindon...@anu.edu.au phone : +61 2 (6125)3473fax : +61 2(6125)5549 Centre for Mathematics Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200. http://www.maths.anu.edu.au/~johnm __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] Compression of largish expression array files in the DAAGbio/inst/doc directory?
Thanks. That seems to work. John Maindonald email: john.maindon...@anu.edu.au phone : +61 2 (6125)3473fax : +61 2(6125)5549 Centre for Mathematics Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200. http://www.maths.anu.edu.au/~johnm On 09/04/2011, at 4:58 PM, Prof Brian Ripley wrote: As far as I can see read.maimages is built on top of R's own file-reading facilties, and they all read compressed (but not zipped) files as from R 2.10.0. So simply use gzip -9 coral55?.spot and rename the files back to *.spot. If you need more compression, use xz -9e. (You can also do this in R: readLines() on the file, writeLines() using gzfile or xzfile.) You will need to make the package 'Depends: R (= 2.10)'. On Sat, 9 Apr 2011, John Maindonald wrote: The inst/doc directory of the DAAG package has 6 files coral551.spot, ... that are around 0.85 MB each. It would be useful to be able to zip then, but that as matters stand interferes with the use of the Sweave file that uses them to demonstrate input of expression array data that is in the spot format. They do not automatically get unzipped when required. I have checked that read.maimages (in limma) does not, unless I have missed something, have an option for reading zipped files. Is there any way to get around this without substantially complicating the exposition in marray-notes.pdf (also in the inst/doc subdirectory)? John Maindonald email: john.maindon...@anu.edu.au phone : +61 2 (6125)3473fax : +61 2(6125)5549 Centre for Mathematics Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200. http://www.maths.anu.edu.au/~johnm __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] Compression of largish expression array files in the DAAGbio/inst/doc directory?
The inst/doc directory of the DAAG package has 6 files coral551.spot, ... that are around 0.85 MB each. It would be useful to be able to zip then, but that as matters stand interferes with the use of the Sweave file that uses them to demonstrate input of expression array data that is in the spot format. They do not automatically get unzipped when required. I have checked that read.maimages (in limma) does not, unless I have missed something, have an option for reading zipped files. Is there any way to get around this without substantially complicating the exposition in marray-notes.pdf (also in the inst/doc subdirectory)? John Maindonald email: john.maindon...@anu.edu.au phone : +61 2 (6125)3473fax : +61 2(6125)5549 Centre for Mathematics Its Applications, Room 1194, John Dedman Mathematical Sciences Building (Building 27) Australian National University, Canberra ACT 0200. http://www.maths.anu.edu.au/~johnm __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
