saveObject() and loadObject() are using save() and load() in base. The default is to compress the data when saving, which take some time. Using saveObject(..., compress=FALSE) is probably faster, but takes up more disk space. ...and make sure you don't work towards a file system over a network, because that can slow things down (doesn't sound like you do).
/H On Sun, Nov 8, 2009 at 6:26 AM, Hao Cen <h...@andrew.cmu.edu> wrote: > Hi Henrik, > > I am using your saveObject/loadObject to handle over 1000 matrices. It > worked beautifully. Because I need to load those matrices often for > evaluating a few functions on them and those matrices do not fit all in > memory at once, is there a way to speed up the loading part? I tried save > all the binary files to /dev/shm (shared memory section in linux) but the > speed of loadObject on /dev/shm remains the same as on the disk. > > Thanks > > Hao > > > > -----Original Message----- > From: henrik.bengts...@gmail.com [mailto:henrik.bengts...@gmail.com] On > Behalf Of Henrik Bengtsson > Sent: Monday, November 02, 2009 12:34 AM > To: David Winsemius > Cc: r-help@r-project.org; jeffc > Subject: Re: [R] save an object by dynamicly created name > > On Sun, Nov 1, 2009 at 9:18 PM, David Winsemius <dwinsem...@comcast.net> > wrote: >> >> On Nov 1, 2009, at 11:28 PM, Henrik Bengtsson wrote: >> >>> On Sun, Nov 1, 2009 at 7:48 PM, David Winsemius <dwinsem...@comcast.net> >>> wrote: >>>> >>>> On Nov 1, 2009, at 10:16 PM, Henrik Bengtsson wrote: >>>> >>>>> path <- "data"; >>>>> dir.create(path); >>>>> >>>>> for (i in 1:10) { >>>>> m <- i:5; >>>>> filename <- sprintf("m%02d.Rbin", i); >>>>> pathname <- file.path(path, filename); >>>>> save(m, file=pathname); >>>>> } >>>>> >>>> >>>> That would result in each of the ten files containing an object with the >>>> same name == "m". (Also on my system R data files have type Rdta.) So I >>>> thought what was requested might have been a slight mod: >>>> >>>> path <- "~/"; >>>> dir.create(path); >>>> >>>> for (i in 1:10) { >>>> assign( paste("m", i, sep=""), i:5) >>>> filename <- sprintf("m%02d.Rdta", i) >>>> pathname <- file.path(path, filename) >>>> obj =get(paste("m", i, sep="")) >>>> save(obj, file=pathname) >>>> } >>> >>> Then a more convenient solution is to use saveObject() and >>> loadObject() of R.utils. saveObject() does not save the name of the >>> object save. >> >> The OP asked for this outcome : >> >> " I would like to save m as m1, m2, m3 ..., >> to file /home/data/m1, /home/data/m2, home/data/m3, ..." >> >> >>> If you want to save multiple objects, the wrap them up >>> in a list. >> >> I agree that a list would makes sense if it were to be stored in one file > , >> although it was not what requested. > > That comment was not for the OP, but for saveObject()/loadObject() in > general. > >> But wouldn't that require assign()-ing a name before list()-wrapping? > > Nope, the whole point of using saveObject()/loadObject() is to save > the objects/values without their names that you happens to choose in > the current session, and to avoid overwriting existing ones in your > next session. My example could also have been: > > library("R.utils"); > saveObject(list(a=1,b=LETTERS,c=Sys.time()), file="foo.Rbin"); > y <- loadObject("foo.Rbin"); > z <- loadObject("foo.Rbin"); > stopifnot(identical(y,z)); > > If you really want to attach the elements of the saved list, do: > > attachLocally(loadObject("foo.Rbin")); >> str(a) > num 1 >> str(b) > chr [1:26] "A" "B" "C" "D" "E" "F" "G" "H" "I" "J" ... >> str(c) > POSIXct[1:1], format: "2009-11-01 21:30:41" > >> >> I suppose we ought to mention that the use of assign to create a variable > is >> a FAQ ... 7.21? Yep, I have now referred to it a sufficient number of > times >> to refer to it by number. >> >> > http://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-turn-a-string-into-a- > variable_003f > > My personal take on assign() and get() is that if you find yourself > using them (at this level), there is a good chance there exists a > better solution that you should use instead. > > My $.02 > > /H > >> >> -- >> David >> >>> loadObject() does not assign variable, but instead return >>> them. Example: >>> >>> library("R.utils"); >>> x <- list(a=1,b=LETTERS,c=Sys.time()); >>> saveObject(x, file="foo.Rbin"); >>> y <- loadObject("foo.Rbin"); >>> stopifnot(identical(x,y)); >> >>> >>> So, for the original example, I'd recommend: >>> >>> library("R.utils"); >>> path <- "data"; >>> mkdirs(path); >>> >>> for (i in 1:10) { >>> m <- i:5; >>> filename <- sprintf("m%02d.Rbin", i); >>> saveObject(m, file=filename, path=path); >>> } >>> >>> and loading the objects back as: >>> >>> for (i in 1:10) { >>> filename <- sprintf("m%02d.Rbin", i); >>> m <- loadObject(filename, path=path); >>> print(m); >>> } >>> /Henrik >>> >>>> >>>> -- >>>> David. >>>> >>>>> /H >>>>> >>>>> On Sun, Nov 1, 2009 at 6:53 PM, jeffc <h...@andrew.cmu.edu> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I would like to save a few dynamically created objects to disk. The >>>>>> following is the basic flow of the code segment >>>>>> >>>>>> for(i = 1:10) { >>>>>> m = i:5 >>>>>> save(m, file = ...) ## ??? >>>>>> } >>>>>> To distinguish different objects to be saved, I would like to save m > as >>>>>> m1, >>>>>> m2, m3 ..., to file /home/data/m1, /home/data/m2, home/data/m3, ... >>>>>> >>>>>> I tried a couple of methods on translating between object names and >>>>>> strings >>>>>> (below) but couldn't get it to work. >>>>>> https://stat.ethz.ch/pipermail/r-help/2008-November/178965.html >>>>>> http://tolstoy.newcastle.edu.au/R/help/04/08/2673.html >>>>>> >>>>>> Any suggestions would be appreciated. >>>>>> >>>>>> thanks >>>>>> >>>>>> Hao >>>>>> >>>>>> -- >>>>>> View this message in context: >>>>>> >>>>>> > http://old.nabble.com/save-an-object-by-dynamicly-created-name-tp26155437p26 > 155437.html >>>>>> Sent from the R help mailing list archive at Nabble.com. >>>>>> >>>>>> ______________________________________________ >>>>>> R-help@r-project.org mailing list >>>>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>>>> PLEASE do read the posting guide >>>>>> http://www.R-project.org/posting-guide.html >>>>>> and provide commented, minimal, self-contained, reproducible code. >>>>>> >>>>> >>>>> ______________________________________________ >>>>> R-help@r-project.org mailing list >>>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>>> PLEASE do read the posting guide >>>>> http://www.R-project.org/posting-guide.html >>>>> and provide commented, minimal, self-contained, reproducible code. >>>> >>>> David Winsemius, MD >>>> Heritage Laboratories >>>> West Hartford, CT >>>> >>>> ______________________________________________ >>>> R-help@r-project.org mailing list >>>> https://stat.ethz.ch/mailman/listinfo/r-help >>>> PLEASE do read the posting guide >>>> http://www.R-project.org/posting-guide.html >>>> and provide commented, minimal, self-contained, reproducible code. >>>> >> >> David Winsemius, MD >> Heritage Laboratories >> West Hartford, CT >> >> > > > ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.