> On Tue, 14 Jun 2005, Prof Brian Ripley wrote: > If your file system does not like 15000 files you can always > save in a DBMS.
Or, switch to a better/more appropriate file system: http://en.wikipedia.org/wiki/Comparison_of_file_systems ReiserFS would allow you to store up to about 1.2 million files in a directory. > -----Original Message----- > From: Prof Brian Ripley [mailto:[EMAIL PROTECTED] > Sent: Tuesday, June 14, 2005 10:41 AM > To: Barry Rowlingson > Cc: [email protected]; Richard Mott > Subject: Re: [R] load ing and saving R objects > > > On Tue, 14 Jun 2005, Barry Rowlingson wrote: > > > Richard Mott wrote: > >> Does anyone know a way to do the following: > >> > >> Save a large number of R objects to a file (like load() > does) but then > >> read back only a small named subset of them . As far as I can see, > >> load() reads back everything. > > > > Save them to individual files when you generate them? > > > > for(i in 1:15000){ > > > > m=generateBigMatrix(i) > > > > filename=paste("BigMatrix-",i,".Rdata",sep='') > > save(m,file=filename) > > } > > > > Note that load will always overwrite 'm', so to load a > sample of them in > > you'll need to do something like this: > > > > bigSamples=list() > > > > for(i in sample(15000,N)){ > > filename=paste("BigMatrix-",i,".Rdata",sep='') > > load(filename) > > bigSamples[[i]]=m > > } > > > > But there may be a more efficient way to string up a big list like > > that, I can never remember - get it working, then worry > about optimisation. > > (Yes, use bigSamples <- vector("list", 15000) first.) > > > I hope your filesystem is happy with 15000 objects in it. I would > > dedicate a folder or directory for just these objects' > files, since it > > then becomes near impossible to see anything other than the > big matrix > > files... > > .readRDS/.saveRDS might be a better way to do this, and avoids always > restoring to "m". > > If your file system does not like 15000 files you can always > save in a > DBMS. > > I did once look into restoring just some of the objects in a save()ed > file, but it is not really possible to do so efficiently due > to sharing > between objects. > > -- > Brian D. Ripley, [EMAIL PROTECTED] > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272866 (PA) > Oxford OX1 3TG, UK Fax: +44 1865 272595 > > ______________________________________________ > [email protected] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
