Hi Jin, The model fitting problem would seem small enough for most computers and methods. You could do the model prediction with the 'raster' package that was designed to be able to handle very large datasets. See ?raster::predict and raster::interpolate. Perhaps also consult the 'sdm' vignette for the 'dismo' package on R-Forge; it has examples (in a species distribution modeling context) for the same methods you mention.
Best, Robert On Mon, Jul 19, 2010 at 4:53 PM, <jin...@ga.gov.au> wrote: > Dear All, > A group of us here need to make spatial predictions of environmental > variables using methods in gstat, SVM in e1071, randomForest in randomForest > in R. The dataset for model development may contain up to 80 variables with > up to 15,000 rows and the dataset for prediction may contain up to 80 > variables with up to 1 billion rows. The figures given here are extremes we > are expecting, and of course we can reduce the variables for some methods but > for some we need to use all variables. It seems we have two solutions at the > moment: 1) run the modelling work on a cluster of computers and 2) run 64-bit > R on a 64-bit computer for Windows with a big RAM. The questions are: can we > do parallel computing using the packages listed if we choose option one? Or > how big RAM do we need if we go option two? Any suggestions and comments are > appreciated. Thanks in advance. > Cheers, > Jin > ____________________________________ > Jin Li, PhD > Spatial Modeller/Computational Statistician > Marine & Coastal Environment > Geoscience Australia > GPO Box 378, Canberra, ACT 2601, Australia > Ph: 61 (02) 6249 9899; email: jin...@ga.gov.au<mailto:jin...@ga.gov.au> > _______________________________________ > > > > [[alternative HTML version deleted]] > > _______________________________________________ > R-sig-Geo mailing list > R-sig-Geo@stat.math.ethz.ch > https://stat.ethz.ch/mailman/listinfo/r-sig-geo > _______________________________________________ R-sig-Geo mailing list R-sig-Geo@stat.math.ethz.ch https://stat.ethz.ch/mailman/listinfo/r-sig-geo
