Dear Aaron, I am really a tool user and not a tool maker (actually an ecologist doing some biostatistics)... so, I take the liberty of sending a copy of this e-mail to the r-help list where capable computer persons and true statisticians may provide more relevant information and also to Paulo Ribeiro and Peter Diggle, the authors of geoR..
I really feel that your huge matrix cannot be handled in R that easy, and I get the same kind of error as you: > m=matrix(1,ncol=18227, nrow=18227) Error: cannot allocate vector of size 2595496 Kb In addition: Warning messages: 1: Reached total allocation of 511Mb: see help(memory.size) 2: Reached total allocation of 511Mb: see help(memory.size) However, if you want to compute a distance matrix, have a look to the function: ?dist and try it... You will not have to create the distance matrix yourself from the coordinates file (but you may meet meory problem anyway). Still more straigthfully, if you intend to use interpolation methods such as kriging, you don't need to manage the distance matrix building by yourself. See: library(geoR) library(help=geoR) ?variog ?variofit ?likfit? etc... If you want further use geostatics (eg via geoR or gstat), you will anyway have to manage with memory limits (not due to R). On my computer (portable HP compaq nx7000) I can hardly manage with more than 2000 observations using geoR (far from your 18227 observations). I had to krige on a dataset with 9000 observations recently and has been led to subsample randomly 2000 values. You can however try and increase the memory allocated to R on your computer. The size limit is hardware dependent, eg: ?memory.size ?memory.limit / memory.limit(size=500000000) / Another way may be to perform local kriging (eg kriging within the dataset on a fixed radius), but to my knowldege this cannot be done with geoR (unfortunately). The library gstat offers this option, but has much more limited possibilities than geoR considering other issues (variogram analysis, etc...). library(gstat) ?krige see argument 'maxdist' Hope this can help, Kind regards Patrick Giraudoux -- Department of Environmental Biology EA3184 usc INRA University of Franche-Comte 25030 Besancon Cedex (France) tel. +33 381 665 745 fax +33 381 665 797 http://lbe.univ-fcomte.fr Aaron Swoboda a écrit : > Dear Sir: > > > I ran across your post to the R-help archive from February 9it is > attached below since it was nearly two years ago!). I am beginning to > learn R and am interested in analyzing some of my data in a spatial > context. I am having a problem that seems similar to the problem you > encountered, trying to work with a large matrix in Windows XP). I am > wondering if you could help steer me in the direction that helped you > solve your problem. I am trying to construct a distance matrix > containing the all of the distances between my 18,000 observations. > Trying to make a matrix that large gets an error message... > > m=matrix(1,ncol=18227, nrow=18227) > > Error: cannot allocate vector of size 2595496 Kb > In addition: Warning messages: > 1: Reached total allocation of 1023Mb: see help(memory.size) > > Thank you for any help you may be able to send my way. > > ~Aaron Swoboda > > Below is your posting to the R-help list... > > > Dear all, > > I a read with great interest the e-mails related to Arnav Sheth about > memory limitation when computing a distance matrix. I suspect > that I will also meet some memory limitation using GeoR. I am > currently running GeoR on a geodata object including 2686 geographical > coordinates. > > krige.conv() can handle it (it takes 10-15 mn of computing) but > requests an increased memory. > > /> memory.limit(size=500000000) / > > When the computing is completed, the computer speed is considerably > slowed down for any application. It is thus most necessary to > shut it down and restart. I will probably have to handle a set of > 5000-6000 coordinates in once in the next few months. I wonder if > it will go through it on my plateform (Windows XP and compaq nx7000). > If not, will the limitation due to R or to Windows? Does an > alternate solution exist? > > Thanks for any hint, > > Patrick Giraudoux > >-- > Dr. Aaron M. Swoboda > 3208 Posvar Hall > Graduate School of Public and International Affairs > University of Pittsburgh > Pittsburgh, PA 15260 > > e-mail: [EMAIL PROTECTED] > Office: 412-648-7604 > Fax: 412-648-2605 > > [[alternative HTML version deleted]]
______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
