[R] Memory Problems with a Simple Bootstrap - Part II

2008-08-02 Thread Tom La Bone
I have distilled my bootstrap problem down to this bit of code, which calculates an estimate of the 95th percentile of 7500 random numbers drawn from a standard normal distribution: library(boot) per95 - function( annual.data, b.index) { sample.data - annual.data[b.index]

Re: [R] Memory Problems with a Simple Bootstrap - Part II

2008-08-02 Thread Prof Brian Ripley
On Sat, 2 Aug 2008, Tom La Bone wrote: I have distilled my bootstrap problem down to this bit of code, which calculates an estimate of the 95th percentile of 7500 random numbers drawn from a standard normal distribution: library(boot) per95 - function( annual.data, b.index) { sample.data -

Re: [R] Memory Problems with a Simple Bootstrap - Part II

2008-08-02 Thread jim holtman
I was suggesting adding the gc() call to help provide some additional information on the utilization of memory. As you indicated, it probably do not help in reducing the fragmentation of memory, but it was worth a try to see if there was any additional information that might be gleaned from the

Re: [R] Memory Problems with a Simple Bootstrap - Part II

2008-08-02 Thread Prof Brian Ripley
The following version of boot:::ordinary.array will enable this to run in 300Mb: ordinary.array - function(n, R, strata) { inds - as.integer(names(table(strata))) if (length(inds) == 1) { output - sample(n, n*R, replace=TRUE) dim(output) - c(R, n) } else {

[R] Memory Problems with a Simple Bootstrap

2008-08-01 Thread Tom La Bone
I have a data file called inputdata.csv that looks something like this ID YearResult Month Date 1 71741954 103 540301 2 7174195443 540322 3 20924 1967 4 2 670223 4 20924

Re: [R] Memory Problems with a Simple Bootstrap

2008-08-01 Thread jim holtman
Use gc() in the loop to possibly free up any fragmented memory. You might also print out the size of B (object.size(B)) since that appears to be the only variable in your loop that might be growing. On Fri, Aug 1, 2008 at 12:09 PM, Tom La Bone [EMAIL PROTECTED] wrote: I have a data file

Re: [R] Memory Problems with a Simple Bootstrap

2008-08-01 Thread Tom La Bone
Same problem. The Windows Task Manager indicated that Rgui.exe was using 1,249,722 K of memory when the error occurred. This is R 2.7.1 by the way. library(boot) setwd(C:/Documents and Settings/Tom/Desktop) data.in - read.csv(inputdata.csv,header=T,as.is=T) per95 - function(

Re: [R] Memory Problems with a Simple Bootstrap

2008-08-01 Thread jim holtman
It seems like the objects are reasonable size and the memory size also seems reasonable. That is what I usually go by to see if there are large objects in my memory. If it was showing that R had 1.2GB of memory allocated to it, I wonder if there might be a memory leak somewhere. On Fri, Aug 1,

Re: [R] Memory Problems with a Simple Bootstrap

2008-08-01 Thread Tom La Bone
Here it is with the gc() in a print statement. Tom library(boot) setwd(C:/Documents and Settings/Tom/Desktop) data.in - read.csv(inputdata.csv,header=T,as.is=T) per95 - function( annual.data, b.index) { + sample.data - annual.data[b.index,] +