On Sat, 26 Feb 2005, Nawaaz Ahmed wrote:
Hi Folks,
Thanks for all your replies and input. In particular, thanks Luke, for explaining what is happening under the covers. In retrospect, my example using save and load to demonstrate the problem I was having was a mistake - I was trying to reproduce the problem I was having in a simple enough way and I thought save and load were showing the same problem (i.e. an extra copy was being made). After carefully examining my gc() traces,
I've come to realize that while there are copies being made, there is nothing unexpected about it - the failure to allocate memory is really because R is hitting the 3GB address limit imposed by my linux box during processing. So as Luke suggests, maybe 32 bits is not the right platform for handling large data in R.
On the other hand, I think the problem can be somewhat alleviated (though not eliminated) if we did garbage collection of temporary variables immediately so that we can reduce the memory footprint and the fragmentation problem that malloc() is going to be faced with (gctorture() is probably too extreme :-). Most of the problems that I am having are in the coercion routines which do create temporary copies. So in code of the form x = as.vector(x), it would be nice if the old value of x was garbage collected (i.e. if there were no references to it)
R does not reference count, so the `i.e.' is not possible AFAIK.
What do you want to achieve via as.vector() there? If something like
dim(x) <- NULL
will do the job, there is an optimization that alters the original copy rather make a copy and alter that. (I haven't checked this exact example, but I am pretty sure I have done so in the past and benefited from the optimization. It is done by only duplicating if named is 2.)
Similarly not naming intermediate copies can help so you will see things like
structure(some_transformation(x), class="y", dimnames=list(rn, NULL))
-- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
______________________________________________ R-devel@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
