I recently started using R 2.14.0 on a new machine and i am experiencing
what seems like unusually greedy memory use. It happens all the time, but
to give a specific example, let's say i run the following code
--------
for(j in 1:length(files)){
load(file.path(dump.dir, files[j]))
mat.data[[j]]<-data
}
save(abind(mat.data, along=2), file.path(dump.dir, filename))
---------
It loads parts of multidimensional matrix into a list, then binds it along
second dimension and saves on disk. Code works, although slowly, but what's
strange is the amount of memory it uses.
In particular, each chunk of data is between 50M to 100M, and altogether
the binded matrix is 1.3G. One would expect that R would use roughly double
that memory - to keep mat.data and its binded version separately, or 1G. I
could imagine that for somehow it could use 3 times the size of matrix. But
in fact it uses more than 5.5 times (almost all of my physical memory) and
i think is swapping a lot to disk . For this particular task, my top output
shows eating more than 7G of memory and using up 11G of virtual memory as
well
$top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8823 user 25 0 11g 7.2g 10m R 99.7 92.9
5:55.05
R
8590 root 15 0 154m 16m 5948 S 0.5 0.2
23:22.40 Xorg
I have strong suspicion that something is off with my R binary, i don't
think i experienced things like that in a long time. Is this in line with
what i am supposed to experience? Are there any ideas for diagnosing what
is going on?
Would appreciate any suggestions
Thanks
Andre
==================================
Here is what i am running on:
CentOS release 5.5 (Final)
> sessionInfo()
R version 2.14.0 (2011-10-31)
Platform: x86_64-unknown-linux-gnu (64-bit)
locale:
[1] en_US.UTF-8
attached base packages:
[1] stats graphics grDevices datasets utils methods base
other attached packages:
[1] abind_1.4-0 rJava_0.9-3 R.utils_1.12.1 R.oo_1.9.3
R.methodsS3_1.2.2
loaded via a namespace (and not attached):
[1] codetools_0.2-8 tcltk_2.14.0 tools_2.14.0
I compiled R configure as follows
/configure --prefix=/usr/local/R --enable-byte-compiled-packages=no
--with-tcltk --enable-R-shlib=yes
[[alternative HTML version deleted]]
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
