The output from gc() indicates that you had a maximum usage of
476MB+119MB=~600MB. If you look at the output of ps you will notice
that the process size is 523MB (or about 500MB if you want to be
exact). So you are using about 25% of the 2GB that you have
available.
mem.limit just shows the current value of the parameters, and as the
help file says:
"Value
mem.limits() returns an integer vector giving the current settings of
the maxima, possibly NA."
On 7/18/07, zhihua li <[EMAIL PROTECTED]> wrote:
Thanks for replying!
i don't think i'm paging. i tried to use a smaller version of my matrix and
do all the checkings as suggested by jim. The smaller matrix caused another
problem, for which I've opened another thread. But i've found something
about memory that I don't understand.
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 269577 14.4 5570995 297.6 8919855 476.4
Vcells 3353395 25.6 9493567 72.5 15666095 119.6
Does this mean the maximum memory I can use for variables is only 120 M?
However, when I tried to check the memory limits:
> mem.limits()
nsize vsize
NA NA
Here it seems the maximum memory is not limited?
When there is no R function is being executed, I checked the system process
by:
ps u
PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
7821 0.0 0.1 10048 2336 pts/0 Ss Jul18 0:00 -bash
8076 2.9 24.5 523088 504004 pts/0 S+ Jul18 2:46 /usr/lib64/R/bi
8918 1.5 0.1 9912 2328 pts/1 Ss 00:44 0:00 -bash
8962 0.0 0.0 3808 868 pts/1 R+ 00:45 0:00 ps u
Does this mean R is using 25% of my memory? But my RAM is 2 GB and the
objects in R only occupy 40 MB from gc().
Did I interpret it wrong?
Thanks a lot!
>From: "jim holtman" <[EMAIL PROTECTED]>
>To: "zhihua li" <[EMAIL PROTECTED]>
>CC: r-help@stat.math.ethz.ch
>Subject: Re: [R] memory error with 64-bit R in linux
>Date: Wed, 18 Jul 2007 17:50:31 -0500
>
>Are you paging? That might explain the long run times. How much
>space
>are your other objects taking up? The matrix by itself should only
>require about 13MB if it is numeric. I would guess it is some of
>the
>other objects that you have in your working space. Put some gc() in
>your loop to see how much space is being used. Run it with a subset
>of the data and see how long it takes. This might give you an
>estimate of the time, and space, that might be needed for the entire
>dataset.
>
>Do a 'ps' to see how much memory your process is using. Do one
>every
>couple of minutes to see if it is growing. You can alway use
>Rprof()
>to get an idea of where time is being spent (use it on a small
>subset).
>
>On 7/18/07, zhihua li <[EMAIL PROTECTED]> wrote:
>>Hi netters,
>>
>>I'm using the 64-bit R-2.5.0 on a x86-64 cpu, with an RAM of 2 GB.
>>The
>>operating system is SUSE 10.
>>The system information is:
>>-uname -a
>>Linux someone 2.6.13-15.15-smp #1 SMP Mon Feb 26 14:11:33 UTC 2007
>>x86_64
>>x86_64 x86_64 GNU/Linux
>>
>>I used heatmap to process a matrix of the dim [16000,100]. After 3
>>hours
>>of desperating waiting, R told me:
>>cannot allocate vector of size 896 MB.
>>
>>I know the matrix is very big, but since I have 2 GB of RAM and in
>>a 64-bit
>>system, there should be no problem to deal with a vector smaller
>>than 1 GB?
>>(I was not running any other applications in my system)
>>
>>Does anyone know what's going on? Is there a hardware limit where
>>I have
>>to add more RAM, or is there some way to resolve it softwarely?
>>Also is it
>>possible to speed up the computing (I don't wanna wait another 3
>>hours to
>>know I get another error message)
>>
>>Thank you in advance!
>>
>>_________________________________________________________________
>>享用世界上最大的电子邮件系统― MSN Hotmail。
>>http://www.hotmail.com
>>
>>
>>______________________________________________
>>R-help@stat.math.ethz.ch mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide
>>http://www.R-project.org/posting-guide.html
>>and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
>
>--
>Jim Holtman
>Cincinnati, OH
>+1 513 646 9390
>
>What is the problem you are trying to solve?
_________________________________________________________________
与联机的朋友进行交流，请使用 MSN Messenger: http://messenger.msn.com/cn
--
Jim Holtman
Cincinnati, OH
+1 513 646 9390
What is the problem you are trying to solve?
______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
