> [snipped] Or maybe some good links > about memory and > garbage collection.
As is mentioned time-to-time on this list when the above subject comes up, Windows memory is a complicated topic. One open-source utility I have found helpful to monitor memory when I work under XP is called RAMpage, authored by John Fitzgibbon, and is available at http://www.jfitz.com/software/RAMpage/ In its FAQ / Help, it touches on a lot of general memory and resource issues, which I found helpful to learn about. http://www.jfitz.com/software/RAMpage/RAMpage_FAQS.html (Though the author clearly warns that its usefulness for "freeing memory" may not be anymore than cosmetic on NT / 2000 / XP systems.) Hope that helps. Bill ---------------------------------------- Bill Pikounis, Ph.D. Biometrics Research Department Merck Research Laboratories PO Box 2000, MailDrop RY33-300 126 E. Lincoln Avenue Rahway, New Jersey 07065-0900 USA Phone: 732 594 3913 Fax: 732 594 1565 > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of > [EMAIL PROTECTED] > Sent: Tuesday, December 09, 2003 12:09 PM > To: [EMAIL PROTECTED] > Subject: Re: [R] Windows Memory Issues > > > I would also like some clarification about R memory > management. Like Doug, > I didn't find anything about consecutive calls to gc() to > free more memory. > We run into memory limit problems every now and then and a better > understanding of R's memory management would go a long way. > I am interested > in learning more and was wondering if there is any specific R > documentation > that explains R's memory usage? Or maybe some good links > about memory and > garbage collection. Thanks. > > Benjamin Stabler > Transportation Planning Analysis Unit > Oregon Department of Transportation > 555 13th Street NE, Suite 2 > Salem, OR 97301 Ph: 503-986-4104 > > ------------------------------------------- > > Message: 21 > Date: Mon, 8 Dec 2003 09:51:12 -0800 (PST) > From: Douglas Grove <[EMAIL PROTECTED]> > Subject: Re: [R] Windows Memory Issues > To: Prof Brian Ripley <[EMAIL PROTECTED]> > Cc: [EMAIL PROTECTED] > Message-ID: > <[EMAIL PROTECTED]> > Content-Type: TEXT/PLAIN; charset=US-ASCII > > On Sat, 6 Dec 2003, Prof Brian Ripley wrote: > > > I think you misunderstand how R uses memory. gc() does not > free up all > > the memory used for the objects it frees, and repeated > calls will free > > more. Don't speculate about how memory management works: do your > > homework! > > Are you saying that consecutive calls to gc() will free more > memory than > a single call, or am I misunderstanding? Reading ?gc and > ?Memory I don't > see anything about this mentioned. Where should I be looking to find > more comprehensive info on R's memory management?? I'm not > writing any > packages, just would like to have a better handle on efficiently using > memory as it is usually the limiting factor with R. FYI, I'm running > R1.8.1 and RedHat9 on a P4 with 2GB of RAM in case there is > any platform > specific info that may be applicable. > > Thanks, > > Doug Grove > Statistical Research Associate > Fred Hutchinson Cancer Research Center > > > > In any case, you are using an outdated version of R, and your first > > course of action should be to compile up R-devel and try > that, as there > > has been improvements to memory management under Windows. > You could also > > try compiling using the native malloc (and that *is* > described in the > > INSTALL file) as that has different compromises. > > > > > > On Sat, 6 Dec 2003, Richard Pugh wrote: > > > > > Hi all, > > > > > > I am currently building an application based on R 1.7.1 > (+ compiled > > > C/C++ code + MySql + VB). I am building this application > to work on 2 > > > different platforms (Windows XP Professional (500mb > memory) and Windows > > > NT 4.0 with service pack 6 (1gb memory)). This is a very memory > > > intensive application performing sophisticated operations > on "large" > > > matrices (typically 5000x1500 matrices). > > > > > > I have run into some issues regarding the way R handles > its memory, > > > especially on NT. In particular, R does not seem able to > recollect some > > > of the memory used following the creation and > manipulation of large data > > > objects. For example, I have a function which receives a (large) > > > numeric matrix, matches against more data (maybe imported > from MySql) > > > and returns a large list structure for further analysis. > A typical call > > > may look like this . > > > > > > > myInputData <- matrix(sample(1:100, 7500000, T), nrow=5000) > > > > myPortfolio <- createPortfolio(myInputData) > > > > > > It seems I can only repeat this code process 2/3 times > before I have to > > > restart R (to get the memory back). I use the same object names > > > (myInputData and myPortfolio) each time, so I am not > create more large > > > objects .. > > > > > > I think the problems I have are illustrated with the > following example > > > from a small R session . > > > > > > > # Memory usage for Rui process = 19,800 > > > > testData <- matrix(rnorm(10000000), 1000) # Create big matrix > > > > # Memory usage for Rgui process = 254,550k > > > > rm(testData) > > > > # Memory usage for Rgui process = 254,550k > > > > gc() > > > used (Mb) gc trigger (Mb) > > > Ncells 369277 9.9 667722 17.9 > > > Vcells 87650 0.7 24286664 185.3 > > > > # Memory usage for Rgui process = 20,200k > > > > > > In the above code, R cannot recollect all memory used, so > the memory > > > usage increases from 19.8k to 20.2. However, the > following example is > > > more typical of the environments I use . > > > > > > > # Memory 128,100k > > > > myTestData <- matrix(rnorm(10000000), 1000) > > > > # Memory 357,272k > > > > rm(myTestData) > > > > # Memory 357,272k > > > > gc() > > > used (Mb) gc trigger (Mb) > > > Ncells 478197 12.8 818163 21.9 > > > Vcells 9309525 71.1 31670210 241.7 > > > > # Memory 279,152k > > > > > > Here, the memory usage increases from 128.1k to 279.1k > > > > > > Could anyone point out what I could do to rectify this > (if anything), or > > > generally what strategy I could take to improve this? > > > > > > Many thanks, > > > Rich. > > > > > > Mango Solutions > > > Tel : (01628) 418134 > > > Mob : (07967) 808091 > > > > > > > > > [[alternative HTML version deleted]] > > > > > > ______________________________________________ > > > [EMAIL PROTECTED] mailing list > > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > > > > > > > > > -- > > Brian D. Ripley, [EMAIL PROTECTED] > > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > > University of Oxford, Tel: +44 1865 272861 (self) > > 1 South Parks Road, +44 1865 272866 (PA) > > Oxford OX1 3TG, UK Fax: +44 1865 272595 > > > > ______________________________________________ > > [EMAIL PROTECTED] mailing list > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > > > ______________________________________________ > [EMAIL PROTECTED] mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help