Make good use of Rprof(): It has helped me a great deal in pinpointing bottlenecks where I would not have suspected.
Cheers, Andy From: Weiwei Shi > object.size(intersect.matrix) > 41314204 > > but my machine has 4 G memory, so it should be ok since after > 12 hours, it finishes 16k out of 60k but still slow non-linearly. > > I am thinking to chop 60k into multiple 5k data.frames to run > the program. but just wondering is there a way around it? > > > version > _ > platform i686-pc-linux-gnu > arch i686 > os linux-gnu > system i686, linux-gnu > status > major 2 > minor 3.1 > year 2006 > month 06 > day 01 > svn rev 38247 > language R > version.string Version 2.3.1 (2006-06-01) > > [EMAIL PROTECTED] ox]$ more /proc/meminfo > total: used: free: shared: buffers: cached: > Mem: 4189724672 3035549696 1154174976 0 282836992 2057129984 > Swap: 4293586944 645042176 3648544768 > > [EMAIL PROTECTED] ox]$ more /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 15 > model : 4 > model name : Intel(R) Xeon(TM) CPU 3.60GHz > stepping : 3 > cpu MHz : 3591.419 > cache size : 2048 KB > > > > thanks. > > On 10/25/06, Weiwei Shi <[EMAIL PROTECTED]> wrote: > > Hi, > > I have a series of lda analysis using the following lapply function: > > > > n <- dim(intersect.matrix)[1] > > net1.lda <- lapply(1:(n), function(k) i.lda(data.list, > > intersect.matrix, i=k, w)) > > > > i.lda is function to do the real lda analysis. > > > > intersect.matrix is a nx1026 matrix, n can be a really huge number > > like 60k. The target is perform a random search. Building a n=120k > > matrix is impossible for my machine. When n=5k, the task > can be done > > in 30 min while n=60k, it is estimated to take 5 days. So I am > > wondering where my coding problem is, which causes this to be a > > nonlinearity. > > > > If more info is needed, I will provide. > > > > thanks > > > > -- > > Weiwei Shi, Ph.D > > Research Scientist > > GeneGO, Inc. > > > > "Did you always know?" > > "No, I did not. But I believed..." > > ---Matrix III > > > > > -- > Weiwei Shi, Ph.D > Research Scientist > GeneGO, Inc. > > "Did you always know?" > "No, I did not. But I believed..." > ---Matrix III > > ______________________________________________ > [email protected] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > > > ------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments,...{{dropped}} ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
