The first thing I would do, especially before letting it run for 48 hours, is to take a small subset of the data and run it with the profiler (Rprof) enable to see where time is being spent. I would also put some print statements in the main loop to periodically output the amount of CPU and memory being used. It is hard to look through the code without a subset of the data. So run the profiler and tell us what the output looks like. You may be calling a function that is not part of your code that is taking all the time, but until you run the profiler, it is hard to tell.
On Fri, Sep 16, 2011 at 4:45 PM, Simon Zehnder <[email protected]> wrote: > Hi guyz, > > I have serious problems with an algorithm I let run on a supercomputer: > > You find the functions under the following URLs: > > simuFunctionCaller: http://pastebin.com/6gw2fJFb > > calls Function simuFunctionNBM (http://pastebin.com/QeJDUnqx) > > after reading a csv-file ordered like the following: > > ,name,theta,std_error_theta,theta_phi,std_error_theta_phi,phi,N,sigma_epsilon,sigma_eta > 31,25%,0.003044,2.28E-09,0.00742575,4.71E-09,0.00452525,68164.75,0.008706072,0.00774016 > 32,50%,0.004536,3.22E-09,0.01045,7.94E-09,0.0052762,110150,0.012387386,0.011169953 > 33,75%,0.007356625,1.24E-08,0.014249075,3.16E-08,0.007518,178840.5,0.018471461,0.016058424 > > Reading and running without an exception is no problem. But the algorithm > needs hours to run and seems to use more memory than available: > > I have given a time limit of 48 h and a memory limit of 128 GB RAM on the > supercomputer (it is a cluster, a BATCH System)…..nevertheless the computer > shuts down the algorithm after a certain time because of memory problems. > > Does anyone of u guys see a problem in the algorithm? Especially with so much > RAM and time available? > > I am very thankful for your suggestions! > > Simon > ______________________________________________ > [email protected] mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > -- Jim Holtman Data Munger Guru What is the problem that you are trying to solve? ______________________________________________ [email protected] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
