There's a function here where the loop takes place.. https://github.com/JuliaLang/julia/issues/6357#issuecomment-39222296
I don't really allocate anything in the loop.. Freddy Chua On Wed, Apr 2, 2014 at 12:24 AM, Stefan Karpinski <[email protected]>wrote: > You still haven't shown any code that actually allocates anything, so it's > pretty hard to say why or how that's happening. > > > On Tue, Apr 1, 2014 at 11:55 AM, Freddy Chua <[email protected]> wrote: > >> Strange, although my for loop does not create any additional memory, the >> memory usage increases to 60GB after turning off GC... >> >> On Tuesday, April 1, 2014 11:44:06 PM UTC+8, Freddy Chua wrote: >>> >>> Alright, these are my timings are disabling gc >>> >>> before disabling gc >>> each for loop takes 911.240040 >>> after disabling gc >>> each for loop takes 30.351131 >>> >>> around 30x improvement.... and if my loop run 10 times, it would have >>> been a 300x improvement... >>> >>> I hope julia devs do consider improving the GC invocation... >>> >>> On Tuesday, April 1, 2014 11:28:44 PM UTC+8, Freddy Chua wrote: >>>> >>>> Alright, I am pretty certain that >>>> >>>> macro nogc(ex) >>>> quote >>>> try >>>> gc_disable() >>>> local val = $(esc(ex)) >>>> finally >>>> gc_enable() >>>> end >>>> val >>>> end >>>> end >>>> >>>> Does the trick... >>>> >>>> My program does an iterative "gradient descent", a kind of mathematical >>>> optimisation algorithm. So I loop through a for loop multiple times. In >>>> each loop, nothing gets created or destroyed, so GC is not needed at all. >>>> It turns out that turning off GC improves the performance significantly, >>>> probably 100x. This issue is serious, I wonder can there be a better way of >>>> determining when to call GC. I guessed disabling GC manually is not the >>>> intention of the compiler designers.. >>>> >>>> On Tuesday, April 1, 2014 11:12:07 PM UTC+8, Freddy Chua wrote: >>>>> >>>>> I found this, https://groups.google.com/forum/#!searchin/julia- >>>>> users/garbage/julia-users/6_XvoLBzN60/EHCrT46tIQYJ >>>>> >>>>> Might try to turn off GC and see whether performance improves, will >>>>> update here later... >>>>> >>>>> On Tuesday, April 1, 2014 11:07:03 PM UTC+8, Stefan Karpinski wrote: >>>>>> >>>>>> This code doesn't seem to create a List, Nodes or insert them into a >>>>>> Dict - it just walks over a preexisting linked list. >>>>>> >>>>>> >>>>>> On Tue, Apr 1, 2014 at 10:59 AM, Freddy Chua <[email protected]>wrote: >>>>>> >>>>>>> A possible flaw I have is the circular dependency in the data >>>>>>> structures between Bus_Stop and Edge. >>>>>>> >>>>>> >>>>>> >
