Strange, although my for loop does not create any additional memory, the memory usage increases to 60GB after turning off GC...
On Tuesday, April 1, 2014 11:44:06 PM UTC+8, Freddy Chua wrote: > > Alright, these are my timings are disabling gc > > before disabling gc > each for loop takes 911.240040 > after disabling gc > each for loop takes 30.351131 > > around 30x improvement.... and if my loop run 10 times, it would have been > a 300x improvement... > > I hope julia devs do consider improving the GC invocation... > > On Tuesday, April 1, 2014 11:28:44 PM UTC+8, Freddy Chua wrote: >> >> Alright, I am pretty certain that >> >> macro nogc(ex) >> quote >> try >> gc_disable() >> local val = $(esc(ex)) >> finally >> gc_enable() >> end >> val >> end >> end >> >> Does the trick... >> >> My program does an iterative "gradient descent", a kind of mathematical >> optimisation algorithm. So I loop through a for loop multiple times. In >> each loop, nothing gets created or destroyed, so GC is not needed at all. >> It turns out that turning off GC improves the performance significantly, >> probably 100x. This issue is serious, I wonder can there be a better way of >> determining when to call GC. I guessed disabling GC manually is not the >> intention of the compiler designers.. >> >> On Tuesday, April 1, 2014 11:12:07 PM UTC+8, Freddy Chua wrote: >>> >>> I found this, >>> https://groups.google.com/forum/#!searchin/julia-users/garbage/julia-users/6_XvoLBzN60/EHCrT46tIQYJ >>> >>> Might try to turn off GC and see whether performance improves, will >>> update here later... >>> >>> On Tuesday, April 1, 2014 11:07:03 PM UTC+8, Stefan Karpinski wrote: >>>> >>>> This code doesn't seem to create a List, Nodes or insert them into a >>>> Dict – it just walks over a preexisting linked list. >>>> >>>> >>>> On Tue, Apr 1, 2014 at 10:59 AM, Freddy Chua <[email protected]> wrote: >>>> >>>>> A possible flaw I have is the circular dependency in the data >>>>> structures between Bus_Stop and Edge. >>>>> >>>> >>>>
