That seems most likely.
On Tue, Apr 1, 2014 at 2:35 PM, Tim Holy <[email protected]> wrote: > Without looking at your code, I'd guess you have a type problem, and the > allocation is happening because Julia is boxing a variable somewhere. > > --Tim > > On Wednesday, April 02, 2014 12:27:16 AM Freddy Chua wrote: > > There's a function here where the loop takes place.. > > > > https://github.com/JuliaLang/julia/issues/6357#issuecomment-39222296 > > > > I don't really allocate anything in the loop.. > > > > Freddy Chua > > > > On Wed, Apr 2, 2014 at 12:24 AM, Stefan Karpinski > <[email protected]>wrote: > > > You still haven't shown any code that actually allocates anything, so > it's > > > pretty hard to say why or how that's happening. > > > > > > On Tue, Apr 1, 2014 at 11:55 AM, Freddy Chua <[email protected]> > wrote: > > >> Strange, although my for loop does not create any additional memory, > the > > >> memory usage increases to 60GB after turning off GC... > > >> > > >> On Tuesday, April 1, 2014 11:44:06 PM UTC+8, Freddy Chua wrote: > > >>> Alright, these are my timings are disabling gc > > >>> > > >>> before disabling gc > > >>> each for loop takes 911.240040 > > >>> after disabling gc > > >>> each for loop takes 30.351131 > > >>> > > >>> around 30x improvement.... and if my loop run 10 times, it would have > > >>> been a 300x improvement... > > >>> > > >>> I hope julia devs do consider improving the GC invocation... > > >>> > > >>> On Tuesday, April 1, 2014 11:28:44 PM UTC+8, Freddy Chua wrote: > > >>>> Alright, I am pretty certain that > > >>>> > > >>>> macro nogc(ex) > > >>>> > > >>>> quote > > >>>> > > >>>> try > > >>>> > > >>>> gc_disable() > > >>>> local val = $(esc(ex)) > > >>>> > > >>>> finally > > >>>> > > >>>> gc_enable() > > >>>> > > >>>> end > > >>>> val > > >>>> > > >>>> end > > >>>> > > >>>> end > > >>>> > > >>>> Does the trick... > > >>>> > > >>>> My program does an iterative "gradient descent", a kind of > mathematical > > >>>> optimisation algorithm. So I loop through a for loop multiple > times. In > > >>>> each loop, nothing gets created or destroyed, so GC is not needed at > > >>>> all. > > >>>> It turns out that turning off GC improves the performance > > >>>> significantly, > > >>>> probably 100x. This issue is serious, I wonder can there be a better > > >>>> way of > > >>>> determining when to call GC. I guessed disabling GC manually is not > the > > >>>> intention of the compiler designers.. > > >>>> > > >>>> On Tuesday, April 1, 2014 11:12:07 PM UTC+8, Freddy Chua wrote: > > >>>>> I found this, https://groups.google.com/forum/#!searchin/julia- > > >>>>> users/garbage/julia-users/6_XvoLBzN60/EHCrT46tIQYJ > > >>>>> > > >>>>> Might try to turn off GC and see whether performance improves, will > > >>>>> update here later... > > >>>>> > > >>>>> On Tuesday, April 1, 2014 11:07:03 PM UTC+8, Stefan Karpinski > wrote: > > >>>>>> This code doesn't seem to create a List, Nodes or insert them > into a > > >>>>>> Dict - it just walks over a preexisting linked list. > > >>>>>> > > >>>>>> On Tue, Apr 1, 2014 at 10:59 AM, Freddy Chua > <[email protected]>wrote: > > >>>>>>> A possible flaw I have is the circular dependency in the data > > >>>>>>> structures between Bus_Stop and Edge. >
