Without looking at your code, I'd guess you have a type problem, and the allocation is happening because Julia is boxing a variable somewhere.
--Tim On Wednesday, April 02, 2014 12:27:16 AM Freddy Chua wrote: > There's a function here where the loop takes place.. > > https://github.com/JuliaLang/julia/issues/6357#issuecomment-39222296 > > I don't really allocate anything in the loop.. > > Freddy Chua > > On Wed, Apr 2, 2014 at 12:24 AM, Stefan Karpinski <[email protected]>wrote: > > You still haven't shown any code that actually allocates anything, so it's > > pretty hard to say why or how that's happening. > > > > On Tue, Apr 1, 2014 at 11:55 AM, Freddy Chua <[email protected]> wrote: > >> Strange, although my for loop does not create any additional memory, the > >> memory usage increases to 60GB after turning off GC... > >> > >> On Tuesday, April 1, 2014 11:44:06 PM UTC+8, Freddy Chua wrote: > >>> Alright, these are my timings are disabling gc > >>> > >>> before disabling gc > >>> each for loop takes 911.240040 > >>> after disabling gc > >>> each for loop takes 30.351131 > >>> > >>> around 30x improvement.... and if my loop run 10 times, it would have > >>> been a 300x improvement... > >>> > >>> I hope julia devs do consider improving the GC invocation... > >>> > >>> On Tuesday, April 1, 2014 11:28:44 PM UTC+8, Freddy Chua wrote: > >>>> Alright, I am pretty certain that > >>>> > >>>> macro nogc(ex) > >>>> > >>>> quote > >>>> > >>>> try > >>>> > >>>> gc_disable() > >>>> local val = $(esc(ex)) > >>>> > >>>> finally > >>>> > >>>> gc_enable() > >>>> > >>>> end > >>>> val > >>>> > >>>> end > >>>> > >>>> end > >>>> > >>>> Does the trick... > >>>> > >>>> My program does an iterative "gradient descent", a kind of mathematical > >>>> optimisation algorithm. So I loop through a for loop multiple times. In > >>>> each loop, nothing gets created or destroyed, so GC is not needed at > >>>> all. > >>>> It turns out that turning off GC improves the performance > >>>> significantly, > >>>> probably 100x. This issue is serious, I wonder can there be a better > >>>> way of > >>>> determining when to call GC. I guessed disabling GC manually is not the > >>>> intention of the compiler designers.. > >>>> > >>>> On Tuesday, April 1, 2014 11:12:07 PM UTC+8, Freddy Chua wrote: > >>>>> I found this, https://groups.google.com/forum/#!searchin/julia- > >>>>> users/garbage/julia-users/6_XvoLBzN60/EHCrT46tIQYJ > >>>>> > >>>>> Might try to turn off GC and see whether performance improves, will > >>>>> update here later... > >>>>> > >>>>> On Tuesday, April 1, 2014 11:07:03 PM UTC+8, Stefan Karpinski wrote: > >>>>>> This code doesn't seem to create a List, Nodes or insert them into a > >>>>>> Dict - it just walks over a preexisting linked list. > >>>>>> > >>>>>> On Tue, Apr 1, 2014 at 10:59 AM, Freddy Chua <[email protected]>wrote: > >>>>>>> A possible flaw I have is the circular dependency in the data > >>>>>>> structures between Bus_Stop and Edge.
