Could you try: @everywhere gc() @everywhere Base.flush_gc_msgs() @everywhere gc()
On Wed, Nov 5, 2014 at 2:07 PM, Seth Yuan <[email protected]> wrote: > Hi Amit, > > Thanks for the response, I tried Base.flush_gc_msgs(), the result is shown > below: > > > <https://lh5.googleusercontent.com/-bEVLUYu-DPk/VFnhFUh28CI/AAAAAAAABk0/WNl3ukSetUk/s1600/mem.png> > It did free some memory on some workers, but not quite. I think the > acceptable number for workers would be around 100MB (like the master is). > > So what's wrong with DArrays? > > > On Wednesday, November 5, 2014 2:32:42 PM UTC+8, Amit Murthy wrote: >> >> Distributed objects are currently gc'ed asynchronously. >> >> You may force this using an internal function flush_gc_msgs like this: >> >> @everywhere Base.flush_gc_msgs() >> @everywhere gc() >> >> >> The above is an internal function and is not meant for general use. It >> will be great if you could open an issue on github requesting a feature to >> force gc of distributed objects. >> >> >> >> On Wed, Nov 5, 2014 at 9:01 AM, Seth Yuan <[email protected]> wrote: >> >>> I did a test on deallocations, and the behavior is not what I expected. >>> >>> The code I ran is: >>> >>> @everywhere using Ops >>> >>> function test_allocation() >>> a = drand(10000, 10000) >>> b = 2a >>> readline() >>> end >>> >>> test_allocation() >>> @everywhere gc() >>> readline() >>> >>> Ops module defined the '*' operator (where a new DArray is created). I >>> can see that master's memory went down after the gc() call, but the >>> workers' memory remained unchanged. >>> >>> >>> <https://lh5.googleusercontent.com/-qo7SKwkoya0/VFmZrIWL0EI/AAAAAAAABkk/FctwkEcQWnk/s1600/mem.png> >>> So the question is: "how to really free a DArray? Both master and >>> workers!" >>> >>> >>> On Thursday, October 16, 2014 10:18:42 PM UTC+8, Amit Murthy wrote: >>>> >>>> meant to say "all references from all processes are removed" >>>> >>>> On Thu, Oct 16, 2014 at 7:47 PM, Amit Murthy <[email protected]> >>>> wrote: >>>> >>>>> Yes, it is gc'ed just like any other object, when all references to it >>>>> from any process is removed. >>>>> >>>>> On Thu, Oct 16, 2014 at 2:59 PM, Seth Yuan <[email protected]> wrote: >>>>> >>>>>> Now, I know by reading the documentation how a DArray is created, but >>>>>> I'm not sure when a DArray is deallocated. >>>>>> >>>>>> Is it GC able when the master looses all references of it? Does >>>>>> somebody know the answer to this question? >>>>>> >>>>> >>>>> >>>> >>
