2.6.1 is when it should be fixed which should be sometime early next year. A better workaround might be to avoid try/except in the generator if you can refactor that out into its own function. But if that doesn't work then I also think calling close explicitly should avoid it and do so without the cost of throwing the exception.
From: [email protected] [mailto:[email protected]] On Behalf Of Idan Zaltzberg Sent: Monday, December 28, 2009 11:52 PM To: Discussion of IronPython Subject: Re: [IronPython] Memory leaks in Ipy 2.6 Thanks for the fast reply. Could you estimate when and how (in what version) the fix will be deployed? Regardless, our current fix is to "throw" exception when we want the generator to be disposed and catch it ourselves, is that the right way to go? From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Dino Viehland Sent: Monday, December 28, 2009 9:47 PM To: Discussion of IronPython Subject: Re: [IronPython] Memory leaks in Ipy 2.6 This is definitely a bug - we're leaking our stack frames on the CLR's finalizer thread when we swallow exceptions while closing the generator. I think this is the correct fix but I haven't fully tested it yet, in our generator we have the code below but it's missing the assignment of null to DynamicStackFrames: try { @throw(new GeneratorExitException()); // Generator should not have exited normally. throw new RuntimeException("generator ignored GeneratorExit"); } catch (StopIterationException) { // Ignore ExceptionHelpers.DynamicStackFrames = null; // adding this fixes it } catch (GeneratorExitException) { // Ignore ExceptionHelpers.DynamicStackFrames = null; // adding this fixes it } This definitely fixes 1 and 2. After running this 3 doesn't actually seem to leak memory when running repeatedly for me but after-before > 10 is True. I'll take a closer look and make sure this is the correct fix after the new year when I'm back in the office. From: [email protected]<mailto:[email protected]> [mailto:[email protected]<mailto:[email protected]>] On Behalf Of Idan Zaltzberg Sent: Monday, December 28, 2009 10:33 AM To: [email protected]<mailto:[email protected]> Subject: [IronPython] Memory leaks in Ipy 2.6 Hi, Working with the new version I have encountered some problems which look like memory leaks to me. I've written 3 test methods that reproduce the problems, and would appreciate your response. Thanks Problem 1 Occurs when you do the all of the following 1. Define a generator method 2. Insert a try/except clause in the method 3. define an inner method that uses some local variable 4. Call the generator method without reaching the "StopIteration" Looks like the local variable used by the inner method is never cleared. This code reproduces the problem: def test_generator_memory_leak(self): """ Ipy 2.6 This test reproduces a memory leak when calling a generator method without reaching the end. """ def coroutine(): try: pass except: pass just_numbers = range(1,1000) def inner_method(): return just_numbers yield None yield None from System import GC def get_memory(): for _ in xrange(4): GC.Collect() GC.WaitForPendingFinalizers() return GC.GetTotalMemory(True)/1e6 before = get_memory() for j in xrange(10000): crt = coroutine() crt.next() after = get_memory() self.assert_(after-before > 10,'There should be a memory leak in this case.before=%s after=%s' % (before,after)) Problem 2 The same as problem, just instead of defining an inner method, just call "eval" with any string def test_generator_memory_leak2(self): """ Ipy 2.6 This test reproduces a memory leak when calling a generator method without reaching the end. """ def coroutine(b_bool): try: pass except: pass if False: eval("") just_numbers = range(1,1000) yield None yield None from System import GC def get_memory(): for _ in xrange(4): GC.Collect() GC.WaitForPendingFinalizers() return GC.GetTotalMemory(True)/1e6 before = get_memory() for j in xrange(10000): crt = coroutine(False) crt.next() after = get_memory() self.assert_(after-before > 10,'There should be a memory leak in this case.before=%s after=%s' % (before,after)) Problem 3 This is actually a our solution to problems 1,2. We noticed that if we end the iteration by using "throw" then the memory doesn't rise on subsequent instances. Still there some memory increase that depends on the size of the local variable which doesn't seem to go away: def test_generator_memory_leak2(self): """ Ipy 2.6 when exiting a generator method with an exception, some objects are never collected This seems to be static to the type (the leak does not grow if we repeat the experiment """ def coroutine(): just_numbers = range(1,1000000) def inner_method(): return just_numbers yield None raise Exception("some exception") # comment out this line to make the test not work from System import GC def get_memory(): for _ in xrange(4): GC.Collect() GC.WaitForPendingFinalizers() return GC.GetTotalMemory(True)/1e6 before = get_memory() crt = coroutine() try: crt.next() crt.next() except: pass crt = None after = get_memory() self.assert_(after-before > 10,'There should be a memory leak in this case.before=%s after=%s' % (before,after))
_______________________________________________ Users mailing list [email protected] http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
