This is definitely a bug - we're leaking our stack frames on the CLR's
finalizer thread when we swallow exceptions while closing the generator. I
think this is the correct fix but I haven't fully tested it yet, in our
generator we have the code below but it's missing the assignment of null to
DynamicStackFrames:
try {
@throw(new GeneratorExitException());
// Generator should not have exited normally.
throw new RuntimeException("generator ignored GeneratorExit");
} catch (StopIterationException) {
// Ignore
ExceptionHelpers.DynamicStackFrames = null; // adding this
fixes it
} catch (GeneratorExitException) {
// Ignore
ExceptionHelpers.DynamicStackFrames = null; // adding this
fixes it
}
This definitely fixes 1 and 2. After running this 3 doesn't actually seem to
leak memory when running repeatedly for me but after-before > 10 is True. I'll
take a closer look and make sure this is the correct fix after the new year
when I'm back in the office.
From: [email protected]
[mailto:[email protected]] On Behalf Of Idan Zaltzberg
Sent: Monday, December 28, 2009 10:33 AM
To: [email protected]
Subject: [IronPython] Memory leaks in Ipy 2.6
Hi,
Working with the new version I have encountered some problems which look like
memory leaks to me.
I've written 3 test methods that reproduce the problems, and would appreciate
your response.
Thanks
Problem 1
Occurs when you do the all of the following
1. Define a generator method
2. Insert a try/except clause in the method
3. define an inner method that uses some local variable
4. Call the generator method without reaching the "StopIteration"
Looks like the local variable used by the inner method is never cleared.
This code reproduces the problem:
def test_generator_memory_leak(self):
"""
Ipy 2.6 This test reproduces a memory leak when calling a
generator method without reaching the end.
"""
def coroutine():
try: pass
except: pass
just_numbers = range(1,1000)
def inner_method():
return just_numbers
yield None
yield None
from System import GC
def get_memory():
for _ in xrange(4):
GC.Collect()
GC.WaitForPendingFinalizers()
return GC.GetTotalMemory(True)/1e6
before = get_memory()
for j in xrange(10000):
crt = coroutine()
crt.next()
after = get_memory()
self.assert_(after-before > 10,'There should be a memory leak
in this case.before=%s after=%s' % (before,after))
Problem 2
The same as problem, just instead of defining an inner method, just call "eval"
with any string
def test_generator_memory_leak2(self):
"""
Ipy 2.6 This test reproduces a memory leak when calling a
generator method without reaching the end.
"""
def coroutine(b_bool):
try: pass
except: pass
if False: eval("")
just_numbers = range(1,1000)
yield None
yield None
from System import GC
def get_memory():
for _ in xrange(4):
GC.Collect()
GC.WaitForPendingFinalizers()
return GC.GetTotalMemory(True)/1e6
before = get_memory()
for j in xrange(10000):
crt = coroutine(False)
crt.next()
after = get_memory()
self.assert_(after-before > 10,'There should be a memory leak
in this case.before=%s after=%s' % (before,after))
Problem 3
This is actually a our solution to problems 1,2. We noticed that if we end the
iteration by using "throw" then the memory doesn't rise on subsequent
instances. Still there some memory increase that depends on the size of the
local variable which doesn't seem to go away:
def test_generator_memory_leak2(self):
"""
Ipy 2.6 when exiting a generator method with an exception,
some objects are never collected
This seems to be static to the type (the leak does not grow
if we repeat the experiment
"""
def coroutine():
just_numbers = range(1,1000000)
def inner_method():
return just_numbers
yield None
raise Exception("some exception") # comment out this line
to make the test not work
from System import GC
def get_memory():
for _ in xrange(4):
GC.Collect()
GC.WaitForPendingFinalizers()
return GC.GetTotalMemory(True)/1e6
before = get_memory()
crt = coroutine()
try:
crt.next()
crt.next()
except:
pass
crt = None
after = get_memory()
self.assert_(after-before > 10,'There should be a memory leak
in this case.before=%s after=%s' % (before,after))
_______________________________________________
Users mailing list
[email protected]
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
