Hello, 1. The solution I proposed in my previous email doesn't work since the last dying tasklet would never get a chance to be cleaned up.
2. I finally sorted my tasklet cleanup problem out by breaking the reference cycle. I didn't succeed to break the cycle by using a weakref on the cycle: some elements were garbage collected too early. I finally added a level of indirection, namely a "current" variable which holds the currently executing wrapper, which works similarly to stackless.current. Then my function use the "current" variable and is not a bound method anymore. Reference cycle broken, and now I can use __del__ to kill my tasklets/wrappers and get a similar behavior than greenlets for cleanup. I hope I am clear ;) 3. I still consider that an TaskletExit should be raised for cleanup, but at least, I have found a workaround ;) Cheers, Sylvain On Mon, May 21, 2012 at 4:24 PM, Sylvain Prat <[email protected]> wrote: > Thanks for the clarification Kristján! So, we should not expect a > TaskletExit to be raised ;) > > I fully understand the problem with __del__ and circular references, > which is also documented there: > http://docs.python.org/reference/datamodel.html#object.__del__ > > The problem is, when I try to cleanup resources by myself, I also get > myself into the __del__ & circular references trap, and I can't find a > workaround. If I try to replace a hard reference by a weakref in the > circular references loop, I also get some errors (I've not > investigated them yet, maybe they are avoidable). I suspect the only > way to get cleanup work is to do something in stackless itself. > > Also, what do you mean by "it were safer to put such tasklets into > gc.garbage"? How can we do the cleanup then? Do you mean we have to > poll gc.garbage (maybe in a cleaning tasklet), identify tasklets that > have to be destroyed, and kill them manually, so that we can control > the moment when the tasklet will be destroyed? > > About the problem of "switching" to the dying tasklet and returning > back to the caller: why not flag the "garbage collectable" tasklet as > "to be killed", put it in the runnables (it is not garbage collectable > anymore, which is not recommended but possible - see > http://docs.python.org/reference/datamodel.html#object.__del__), and > let it wait for its turn (scheduling) to actually kill it? In this > case, it would have a chance to cleanup, just not immediately. > > Any suggestions/workarounds for resources cleanup? > > Cheers, > Sylvain > > > On Mon, May 21, 2012 at 1:21 PM, Kristján Valur Jónsson > <[email protected]> wrote: >> Well. >> It is due to this code here, in taskletojbect: >> >> static int >> tasklet_traverse(PyTaskletObject *t, visitproc visit, void *arg) >> { >> PyFrameObject *f; >> PyThreadState *ts = PyThreadState_GET(); >> >> /* tasklets that need to be switched to for the kill, can't be collected. >> * Only trivial decrefs are allowed during GC collect >> */ >> if (tasklet_has_c_stack(t)) >> PyObject_GC_Collectable((PyObject *)t, visit, arg, 0); >> >> What can be done during garbage collection has always been a bit restricted. >> Tranditionally, gcmodule disallows all objects with __del__ methods and >> puts them in gc.garbage. I have recently discovered, by asking on >> python-dev, that this is not due to a fundamental fragility of the garbage >> collection process, but rather the fact that it is likely that __del__ >> methods invoked during garbage collection will cause exceptions, since the >> members of the cycle would refer to each other. See this discussion: >> http://grokbase.com/p/python/python-dev/124hc36dtk/issue-9141-finalizers-and-gc-module >> >> Anyway, for tasklets, the problem is that when killing tasklets, we have to >> "switch" to them. If other tasklets are runnable, this may kick off a >> sequence of running tasklets, there is no _guarantee_ that we will switch >> immediately back to the tasklet currently doing the garbage collection. >> In the past, we have had a number of crashes related to this. I don't have >> the details anymore. >> >> So, anyway, at one point I decided that it were safer to put such tasklets >> into gc.garbage. >> >> It is possible that my fears are unfounded. We could possibly engineer >> tasklet.kill() in such a way that we are very sure that no other tasklets >> will run and we will switch immediately back to the originating tasklet. >> >> K >> >> >> >>> -----Original Message----- >>> From: Richard Tew [mailto:[email protected]] >>> Sent: 20. maí 2012 00:06 >>> To: The Stackless Python Mailing List >>> Cc: Kristján Valur Jónsson >>> Subject: Re: [Stackless] Tasklet cleanup? >>> >>> On Sun, May 20, 2012 at 10:29 AM, Sylvain Prat <[email protected]> >>> wrote: >>> > I'm wondering how tasklets can clean themselves up when they are >>> > destroyed due to garbage collection (i.e. when they are not in the >>> > runnables and not referenced by any object anymore). Greenlet solves >>> > this problem by raising a GreenletExit exception in the greenlet's run >>> > function when the greenlet is about to die due to garbage collection. >>> > However, in stackless, it seems that no TaskletExit exception is >>> > raised when the tasklet is about to die, so we can't simply use a >>> > try/finally in the tasklet's callable to clean up resources. >>> >>> Greenlet is derived from Stackless, so similarly TaskletExit should be >>> raised >>> on a tasklet that is being garbage collected. >>> >>> > I tried to wrap my tasklet in a parent object which has the same >>> > lifespan as my tasklet and has a __del__ function for cleaning up, but >>> > I keep having problems with circular references (the wrapper/parent >>> > object also provides the callable of the tasklet, i.e. a bound method) >>> > that make the tasklet/parent object uncollectable (circular references >>> > : wrapper --> tasklet --> stackless machinery? --> callable stack >>> > frame (bound method of wrapper) --> wrapper). Same problem when >>> trying >>> > to inherit from tasklet. >>> > >>> > So, how can I clean up resources in tasklets? (I'm pretty sure I've >>> > missed something obvious) >>> >>> I've attached two example scripts where a tasklet dies, but does not get the >>> TaskletExit raised on it. This is something Kristjan Valur has been >>> working on, >>> but there have been upsides and downsides to the different approaches and >>> as I understand it the ideal solution to tasklet destruction is yet to be >>> found. >>> >>> Kristjan, why is TaskletExit not being raised? Any ideas? >>> >>> Cheers, >>> Richard. >> >> >> _______________________________________________ >> Stackless mailing list >> [email protected] >> http://www.stackless.com/mailman/listinfo/stackless > > > > -- > Sylvain PRAT -- Sylvain PRAT _______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless
