Yes, i´m sure we need to revisit this. > -----Original Message----- > From: [email protected] [mailto:stackless- > [email protected]] On Behalf Of Sylvain Prat > Sent: 23. maí 2012 14:41 > To: The Stackless Python Mailing List > Subject: Re: [Stackless] Tasklet cleanup? > > Hello, > > 1. The solution I proposed in my previous email doesn't work since the last > dying tasklet would never get a chance to be cleaned up. > > 2. I finally sorted my tasklet cleanup problem out by breaking the reference > cycle. I didn't succeed to break the cycle by using a weakref on the cycle: > some elements were garbage collected too early. > I finally added a level of indirection, namely a "current" variable which > holds > the currently executing wrapper, which works similarly to stackless.current. > Then my function use the "current" variable and is not a bound method > anymore. Reference cycle broken, and now I can use __del__ to kill my > tasklets/wrappers and get a similar behavior than greenlets for cleanup. I > hope I am clear ;) > > 3. I still consider that an TaskletExit should be raised for cleanup, but at > least, > I have found a workaround ;) > > Cheers, > Sylvain > > > On Mon, May 21, 2012 at 4:24 PM, Sylvain Prat <[email protected]> > wrote: > > Thanks for the clarification Kristján! So, we should not expect a > > TaskletExit to be raised ;) > > > > I fully understand the problem with __del__ and circular references, > > which is also documented there: > > http://docs.python.org/reference/datamodel.html#object.__del__ > > > > The problem is, when I try to cleanup resources by myself, I also get > > myself into the __del__ & circular references trap, and I can't find a > > workaround. If I try to replace a hard reference by a weakref in the > > circular references loop, I also get some errors (I've not > > investigated them yet, maybe they are avoidable). I suspect the only > > way to get cleanup work is to do something in stackless itself. > > > > Also, what do you mean by "it were safer to put such tasklets into > > gc.garbage"? How can we do the cleanup then? Do you mean we have to > > poll gc.garbage (maybe in a cleaning tasklet), identify tasklets that > > have to be destroyed, and kill them manually, so that we can control > > the moment when the tasklet will be destroyed? > > > > About the problem of "switching" to the dying tasklet and returning > > back to the caller: why not flag the "garbage collectable" tasklet as > > "to be killed", put it in the runnables (it is not garbage collectable > > anymore, which is not recommended but possible - see > > http://docs.python.org/reference/datamodel.html#object.__del__), and > > let it wait for its turn (scheduling) to actually kill it? In this > > case, it would have a chance to cleanup, just not immediately. > > > > Any suggestions/workarounds for resources cleanup? > > > > Cheers, > > Sylvain > > > > > > On Mon, May 21, 2012 at 1:21 PM, Kristján Valur Jónsson > > <[email protected]> wrote: > >> Well. > >> It is due to this code here, in taskletojbect: > >> > >> static int > >> tasklet_traverse(PyTaskletObject *t, visitproc visit, void *arg) { > >> PyFrameObject *f; > >> PyThreadState *ts = PyThreadState_GET(); > >> > >> /* tasklets that need to be switched to for the kill, can't be > >> collected. > >> * Only trivial decrefs are allowed during GC collect > >> */ > >> if (tasklet_has_c_stack(t)) > >> PyObject_GC_Collectable((PyObject *)t, visit, arg, 0); > >> > >> What can be done during garbage collection has always been a bit > restricted. Tranditionally, gcmodule disallows all objects with __del__ > methods and puts them in gc.garbage. I have recently discovered, by asking > on python-dev, that this is not due to a fundamental fragility of the garbage > collection process, but rather the fact that it is likely that __del__ methods > invoked during garbage collection will cause exceptions, since the members > of the cycle would refer to each other. See this discussion: > >> http://grokbase.com/p/python/python-dev/124hc36dtk/issue-9141- > finaliz > >> ers-and-gc-module > >> > >> Anyway, for tasklets, the problem is that when killing tasklets, we have to > "switch" to them. If other tasklets are runnable, this may kick off a > sequence > of running tasklets, there is no _guarantee_ that we will switch immediately > back to the tasklet currently doing the garbage collection. > >> In the past, we have had a number of crashes related to this. I don't have > the details anymore. > >> > >> So, anyway, at one point I decided that it were safer to put such tasklets > into gc.garbage. > >> > >> It is possible that my fears are unfounded. We could possibly engineer > tasklet.kill() in such a way that we are very sure that no other tasklets > will run > and we will switch immediately back to the originating tasklet. > >> > >> K > >> > >> > >> > >>> -----Original Message----- > >>> From: Richard Tew [mailto:[email protected]] > >>> Sent: 20. maí 2012 00:06 > >>> To: The Stackless Python Mailing List > >>> Cc: Kristján Valur Jónsson > >>> Subject: Re: [Stackless] Tasklet cleanup? > >>> > >>> On Sun, May 20, 2012 at 10:29 AM, Sylvain Prat > >>> <[email protected]> > >>> wrote: > >>> > I'm wondering how tasklets can clean themselves up when they are > >>> > destroyed due to garbage collection (i.e. when they are not in the > >>> > runnables and not referenced by any object anymore). Greenlet > >>> > solves this problem by raising a GreenletExit exception in the > >>> > greenlet's run function when the greenlet is about to die due to > garbage collection. > >>> > However, in stackless, it seems that no TaskletExit exception is > >>> > raised when the tasklet is about to die, so we can't simply use a > >>> > try/finally in the tasklet's callable to clean up resources. > >>> > >>> Greenlet is derived from Stackless, so similarly TaskletExit should > >>> be raised on a tasklet that is being garbage collected. > >>> > >>> > I tried to wrap my tasklet in a parent object which has the same > >>> > lifespan as my tasklet and has a __del__ function for cleaning up, > >>> > but I keep having problems with circular references (the > >>> > wrapper/parent object also provides the callable of the tasklet, > >>> > i.e. a bound method) that make the tasklet/parent object > >>> > uncollectable (circular references > >>> > : wrapper --> tasklet --> stackless machinery? --> callable stack > >>> > frame (bound method of wrapper) --> wrapper). Same problem when > >>> trying > >>> > to inherit from tasklet. > >>> > > >>> > So, how can I clean up resources in tasklets? (I'm pretty sure > >>> > I've missed something obvious) > >>> > >>> I've attached two example scripts where a tasklet dies, but does not > >>> get the TaskletExit raised on it. This is something Kristjan Valur > >>> has been working on, but there have been upsides and downsides to > >>> the different approaches and as I understand it the ideal solution to > tasklet destruction is yet to be found. > >>> > >>> Kristjan, why is TaskletExit not being raised? Any ideas? > >>> > >>> Cheers, > >>> Richard. > >> > >> > >> _______________________________________________ > >> Stackless mailing list > >> [email protected] > >> http://www.stackless.com/mailman/listinfo/stackless > > > > > > > > -- > > Sylvain PRAT > > -- > Sylvain PRAT > > _______________________________________________ > Stackless mailing list > [email protected] > http://www.stackless.com/mailman/listinfo/stackless
_______________________________________________ Stackless mailing list [email protected] http://www.stackless.com/mailman/listinfo/stackless
