On 10/18/06, Tim Peters <[EMAIL PROTECTED]> wrote: > [Mike Klaas] > > Indeed. > > Note that I just attached a much simpler pure-Python script that fails > very quickly, on Windows, using a debug build. Read the new comment > to learn why both "Windows" and "debug build" are essential to it > failing reliably and quickly ;-)
Thanks! Next time I find a bug, installing Windows will certainly be my first step <g>. <> > Yes, but you did good! This is still just an educated guess on my > part, but my education here is hard to match ;-): this new business > of generators deciding to "clean up after themselves" if they're left > hanging appears to have made it possible for a generator to hold on to > a frame whose thread state has been free()'d, after the thread that > created the generator has gone away. Then when the generator gets > collected as trash, the new exception-based "clean up abandoned > generator" gimmick tries to access the generator's frame's thread > state, but that's just a raw C struct (not a Python object with > reachability-based lifetime), and the thread free()'d that struct when > the thread went away. The important timing-based vagary here is > whether dead-thread cleanup gets performed before the main thread > tries to clean up the trash generator. Indeed--and normally it doesn't happen that way. My/your script never crashes on the first iteration because the thread's target is the generator and thus it gets DECREF'd before the thread terminates. But the exception from the first iteration holds on to a reference to the frame/generator so when it gets cleaned up (in the second iteration, due to a new exception overwriting it) the generator is freed after the thread is destroyed. At least, I think... <> > Offhand I don't know how to repair it. Thread states /aren't/ Python > objects, and there's no provision for a thread state to outlive the > thread it represents. Take this with a grain of salt, but ISTM that the problem can be repaired by resetting the generator's frame threadstate to the current threadstate: (in genobject.c:gen_send_ex():80) Py_XINCREF(tstate->frame); assert(f->f_back == NULL); f->f_back = tstate->frame; + f->f_tstate = tstate; gen->gi_running = 1; result = PyEval_EvalFrameEx(f, exc); gen->gi_running = 0; Shouldn't the thread state generally be the same anyway? (I seem to recall some gloomy warning against resuming generators in separate threads). This solution is surely wrong--if f_tstate != tstate, then the generator _is_ being resumed in another thread and so the generated traceback will be wrong (among other issues which surely occur by fudging a frame's threadstate). Perhaps it could be set conditionally by gen_close before signalling the exception? A lie, but a smaller lie than a segfault. We could advertise that the exception ocurring from generator .close() isn't guaranteed to have an accurate traceback in this case. Take all this with a grain of un-core-savvy salt. Thanks again for investigating this, Tim, -Mike _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com