On 17 March 2011 09:27, Stefan Behnel <stefan...@behnel.de> wrote: > Dag Sverre Seljebotn, 17.03.2011 08:38: >> >> On 03/17/2011 12:24 AM, Greg Ewing wrote: >>> >>> Stefan Behnel wrote: >>> >>>> I'm not sure if this is a good idea. "nogil" blocks don't have a way to >>>> handle exceptions, so simply jumping out of them because an inner 'with >>>> gil' block raised an exception can have unexpected side effects. >>> >>> Seems to me that the __Pyx_WriteUnraisable should be done at >>> the end of the 'with gil' block, and execution should then >>> continue from there. >>> >>> In other words, the effect on exception handling should be >>> the same as if the 'with gil' block had been factored out into >>> a separate function having no exception return value. >>> >> >> -1. >> >> I consider the fact that exceptions don't propagate from some functions a >> "currently unfixable bug". We should plan for it being fixed some day. > > It can't be fixed in general, because there are cases where exceptions > simply cannot be propagated. Think of C callbacks, for example. C doesn't > have a "normal" way of dealing with exceptions, so if an exception that > originated from a callback simply leads to returning from the function, it > may mean that the outer C code will simply continue to execute normally. > Nothing's won in that case. > > In code: > > cdef void c_callback(...) nogil: > ... do some C stuff ... > with gil: > ... do some Python stuff ... > ... do some more C stuff ... > > So far, there are two proposed ways of doing this. > > 1) acquire the GIL on entry and exit, handling unraisable exceptions right > before exiting. > > 2) keep all GIL requiring code inside of the "with gil" block, including > unraisable exceptions. > > I find (2) a *lot* more intuitive, as well as much safer. We can't know what > effects the surrounding "do C stuff" code has. It may contain thread-safe C > level cleanup code for the "with gil" block, for example, or preparation > code that enables returning into the calling C code. Simply jumping out of > the GIL block without executing the trailing code may simply not work at > all.
Which is exactly why users will have to handle exceptions if they want their C code to execute. > >> We could perhaps fix exception propagation from nogil functions by using >> some conventions + setjmp/longjmp. Mono does this when calling into native >> code, and I recently did it manually in Cython to propagate exceptions >> through the Fortran wrappers in SciPy. > > Regardless of the topic of this thread, it would be nice to have longjmp > support in Cython. Lupa, my Cython wrapper for LuaJIT, currently has to work > around several quirks in that area. > > >> Also, the GIL may not be around >> forever even in CPython? (All arguments I've seen for keeping it has been >> along the lines of "it slows down serial code", not that it is considered >> a >> good thing.) > > If it ever gets removed, there will surely have to be an emulation layer for > C modules. Many of them simply use it as thread-lock, and that's totally > reasonable IMHO. > > >> Designing a language around the GIL feels like a dead-end to me. > > We keep having diverging opinions about the GIL. I like it, and I keep > repeating myself by saying that "threading should be explicit". Having a way > to lock the whole interpreter and to keep parallel execution and reentry > points to well defined places in your code is a great feature. > > >> I'm OK >> with being practical in the face of the limitations of today; but let's >> keep "with gil" and "with nogil" something that can become noops in the >> future without too much pain. Yes, I know that if the GIL goes it will >> break Stefan's lxml code, and I'm sure other code -- I'm just saying that >> we shouldn't make the language design even more GIL-centric than it >> already >> is. > > It's not. Even a removal of the GIL won't remove the fact that C can't > propagate exceptions. > > Stefan > _______________________________________________ > cython-devel mailing list > cython-devel@python.org > http://mail.python.org/mailman/listinfo/cython-devel > _______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
