Deron Meranda commented on MODPYTHON-217:

Another real-world case where this comes up is with the Xapian
search engine (http://xapian.org/).  It also is SWIG generated.
See their bug #185,

> Python 2.3 and simplified GIL state API still causes problems.
> --------------------------------------------------------------
>                 Key: MODPYTHON-217
>                 URL: https://issues.apache.org/jira/browse/MODPYTHON-217
>             Project: mod_python
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 3.3.x, 3.2.10
>            Reporter: Graham Dumpleton
> There are still problems in mod_python when third party extension modules are 
> used which make use of the Python simplified GIL state API. Specifically, if 
> C code makes calls to PyGILState_Ensure() when it already holds the Python 
> GIL and has an active thread state, but the active thread was one which was 
> originally created outside of Python, such as by Apache with mod_python, then 
> the call to PyGILState_Ensure() results in a dead lock. This is occurring 
> because the PyGILState_Ensure() function finds it does not know anything 
> about the current thread ID and therefore assumes it needs to create a new 
> thread state for it and make that the active thread state. In doing that 
> though it tries to acquire the GIL when the thread already holds it, 
> resulting in the deadlock.
> At this stage it is believed this only occurs with Python 2.3 and shouldn't 
> be a problem with later versions of Python. This is the case as later 
> versions of Python were modified so that PyThreadState_New() will register 
> the thread with the underpinnings of the PyGILState_???() mechanism and 
> because it already knows about it, it will not attempt to create a new thread 
> state object and so the deadlock will not occur.
> When C code is hand crafted it is unlikely that anyone would call 
> PyGILState_Ensure() when they already know they hold the GIL, ie., when 
> calling out of Python code, but SWIG when used with the -threads option does 
> exactly this. For example:
> SWIGINTERN PyObject *_wrap_ap_get_server_version(PyObject 
> *SWIGUNUSEDPARM(self), PyObject *args) {  PyObject *resultobj = 0;
>   char *result = 0 ;
>   if (!PyArg_ParseTuple(args,(char *)":ap_get_server_version")) SWIG_fail;
>   {
>     result = (char *)ap_get_server_version();
>   }
>   resultobj = SWIG_FromCharPtr((const char *)result);
>   return resultobj;
> fail:
>   return NULL;
> }
> where SWIG_PYTHON_THREAD_BEGIN_BLOCK eventually expands to a call to 
> PyGILState_Ensure().
> The only solution to this problem would be for mod_python to behave 
> differently when it is acquiring a thread state against the main Python 
> interpreter, ie. 'main_interpreter'. Specifically, rather than use 
> PyThreadState_New() etc, it should call PyGILState_Ensure() instead. When 
> needing to release the main Python interpreter it should call 
> PyGILState_Release(). It should continue to work as before for all other 
> interpreters and anyone with GILState code would need to ensure they are 
> using 'main_interpreter'.
> Unfortunately at the moment making this change is not completely straight 
> forward as when an interpreter is being released it isn't known that it is 
> the main Python interpreter. Thus some knowledge that it is the main 
> interpreter would need to be carried into the call. The only other option is 
> that the list of all active interpreters is traversed and the last one in the 
> list is assumed to be the main interpreter. Then if the interpreter being 
> released matches that then act differently.
> Note that at present the acquire/release methods are exported as optional 
> functions so that other Apache modules use them. It is unlikely that anyone 
> is making use of them, so changing the in/out parameters of the functions 
> would possibly be acceptable.
> Also note that maintaining support for threaded and non threaded Python and 
> now this GIL state variation is going to make the code even more messy. As 
> such it might be worthwhile considering dropping support for non threaded 
> Python. These days Python defaults to being threaded anyway, and highly 
> unlikely that someone would want to use non thread Python with mod_python 
> anyway. When ever any development of mod_python is done, no testing is ever 
> done with a non threaded Python anyway and so it is possibly questionable if 
> that support still works.
> FWIW, this issue came up as a result of queries expressed in:
>   http://www.modpython.org/pipermail/mod_python/2007-April/023445.html
> The GIL state issue has attempted to be addressed before in MODPYTHON-77 but 
> this particular issue not captured in that.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to