Python 2.3 and simplified GIL state API still causes problems.
--------------------------------------------------------------
Key: MODPYTHON-217
URL: https://issues.apache.org/jira/browse/MODPYTHON-217
Project: mod_python
Issue Type: Bug
Components: core
Affects Versions: 3.2.10, 3.3
Reporter: Graham Dumpleton
There are still problems in mod_python when third party extension modules are
used which make use of the Python simplified GIL state API. Specifically, if C
code makes calls to PyGILState_Ensure() when it already holds the Python GIL
and has an active thread state, but the active thread was one which was
originally created outside of Python, such as by Apache with mod_python, then
the call to PyGILState_Ensure() results in a dead lock. This is occurring
because the PyGILState_Ensure() function finds it does not know anything about
the current thread ID and therefore assumes it needs to create a new thread
state for it and make that the active thread state. In doing that though it
tries to acquire the GIL when the thread already holds it, resulting in the
deadlock.
At this stage it is believed this only occurs with Python 2.3 and shouldn't be
a problem with later versions of Python. This is the case as later versions of
Python were modified so that PyThreadState_New() will register the thread with
the underpinnings of the PyGILState_???() mechanism and because it already
knows about it, it will not attempt to create a new thread state object and so
the deadlock will not occur.
When C code is hand crafted it is unlikely that anyone would call
PyGILState_Ensure() when they already know they hold the GIL, ie., when calling
out of Python code, but SWIG when used with the -threads option does exactly
this. For example:
SWIGINTERN PyObject *_wrap_ap_get_server_version(PyObject
*SWIGUNUSEDPARM(self), PyObject *args) { PyObject *resultobj = 0;
char *result = 0 ;
SWIG_PYTHON_THREAD_BEGIN_BLOCK;
if (!PyArg_ParseTuple(args,(char *)":ap_get_server_version")) SWIG_fail;
{
SWIG_PYTHON_THREAD_BEGIN_ALLOW;
result = (char *)ap_get_server_version();
SWIG_PYTHON_THREAD_END_ALLOW;
}
resultobj = SWIG_FromCharPtr((const char *)result);
SWIG_PYTHON_THREAD_END_BLOCK;
return resultobj;
fail:
SWIG_PYTHON_THREAD_END_BLOCK;
return NULL;
}
where SWIG_PYTHON_THREAD_BEGIN_BLOCK eventually expands to a call to
PyGILState_Ensure().
The only solution to this problem would be for mod_python to behave differently
when it is acquiring a thread state against the main Python interpreter, ie.
'main_interpreter'. Specifically, rather than use PyThreadState_New() etc, it
should call PyGILState_Ensure() instead. When needing to release the main
Python interpreter it should call PyGILState_Release(). It should continue to
work as before for all other interpreters and anyone with GILState code would
need to ensure they are using 'main_interpreter'.
Unfortunately at the moment making this change is not completely straight
forward as when an interpreter is being released it isn't known that it is the
main Python interpreter. Thus some knowledge that it is the main interpreter
would need to be carried into the call. The only other option is that the list
of all active interpreters is traversed and the last one in the list is assumed
to be the main interpreter. Then if the interpreter being released matches that
then act differently.
Note that at present the acquire/release methods are exported as optional
functions so that other Apache modules use them. It is unlikely that anyone is
making use of them, so changing the in/out parameters of the functions would
possibly be acceptable.
Also note that maintaining support for threaded and non threaded Python and now
this GIL state variation is going to make the code even more messy. As such it
might be worthwhile considering dropping support for non threaded Python. These
days Python defaults to being threaded anyway, and highly unlikely that someone
would want to use non thread Python with mod_python anyway. When ever any
development of mod_python is done, no testing is ever done with a non threaded
Python anyway and so it is possibly questionable if that support still works.
FWIW, this issue came up as a result of queries expressed in:
http://www.modpython.org/pipermail/mod_python/2007-April/023445.html
The GIL state issue has attempted to be addressed before in MODPYTHON-77 but
this particular issue not captured in that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
