Jim, is this just confirming that test shows a problem with unpatched
3.2
or when Boyan's GIL state fixes are also applied? I haven't tried the
suggested patches yet, as wanted an answer as to why GIL state API had
to be
explicitly used, which he has now done so.
Graham
On 03/11/2005, at 2:45 AM, Jim Gallacher (JIRA) wrote:
[
http://issues.apache.org/jira/browse/MODPYTHON-77?
page=comments#action_12356614 ]
Jim Gallacher commented on MODPYTHON-77:
----------------------------------------
Tested gilstate.tar.gz on mpm-prefork. Error log output:
[Wed Nov 02 10:40:20 2005] [notice] Apache/2.0.54 (Debian GNU/Linux)
DAV/2 SVN/1.2.0 mod_python/3.2.4b Python/2.3.5 PHP/4.4.0-1 configured
-- resuming normal operations
[Wed Nov 02 10:40:57 2005] [notice] mod_python: (Re)importing module
'mptest'
[Wed Nov 02 10:40:57 2005] [error] BEGIN: CALLBACK
[Wed Nov 02 10:40:57 2005] [error] EXCEPTION: file() constructor not
accessible in restricted mode
[Wed Nov 02 10:40:57 2005] [error] FINISH: CALLBACK
The multiple interpreter concept of mod_python is broken for Python
extension modules since Python 2.3
----------------------------------------------------------------------
--------------------------------
Key: MODPYTHON-77
URL: http://issues.apache.org/jira/browse/MODPYTHON-77
Project: mod_python
Type: Bug
Components: core
Versions: 3.1.4
Environment: Python >= 2.3
Reporter: Boyan Boyadjiev
Attachments: diff.txt, diff2.txt, diff3.txt, gil_test.c,
gilstate.tar.gz, mod_python.c, mod_python.c.diff, mod_python.h.diff,
src.zip
The multiple interpreter concept of mod_python is broken for Python
extension modules since Python 2.3 because of the PEP 311 (Simplified
Global Interpreter Lock Acquisition for Extensions):
...
Limitations and Exclusions
This proposal identifies a solution for extension authors with
complex multi-threaded requirements, but that only require a
single "PyInterpreterState". There is no attempt to cater for
extensions that require multiple interpreter states. At the time
of writing, no extension has been identified that requires
multiple PyInterpreterStates, and indeed it is not clear if that
facility works correctly in Python itself.
...
For mod_python this means, that complex Python extensions won't work
any more with Python >= 2.3, because they are supposed to work only
with the first interpreter state initialized for the current process
(a problem we experienced). The first interpreter state is not used
by mod_python after the python_init is called.
One solution, which works fine for me, is to save the first
interpreter state into the "interpreters" dictionary in the function
python_init (MAIN_INTERPRETER is used as a key):
static int python_init(apr_pool_t *p, apr_pool_t *ptemp,
apr_pool_t *plog, server_rec *s)
{
...
/* initialize global Python interpreter if necessary */
if (! Py_IsInitialized())
{
/* initialze the interpreter */
Py_Initialize();
#ifdef WITH_THREAD
/* create and acquire the interpreter lock */
PyEval_InitThreads();
#endif
/* create the obCallBack dictionary */
interpreters = PyDict_New();
if (! interpreters) {
ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s,
"python_init: PyDict_New() failed! No more
memory?");
exit(1);
}
{
/*
Workaround PEP 311 - Simplified Global Interpreter Lock
Acquisition for Extensions
BEGIN
*/
PyObject *p = 0;
interpreterdata * idata = (interpreterdata
*)malloc(sizeof(interpreterdata));
PyThreadState* currentThreadState = PyThreadState_Get();
PyInterpreterState *istate = currentThreadState->interp;
idata->istate = istate;
/* obcallback will be created on first use */
idata->obcallback = NULL;
p = PyCObject_FromVoidPtr((void ) idata, NULL);
/*p->refcout = 1*/
PyDict_SetItemString(interpreters, MAIN_INTERPRETER, p);
/*p->refcout = 2*/
Py_DECREF(p); /*p->refcout = 1*/
/*
END
Workaround PEP 311 - Simplified Global Interpreter Lock
Acquisition for Extensions
*/
}
/* Release the thread state because we will never use
* the main interpreter, only sub interpreters created later.
*/
PyThreadState_Swap(NULL);
#ifdef WITH_THREAD
/* release the lock; now other threads can run */
PyEval_ReleaseLock();
#endif
}
return OK;
}
Another change I've made in the attached file is to Py_DECREF(p) in
get_interpreter, which will remove leaky reference to the PyCObject
with the interpreter data. This was not a real problem, but now I see
fewer leaks in BoundsChecker :-).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
