[ http://issues.apache.org/jira/browse/MODPYTHON-77?page=all ]
Graham Dumpleton updated MODPYTHON-77:
--------------------------------------
Attachment: gilstate.tar.gz
Here is my own simplified test case for the GIL state API issue which exhibits
the problem in a different way. In this case it is where the exception:
file() constructor not accessible in restricted mode
is raised.
Have constructed this test case as this particular problem has come up numerous
times before. It is known that in some cases the problem is due to an actual
error in Python (when Python 2.3.5 or later is used), ie.:
http://sourceforge.net/tracker/index.php?func=detail&aid=1163563&group_id=5470&atid=105470
The attached test case causes a problem though when using Python on Mac OS X
(10.3) which is supposed to be Python 2.3.0, thus cannot be related to the bug
in Python and is more likely to be the GIL state API issue.
In respect of the changes already posted here now that I have started looking
at it, what I don't understand is why the mod_python.c file has to use the GIL
state API functions at all. I would have thought that it was enough to save the
initial interpreter as "main_interpreter" and for users to ensure they then
explicitly named the interpreter they used as being "main_interpreter".
Can someone please explain the reason why there has to be this special case
processing whereby GIL state API is only used when making calls into the
main_interpreter. This doesn't make sense, the full API for acquiring the
thread lock as it existed should have been sufficient. The only thing that
might make sense is if it is done this way as a workaround for the bug in
Python listed above on SourceForge.
> The multiple interpreter concept of mod_python is broken for Python extension
> modules since Python 2.3
> ------------------------------------------------------------------------------------------------------
>
> Key: MODPYTHON-77
> URL: http://issues.apache.org/jira/browse/MODPYTHON-77
> Project: mod_python
> Type: Bug
> Components: core
> Versions: 3.1.4
> Environment: Python >= 2.3
> Reporter: Boyan Boyadjiev
> Attachments: diff.txt, diff2.txt, diff3.txt, gil_test.c, gilstate.tar.gz,
> mod_python.c, mod_python.c.diff, mod_python.h.diff, src.zip
>
> The multiple interpreter concept of mod_python is broken for Python extension
> modules since Python 2.3 because of the PEP 311 (Simplified Global
> Interpreter Lock Acquisition for Extensions):
> ...
> Limitations and Exclusions
> This proposal identifies a solution for extension authors with
> complex multi-threaded requirements, but that only require a
> single "PyInterpreterState". There is no attempt to cater for
> extensions that require multiple interpreter states. At the time
> of writing, no extension has been identified that requires
> multiple PyInterpreterStates, and indeed it is not clear if that
> facility works correctly in Python itself.
> ...
> For mod_python this means, that complex Python extensions won't work any more
> with Python >= 2.3, because they are supposed to work only with the first
> interpreter state initialized for the current process (a problem we
> experienced). The first interpreter state is not used by mod_python after the
> python_init is called.
> One solution, which works fine for me, is to save the first interpreter state
> into the "interpreters" dictionary in the function python_init
> (MAIN_INTERPRETER is used as a key):
> static int python_init(apr_pool_t *p, apr_pool_t *ptemp,
> apr_pool_t *plog, server_rec *s)
> {
> ...
> /* initialize global Python interpreter if necessary */
> if (! Py_IsInitialized())
> {
> /* initialze the interpreter */
> Py_Initialize();
> #ifdef WITH_THREAD
> /* create and acquire the interpreter lock */
> PyEval_InitThreads();
> #endif
> /* create the obCallBack dictionary */
> interpreters = PyDict_New();
> if (! interpreters) {
> ap_log_error(APLOG_MARK, APLOG_NOERRNO|APLOG_ERR, 0, s,
> "python_init: PyDict_New() failed! No more memory?");
> exit(1);
> }
> {
> /*
> Workaround PEP 311 - Simplified Global Interpreter Lock
> Acquisition for Extensions
> BEGIN
> */
> PyObject *p = 0;
> interpreterdata * idata = (interpreterdata
> *)malloc(sizeof(interpreterdata));
> PyThreadState* currentThreadState = PyThreadState_Get();
> PyInterpreterState *istate = currentThreadState->interp;
> idata->istate = istate;
> /* obcallback will be created on first use */
> idata->obcallback = NULL;
> p = PyCObject_FromVoidPtr((void ) idata, NULL); /*p->refcout = 1*/
> PyDict_SetItemString(interpreters, MAIN_INTERPRETER, p);
> /*p->refcout = 2*/
> Py_DECREF(p); /*p->refcout = 1*/
> /*
> END
> Workaround PEP 311 - Simplified Global Interpreter Lock
> Acquisition for Extensions
> */
> }
> /* Release the thread state because we will never use
> * the main interpreter, only sub interpreters created later. */
> PyThreadState_Swap(NULL);
> #ifdef WITH_THREAD
> /* release the lock; now other threads can run */
> PyEval_ReleaseLock();
> #endif
> }
> return OK;
> }
> Another change I've made in the attached file is to Py_DECREF(p) in
> get_interpreter, which will remove leaky reference to the PyCObject with the
> interpreter data. This was not a real problem, but now I see fewer leaks in
> BoundsChecker :-).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
