[
https://issues.apache.org/jira/browse/MODPYTHON-109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Graham Dumpleton reopened MODPYTHON-109:
----------------------------------------
Am reopening this issue as due to work on mod_wsgi I believe I have discovered
that problems with this are nothing to do with signal handlers.
The first source of Python exceptions that were seen when process was being
shutdown were due to the fact that PyFinalize() was not being called from what
the main Python interpreter's 'threading' module saw as being a valid Python
thread, ie., it expected it to be the main thread for the interpreter, when in
fact it was a foreign thread created outside of Python. The consequence of this
was that when Py_Finalize() was called and it triggered sys.exitfunc() to call
any atexit registered callbacks, a callback registered by the 'threading'
module was called and because the thread ID wasn't in the table of Python
threads, it threw a Python exception.
The next problem is with sub interpreters other than the main Python
interpreter. The issue here was that Py_Finalize() only calls sys.exitfunc() on
the main Python interpreter. As a consequence, any thread cleanup is not done
in sub interpreters when they are being destroyed and depending on what a
thread was doing, this could cause the process to crash.
What should happen on process shutdown is that it should iterate over all
interpreters, create a thread state against each interpreter in turn, call
'threading.currentThread()' to ensure that 'threading' creates a dummy thread
object for that thread state, then call sys.exitfunc(). After having done that,
then it can call Py_Finalize().
By ensuring that exit funcs are called, means that atexit.register() then
becomes an alternate way to register functions to be called on process
shutdown, which is a much better option for generic modules than trying to hook
into apache.register_cleanup().
> Signal handler calling Py_Finalize() when child processes being killed.
> -----------------------------------------------------------------------
>
> Key: MODPYTHON-109
> URL: https://issues.apache.org/jira/browse/MODPYTHON-109
> Project: mod_python
> Issue Type: Bug
> Components: core
> Affects Versions: 3.2.7
> Reporter: Graham Dumpleton
> Assigned To: Graham Dumpleton
> Fix For: 3.3
>
> Attachments: MP109_20060308_grahamd_1.diff
>
>
> When Apache is killing off child processes as part of actions taken when the
> "apachectl restart" or "apachectl graceful" command is run, it sends a
> SIGTERM signal to the child processes. This causes a signal handler
> registered by Apache to be run. That signal handler destroys the main child
> memory pool. That memory pool has though a cleanup handler associated with it
> which was registered by mod_python. That cleanup handler ultimately calls
> Py_Finalize().
> The problem with this is that Py_Finalize() isn't safe to be called from a
> signal handler and if a handler is still executing or there is a separate
> thread running in the context of Python, a deadlock will likely ensue. This
> will prevent the child process exiting due to the SIGTERM causing the Apache
> parent process to send it a SIGKILL to really kill it.
> For a more detailed assessment of the problem and what lead to this
> conclusion see:
> http://www.modpython.org/pipermail/mod_python/2006-January/019865.html
> http://www.modpython.org/pipermail/mod_python/2006-January/019866.html
> http://www.modpython.org/pipermail/mod_python/2006-January/019870.html
> To avoid the problem, the only choice seems to be avoid calling Py_Finalize()
> from the signal handler. The simplistic way of doing this seems to be to add:
> if (child_init_pool)
> return APR_SUCCESS;
> at the start of python_finalize(). This will mean that Py_Finalize() is never
> called in child processes. The full consequences of this is unknown, but on
> face value it would seem that it might be a reasonable thing to do. More
> research may be required.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
