[jira] Commented: (MODPYTHON-109) Signal handler calling Py_Finalize() when child processes being killed.
[ https://issues.apache.org/jira/browse/MODPYTHON-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12487986 ] Graham Dumpleton commented on MODPYTHON-109: In Python 2.5.1c1 they have changed how the 'threading' module makes use of the atexit module to shutdown threading. Instead of using the atexit module, the C code Python main routine now explicitly calls 'threading._shutdown()' instead. Any changes to mod_python will thus have to call 'threading._shutdown()' if it exists and then call 'sys.exitfunc()'. Signal handler calling Py_Finalize() when child processes being killed. --- Key: MODPYTHON-109 URL: https://issues.apache.org/jira/browse/MODPYTHON-109 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.7 Reporter: Graham Dumpleton Assigned To: Graham Dumpleton Fix For: 3.3 Attachments: MP109_20060308_grahamd_1.diff When Apache is killing off child processes as part of actions taken when the apachectl restart or apachectl graceful command is run, it sends a SIGTERM signal to the child processes. This causes a signal handler registered by Apache to be run. That signal handler destroys the main child memory pool. That memory pool has though a cleanup handler associated with it which was registered by mod_python. That cleanup handler ultimately calls Py_Finalize(). The problem with this is that Py_Finalize() isn't safe to be called from a signal handler and if a handler is still executing or there is a separate thread running in the context of Python, a deadlock will likely ensue. This will prevent the child process exiting due to the SIGTERM causing the Apache parent process to send it a SIGKILL to really kill it. For a more detailed assessment of the problem and what lead to this conclusion see: http://www.modpython.org/pipermail/mod_python/2006-January/019865.html http://www.modpython.org/pipermail/mod_python/2006-January/019866.html http://www.modpython.org/pipermail/mod_python/2006-January/019870.html To avoid the problem, the only choice seems to be avoid calling Py_Finalize() from the signal handler. The simplistic way of doing this seems to be to add: if (child_init_pool) return APR_SUCCESS; at the start of python_finalize(). This will mean that Py_Finalize() is never called in child processes. The full consequences of this is unknown, but on face value it would seem that it might be a reasonable thing to do. More research may be required. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MODPYTHON-109) Signal handler calling Py_Finalize() when child processes being killed.
[ http://issues.apache.org/jira/browse/MODPYTHON-109?page=comments#action_12440731 ] Graham Dumpleton commented on MODPYTHON-109: After more digging on this issue my opinion is now that we should just leave the register_cleanup() functions as is. That is don't allow Py_Finalize() to be called, but with the register_cleanup() functions being left in place. If it is shown somehow that the area is definitely a problem, we can revisit it then, but right now all I can say is that I can't set up test cases which show an issue. The implications of not calling Py_Finalize() are that any functions registered with the Python atexit module will not be called and Python objects will not be destroyed. The atexit limitation isn't a big deal as only functions which were registered in the main_interpreter (after MODPYTHON-77 changes) were called anyway, and the register_cleanup() functions should be used instead. More importantly, by not calling Py_Finalize() you don't start destroying all your interpreters while there are potentially user created threads still running. If one does call it and threads are still running, they can start crashing with Python exceptions. These wouldn't probably be noticed as stderr at that point probably isn't going to go anywhere, but when you catch exceptions and log them to the Apache error log you do see them. Thus, probably better to just let the threads run with a full environment up to the very point that the process it exited. Consequently, am going to mark this as resolved. Signal handler calling Py_Finalize() when child processes being killed. --- Key: MODPYTHON-109 URL: http://issues.apache.org/jira/browse/MODPYTHON-109 Project: mod_python Issue Type: Bug Components: core Affects Versions: 3.2.7 Reporter: Graham Dumpleton Assigned To: Graham Dumpleton Fix For: 3.3 Attachments: MP109_20060308_grahamd_1.diff When Apache is killing off child processes as part of actions taken when the apachectl restart or apachectl graceful command is run, it sends a SIGTERM signal to the child processes. This causes a signal handler registered by Apache to be run. That signal handler destroys the main child memory pool. That memory pool has though a cleanup handler associated with it which was registered by mod_python. That cleanup handler ultimately calls Py_Finalize(). The problem with this is that Py_Finalize() isn't safe to be called from a signal handler and if a handler is still executing or there is a separate thread running in the context of Python, a deadlock will likely ensue. This will prevent the child process exiting due to the SIGTERM causing the Apache parent process to send it a SIGKILL to really kill it. For a more detailed assessment of the problem and what lead to this conclusion see: http://www.modpython.org/pipermail/mod_python/2006-January/019865.html http://www.modpython.org/pipermail/mod_python/2006-January/019866.html http://www.modpython.org/pipermail/mod_python/2006-January/019870.html To avoid the problem, the only choice seems to be avoid calling Py_Finalize() from the signal handler. The simplistic way of doing this seems to be to add: if (child_init_pool) return APR_SUCCESS; at the start of python_finalize(). This will mean that Py_Finalize() is never called in child processes. The full consequences of this is unknown, but on face value it would seem that it might be a reasonable thing to do. More research may be required. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MODPYTHON-109) Signal handler calling Py_Finalize() when child processes being killed.
[ http://issues.apache.org/jira/browse/MODPYTHON-109?page=comments#action_12365805 ] Graham Dumpleton commented on MODPYTHON-109: The problem is actually bigger than just Py_Finalize(). When a server cleanup handler is registered using either apache.register_cleanup() or req.server.register_cleanup(), the execution of that handler is also registered to be called by associating it with the destruction of the main child memory pool. This means that such server cleanup handlers are also going to be executed from the context of a signal handler and thus are going to be a problem. It may be necessary to remove the feature of registering server cleanup handlers as there seems to be no other way of doing the same thing. Signal handler calling Py_Finalize() when child processes being killed. --- Key: MODPYTHON-109 URL: http://issues.apache.org/jira/browse/MODPYTHON-109 Project: mod_python Type: Bug Components: core Versions: 3.2 Reporter: Graham Dumpleton When Apache is killing off child processes as part of actions taken when the apachectl restart or apachectl graceful command is run, it sends a SIGTERM signal to the child processes. This causes a signal handler registered by Apache to be run. That signal handler destroys the main child memory pool. That memory pool has though a cleanup handler associated with it which was registered by mod_python. That cleanup handler ultimately calls Py_Finalize(). The problem with this is that Py_Finalize() isn't safe to be called from a signal handler and if a handler is still executing or there is a separate thread running in the context of Python, a deadlock will likely ensue. This will prevent the child process exiting due to the SIGTERM causing the Apache parent process to send it a SIGKILL to really kill it. For a more detailed assessment of the problem and what lead to this conclusion see: http://www.modpython.org/pipermail/mod_python/2006-January/019865.html http://www.modpython.org/pipermail/mod_python/2006-January/019866.html http://www.modpython.org/pipermail/mod_python/2006-January/019870.html To avoid the problem, the only choice seems to be avoid calling Py_Finalize() from the signal handler. The simplistic way of doing this seems to be to add: if (child_init_pool) return APR_SUCCESS; at the start of python_finalize(). This will mean that Py_Finalize() is never called in child processes. The full consequences of this is unknown, but on face value it would seem that it might be a reasonable thing to do. More research may be required. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
