New submission from Gregory P. Smith <g...@krypto.org>: The python logging module uses a lock to surround many operations, in particular. This causes deadlocks in programs that use logging, fork and threading simultaneously.
1) spawn one or more threads in your program 2) have at least one of those threads make logging calls that will be emitted. 3) have your main thread or another thread use os.fork() to run some python code in a child process. 4) If the fork happened while one of your threads was within the logging.Handler.handle() critical section (or anywhere else where handler.lock is acquired), your child process will deadlock as soon as it tries to log anything. It inherited a held lock. The deadlock is more likely to happen on a highly loaded system which tends to widen the deadlock opportunity window due to context switching. A demo of the problem simplified into one file is attached. The Python standard library should not be the cause of these deadlocks. We need a way for all standard library locks to be cleaned up when forking. By doing one of the following: A) acquire all locks before forking, release them immediately after. B) forceably release all standard library locks after forking in the child process. Code was added to call some cleanups after forking in http://bugs.python.org/issue874900 but there are more things that also need this same sort of cleanup (logging for example). Rather than having to manually add after fork code hooks into every file in the standard library that uses locks, a more general solution to track and manage locks across fork would be a good idea. ---------- assignee: gregory.p.smith files: lock_fork_thread_deadlock_demo.py messages: 91674 nosy: gregory.p.smith priority: high severity: normal status: open title: Locks in python standard library should be sanitized on fork versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file14740/lock_fork_thread_deadlock_demo.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6721> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
