New submission from Gary Yee <[email protected]>:
Background:
I'm using multiprocessing not to run jobs in parallel, but to run functions in
a different process space so they can be done as a different user. I am thus
using multiprocessing in a multithreaded (Linux) application.
Problem:
In multiprocessing/forking.py the poll() function is not thread safe. If
multiple threads call poll() you could have two back-to-back calls to
os.waitpid() on the same PID (this happens frequently when multiprocessing's
_cleanup() function is called).
Traceback (most recent call last):
File "/opt/scyld/foo.py", line 178, in call
pool = Pool(processes=1)
File "/opt/scyld/python/2.6.5/lib/python2.6/multiprocessing/__init__.py",
line 227, in Pool
return Pool(processes, initializer, initargs)
File "/opt/scyld/python/2.6.5/lib/python2.6/multiprocessing/pool.py", line
104, in __init__
w.start()
File "/opt/scyld/python/2.6.5/lib/python2.6/multiprocessing/process.py", line
99, in start
_cleanup()
File "/opt/scyld/python/2.6.5/lib/python2.6/multiprocessing/process.py", line
53, in _cleanup
if p._popen.poll() is not None:
File "/opt/scyld/python/2.6.5/lib/python2.6/multiprocessing/forking.py", line
106, in poll
pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 10] No child processes
Suggested Fix:
Wrap the os.waitpid() call in a try/except block looking for OSError 10
exceptions and return the returncode currently available in that event. The
one potential problem this introduces is if someone calls os.waitpid() on that
PID on the process without going through forking.py. This will result in
self.returncode never being set to a non-None value. If you're using the
multiprocessing module to create processes, however, you should be also using
it to clean up after itself.
I've attached a test file.
----------
components: Library (Lib)
files: foo.py
messages: 134165
nosy: Gary.Yee
priority: normal
severity: normal
status: open
title: Poll call in multiprocessing/forking.py is not thread safe. Results in
"OSError: [Errno 10] No child processes" exceptions.
type: crash
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python
3.3, Python 3.4
Added file: http://bugs.python.org/file21741/foo.py
_______________________________________
Python tracker <[email protected]>
<http://bugs.python.org/issue11891>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
