That's very interesting, can you produce a standalone example that does not use multiprocessing? That would make it much easier to fix the bug (e.g. os.fork followed by os.urandom failing)
On Wed, Dec 23, 2015 at 1:54 PM, hubo <h...@jiedaibao.com> wrote: > Thanks for the response. Should I put it directly in the bug tracker? > > FYI, I've located the reason to be the incompatibility with python-daemon > (or rather the standard unix-daemon behavior) and PyPy *posix.urandom* > implementation. > > It seems that in PyPy 4.0.1, when module *random* loaded, a file > descriptor is created on /dev/urandom. I think PyPy implementation use the > shared descriptor to read from /dev/urandom. Sadly when python-daemon fork > the process and turns it into an unix daemon, it closes all the currently > open file descriptors. After that all os.urandom calls failed with OSError. > I think maybe the other functions of Random class is also using the file > descriptor in C code and just never detects if the return value is 0, and > causes the dead loop. > > I think the problem will be solved if the implementation re-open the > handle when it is closed somehow. > > multiprocessing is using random internally. Also there are lots of other > modules using random, like email etc. The dead loop occurs when you use any > of the libraries in a daemon. > > > > 2015-12-23 > ------------------------------ > hubo > ------------------------------ > > *发件人：*Maciej Fijalkowski <fij...@gmail.com> > *发送时间：*2015-12-23 19:35 > *主题：*Re: [pypy-dev] Dead loop occurs when using python-daemon and > multiprocessing together in PyPy 4.0.1 > *收件人：*"hubo"<h...@jiedaibao.com> > *抄送：*"pypy-dev"<pypy-dev@python.org> > > Hi hubo > > Can you put it as a bug report? Those things get easily lost on the > mailing list (and sadly I won't look at it right now, multiprocessing > scares me) > > On Wed, Dec 23, 2015 at 12:03 PM, hubo <h...@jiedaibao.com> wrote: > >> Hello devs, >> >> A (possible) dead loop is found when I use python-daemon and >> multiprocessing together in PyPy 4.0.1, which does not appear in Python(2.6 >> or 2.7). Also it does not appear in earlier PyPy versions (2.0.2) >> >> *Reproduce*: >> >> First install python-daemon: >> pypy_pip install python-daemon >> >> Use the following test script (also available in attachment): >> >> #!/usr/bin/pypy >> import daemon >> import multiprocessing >> def test(): >> q = multiprocessing.Queue(64) >> if __name__ == '__main__': >> with daemon.DaemonContext(): >> test() >> >> When executing the script with pypy: >> pypy test.py >> >> The background service does not exit, and is consuming 100% CPU: >> ps aux | grep pypy >> root 7769 99.1 0.5 235332 46812 ? R 17:52 2:09 pypy >> test.py >> root 7775 0.0 0.0 103252 804 pts/1 S+ 17:54 0:00 grep pypy >> >> >> >> Executing the script with python: >> python2.7 test.py >> And the background service normally exits. >> >> *Environment:* >> I'm using CentOS 6.5, with portable PyPy distribution for linux ( >> https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-4.0.1-linux_x86_64-portable.tar.bz2 >> ) >> I run the script on system built-in python (python 2.6.6), a compiled >> CPython (2.7.11), and pypy from epel-release(pypy 2.0.2, python 2.7.2), and >> the problem does not appear. Though the compiled CPython is 2.7.11 and PyPy >> 4.0.4 is python 2.7.10, I think that does not matter much. >> >> Please contact if you have any questions or ideas. >> >> >> 2015-12-23 >> ------------------------------ >> hubo >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev@python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> >> >
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
