[issue23285] PEP 475 - EINTR handling

Changes by Berker Peksag : Removed file: http://bugs.python.org/file43674/b3b439d7dd40.diff ___ Python tracker ___

[issue23285] PEP 475 - EINTR handling

Changes by mostafa shahdadi : Added file: http://bugs.python.org/file43674/b3b439d7dd40.diff ___ Python tracker ___

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: For the record, it seems test_eintr sometimes left zombie processes in my machine where I run reference leak tests every night. I didn't investigate and just disabled the tests. (I'll add that that machine is hosted on an OpenVZ-based VPS, so perhaps

[issue23285] PEP 475 - EINTR handling

Changes by STINNER Victor victor.stin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: test_setitimer.patch: hack unittest.TestCase.run() to inject signals every 1 ms, send the first signal in 100 ms. This patch helped me to find that socket.connect() doesn't handle EINTR yet: see issue #23618 PEP 475: handle EINTR in the socket module.

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: (I'll add that that machine is hosted on an OpenVZ-based VPS, so perhaps there are issues with the old patched kernel and whatnot?) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: For the record, it seems test_eintr sometimes left zombie processes in my machine where I run reference leak tests every night. I didn't investigate and just disabled the tests. -- ___ Python tracker

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: Would be nice to have a test for it; There are already some tests on EINTR in test_file_eintr and test_subprocess (test_communicate_eintr()). -- ___ Python tracker rep...@bugs.python.org

[issue23285] PEP 475 - EINTR handling

Roundup Robot added the comment: New changeset cad6eac598ec by Victor Stinner in branch 'default': Issue #23285: Fix handling of EINTR in fileio.c https://hg.python.org/cpython/rev/cad6eac598ec -- ___ Python tracker rep...@bugs.python.org

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: @Victor: please commit. Would be nice to have a test for it; -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: Note: I found the bug while working on a patch for #22181. My test is this shell script: $ while true; do ./python -c 'import os, signal; signal.setitimer(signal.ITIMER_REAL, 0.001, 0.0001); signal.signal(signal.SIGALRM, lambda *args: print(., end=));

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: The change on Modules/_io/fileio.c is wrong: functions may return None with an exception set. It is wrong because a function must return a result with no exception set, or NULL and an exception set. Attached patch fixes this issue. -- Added file:

[issue23285] PEP 475 - EINTR handling

Roundup Robot added the comment: New changeset 5b63010be19e by Charles-François Natali in branch 'default': Issue #23285: PEP 475 -- Retry system calls failing with EINTR. https://hg.python.org/cpython/rev/5b63010be19e -- nosy: +python-dev ___ Python

[issue23285] PEP 475 - EINTR handling

Roundup Robot added the comment: New changeset 000bbdf0ea76 by Ned Deily in branch 'default': Issue #23285: Install new test directory. https://hg.python.org/cpython/rev/000bbdf0ea76 -- ___ Python tracker rep...@bugs.python.org

[issue23285] PEP 475 - EINTR handling

Changes by Martin Panter vadmium...@gmail.com: -- nosy: +vadmium ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___ ___ Python-bugs-list

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: Would it be possible to push the latest patch right now It's ok for me. Please watch the buildbots :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: Antoine Pitrou added the comment: Would it be possible to push the latest patch right now It's ok for me. Please watch the buildbots :) Cool, I'll push on Friday evening or Saturday. -- ___ Python

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: Would it be possible to push the latest patch right now, and fix remaining issues (if there are known issues?), before Python 3.4 alpha 1? According to the PEP 478, the alpha 1 is scheduled for this sunday (February 8, 2015). --

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: PEP is now updated. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___ ___ Python-bugs-list mailing list

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: I just realized I didn't retry upon EINTR for open(): eintr-4.diff adds this, along with tests (using a fifo). Also, I added comments explaining why we don't retry upon close() (see e.g. http://lwn.net/Articles/576478/ and

[issue23285] PEP 475 - EINTR handling

STINNER Victor added the comment: Charles-François Natali added the comment: I just realized I didn't retry upon EINTR for open(): eintr-4.diff adds this, along with tests (using a fifo). Also, I added comments explaining why we don't retry upon close() (see e.g.

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: With eintr-2.diff, fast!: Victory \°/. Instrumented test_send, 3 socket.send calls, many socket.recv_into calls: Yep, that's expected. I think we should keep the default socket buffer size: it increases the test coverage, and it's probably not

[issue23285] PEP 475 - EINTR handling

Ned Deily added the comment: eintr-1.diff doesn't seem to make any significant difference from eintr.diff on my system. It's still pegging a CPU at 100% and takes 7 minutes wall time to complete. $ time ./python ~/Projects/PyDev/active/dev/3x/source/Lib/test/eintrdata/eintr_tester.py

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: OK, it turns out the culprit was repeated calls to BytesIO.getvalue(), which forced large allocations upon reception of every message. The patch attached fixes this (without changing the socket buffer size). -- Added file:

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: eintr-1.diff doesn't seem to make any significant difference from eintr.diff on my system. It's still pegging a CPU at 100% and takes 7 minutes wall time to complete. Alright, enough played: the patch attached uses a memoryview and

[issue23285] PEP 475 - EINTR handling

Ned Deily added the comment: With eintr-2.diff, fast!: $ time ./python ~/Projects/PyDev/active/dev/3x/source/Lib/test/eintrdata/eintr_tester.py test_read (__main__.OSEINTRTest) ... ok test_wait (__main__.OSEINTRTest) ... ok test_wait3 (__main__.OSEINTRTest) ... ok test_wait4

[issue23285] PEP 475 - EINTR handling

Changes by Antoine Pitrou pit...@free.fr: -- stage: - patch review versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: Victor, do you think there's anything left to do? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___ ___

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: I added a few prints to the send and receive loops of _test_send. When running on a reasonably current Debian testing Linux: Thanks, that's what I was suspecting, but I really don't understand why 200ms isn't enough for a socket write to actually

[issue23285] PEP 475 - EINTR handling

Ned Deily added the comment: It turns out the times are not important; the hangup is the default size of the socket buffers on OS X and possibly BSD in general. In my case, the send and receive buffers are 8192, which explains why the chunks written are so small. I somewhat arbitrarily

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: Charles-François Natali added the comment: Hmmm... Basically, with a much smaller socket buffer, we get much more context switches, which increases drastically the test runtime. But I must admit I'm still really surprised by the time it takes on

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: It turns out the times are not important; the hangup is the default size of the socket buffers on OS X and possibly BSD in general. In my case, the send and receive buffers are 8192, which explains why the chunks written are so small. Hmmm...

[issue23285] PEP 475 - EINTR handling

Ned Deily added the comment: I added a few prints to the send and receive loops of _test_send. When running on a reasonably current Debian testing Linux: ./python Lib/test/eintrdata/eintr_tester.py test_read (__main__.OSEINTRTest) ... ok test_wait (__main__.OSEINTRTest) ... ok test_wait3

[issue23285] PEP 475 - EINTR handling

Charles-François Natali added the comment: The review diff is weird: it seems it contains changes that aren't EINTR-related (see e.g. argparse.rst). Here's a manually generated diff. -- Added file: http://bugs.python.org/file37802/eintr.diff ___

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: Perhaps Ned can help on the OS X side of things. -- nosy: +ned.deily ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___

[issue23285] PEP 475 - EINTR handling

Antoine Pitrou added the comment: The review diff is weird: it seems it contains changes that aren't EINTR-related (see e.g. argparse.rst). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285

[issue23285] PEP 475 - EINTR handling

Ned Deily added the comment: (It may be several days before I can spend much time on it but I will take a look.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___

[issue23285] PEP 475 - EINTR handling

Changes by Charles-François Natali cf.nat...@gmail.com: -- title: PEP 475 - EINTR hanndling - PEP 475 - EINTR handling ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23285 ___