[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: Thanks Cornelius Diekmann for your fix ;-) -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: New changeset 20cbc1d29facead32c2da06c81a09af0e057015c by Victor Stinner in branch '2.7': bpo-31158: Fix nondeterministic read in test_pty (#3808) (#3853)

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: New changeset 66fb5ef3bb9e36187a0e5052dfd99899447df671 by Victor Stinner (Miss Islington (bot)) in branch '3.6': [3.6] bpo-31158: Fix nondeterministic read in test_pty (GH-3808) (GH-3852)

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Change by STINNER Victor : -- pull_requests: +3833 ___ Python tracker ___ ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: > I prefer Cornelius’s current proposal (rev 4f8137b) Ok, fine. I merged his PR 3852. I created backports to 2.7 and 3.6. -- versions: +Python 2.7, Python 3.6 ___ Python tracker

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Change by Roundup Robot : -- pull_requests: +3832 ___ Python tracker ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: New changeset e6f62f69f07892b993910ff03c9db3ffa5cb9ca5 by Victor Stinner (Cornelius Diekmann) in branch 'master': bpo-31158: Fix nondeterministic read in test_pty (#3808)

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Martin Panter added the comment: I prefer Cornelius’s current proposal (rev 4f8137b) because it fixes both sites, rather than just patching the immediate problem site. I don’t think read(1) is a big problem, just less efficient. But if you prefer to do larger reads,

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: Yes, I get the AssertionError with the latest version of PR 3802. From the high load avg of my system, you can see that the error occurs very rarely and that I need to stress my system to trigger it. With PR 3802, the error

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: Hum, let me clarify: my main goal is to get a stable CI. I don't really care of the exact implementation of one specific unit test in test_pty ;-) -- ___ Python tracker

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: Cornelius:Here is a PoC: I checked out your code and stressed my system with it. (...) AssertionError: ..." Did you get this failure with my PR 3802? Cornelius: "We have b'For my pet fish, Eric.\r\n' in the pty buffer. We read

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Change by Cornelius Diekmann : -- pull_requests: +3792 ___ Python tracker ___ ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: I also added a pull request which I cannot get to fail on my system. What do you think, haypo? Sorry for the noise! -- ___ Python tracker

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: Oh, one more thing, since it is unclear whether read or write is causing the error. I replaced os.write(slave_fd, TEST_STRING_1) by pty._writen(slave_fd, TEST_STRING_1) which makes sure all bytes are written. After some time,

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: >> Does reading the string b'I wish to buy a fish license.\n' not cause a >> problem, too? > >This string TEST_STRING_1 is used for a single os.write() call, whereas >TEST_STRING_2 is splitted and written in two parts with two

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: > Does reading the string b'I wish to buy a fish license.\n' not cause a > problem, too? This string TEST_STRING_1 is used for a single os.write() call, whereas TEST_STRING_2 is splitted and written in two parts with two os.write()

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: Regarding PR 3802: * Does reading the string b'I wish to buy a fish license.\n' not cause a problem, too? * Is reading len(expected) bytes the correct behavior for systems where normalize_output is needed? I also fixed these

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Change by STINNER Victor : -- keywords: +patch pull_requests: +3788 stage: -> patch review ___ Python tracker ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: Oh, in fact, it's not that hard to reproduce the failure on Linux: haypo@selma$ ./python -m test test_pty -m test_basic -F Run tests sequentially 0:00:00 load avg: 2.17 [ 1] test_pty 0:00:00 load avg: 2.17 [ 2] test_pty 0:00:00

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: Failure on x86 Gentoo Refleaks 3.x too: http://buildbot.python.org/all/builders/x86%20Gentoo%20Refleaks%203.x/builds/104/steps/test/logs/stdio == FAIL: test_basic

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Changes by Xavier de Gaye : -- nosy: +xdegaye ___ Python tracker ___ ___ Python-bugs-list

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Cornelius Diekmann added the comment: I observed the same issue, but the problem occurs mainly when reading data. In my proposed patch in issue29070, I use the existing pty._writen() to make sure all data is written. As Martin mentioned, reading is a problem. My patch proposes

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Charalampos Stratakis added the comment: I've observed the exact same issue on Fedora at random times. -- nosy: +cstratak ___ Python tracker ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Martin Panter added the comment: Checking for short writes is worthwhile, but in Issue 29070 it looks like Cornelius identified the main problem was short _reads_. See the parts of his patch to do with “_os_read_exactly” and related functions. -- nosy: +Cornelius Diekmann,

[issue31158] test_pty: test_basic() fails randomly on Travis CI

Changes by STINNER Victor : -- pull_requests: +3070 ___ Python tracker ___ ___

[issue31158] test_pty: test_basic() fails randomly on Travis CI

STINNER Victor added the comment: I don't know why test_pty fails. https://github.com/python/cpython/pull/3037 makes sure that all data is written into the master fd using a new write_all() helper function which loops until all data is written: handle os.write() partial write. --

[issue31158] test_pty: test_basic() fails randomly on Travis CI

New submission from STINNER Victor: https://travis-ci.org/python/cpython/jobs/262606831 0:13:26 load avg: 134.81 [256/403] test_pty test test_pty failed -- Traceback (most recent call last): File "/home/travis/build/python/cpython/Lib/test/test_pty.py", line 109, in test_basic