[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Charles-François Natali added the comment: Greg, the original issue was about an OS X failure, and it's still a problem AFAICT. However it's definitely not a problem with your patch, but an OS X kernel bug (we've had another similar issue some time ago I think), so we might as well skip the offending tests on this OS X version. As for close() and EINTR, if you plan to do the change, it would probably be interesting to check if there are other such patterns in the stdlib. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
jan matejek added the comment: i don't think that's it, look at the result output in comment #183756: the list of fds is correct, except that they aren't closed as they should be. i can't reproduce the result when running code by hand, so it's either a race or an issue with test setup. so far i figured out that this goes away when patch from issue #16962 is reverted -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Charles-François Natali added the comment: i don't think that's it, look at the result output in comment #183756: the list of fds is correct, except that they aren't closed as they should be. Of course the list is correct: the message is printed by the parent process, which opened the FDs ;-) i can't reproduce the result when running code by hand, so it's either a race or an issue with test setup. If it's random, yes. One way to debug this would be to run the test with strace -f. so far i figured out that this goes away when patch from issue #16962 is reverted Note that the patch doesn't retry getdents64() on EINTR, and retries close() on EINTR while it's not advised, see this post by Linus: http://article.gmane.org/gmane.linux.kernel/330619 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
jan matejek added the comment: so apparently the problem was that we're using the borked original patch for issue #16962, instead of the fixed version that's actually in repos. (The d_type field was missing, so every filename had an extra character prepended to it. AFAICT, all mysterious circumstances that I reported would trace back to workplace mess, like running the test in a terminal with a different build than i thought...) sorry for taking your time, folks, the bug can be closed now. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Gregory P. Smith added the comment: phew, yay. because i didn't see anything obviously wrong with the #16962 commit. also, regarding checking for EINTR on some of the close() calls. It isn't a big deal. This _posixsubprocess.c code that contains those checks is always executing single-threaded post-fork so there is no chance of a race condition even though the EINTR check and retry are pointless as Linus mentions. They will never cause errant behavior but I'll fix them in a future commit for consistency. -- resolution: - invalid status: open - closed type: - behavior versions: +Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Charles-François Natali added the comment: Hummm... For those experiencing failures when /proc is mounted: do you have a grsecurity-patched kernel? If RBAC is enabled, /proc/self/fd is empty, so this approach won't work... -- nosy: +neologix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
jan matejek added the comment: Ismail, the sem_open thing is because in autobuild you don't get /proc or somesuch - it's a problem of our buildsystem, otherwise the test passes. I can reproduce the failure in autobuild, but not separately yet. Will keep you posted, signs point to yet another misfeature of our buildsystem, but maybe it's something with test ordering... -- nosy: +matejcik ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Ismail Donmez added the comment: Jan, thanks for the excellent explanation. We can close this bug I guess then? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
jan matejek added the comment: Not just yet, please. I can now reproduce the test_subprocess failures outside autobuild too. Interestingly, it's the reverse here: test_subprocess passes when /proc is not present, but fails when it is. I'm pretty sure that we hijacked a completely different bug with a similar symptom :e) but whatever. I'll just go ahead and debug this. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Gregory P. Smith added the comment: fyi - subprocess uses /proc to get a list of open fd's in the child process before exec when possible. search for FD_DIR in http://hg.python.org/cpython/file/d674bbbed333/Modules/_posixsubprocess.c -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Gregory P. Smith added the comment: Some way for me to reproduce this is needed. -- stage: - test needed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Ezio Melotti added the comment: Ismail, does it always fail or is it a sporadic failure? -- nosy: +ezio.melotti ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +gregory.p.smith ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Ismail Donmez added the comment: I can reproduce the error locally. But I see something in the log might be related to this (possibly not but anyway): [352/368/1] test_multiprocessing test_multiprocessing skipped -- This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770. This is weird because pyconfig.h has HAVE_SEM_OPEN set to 1. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Gregory P. Smith added the comment: fyi - sem_open is unrelated, nothing in subprocess needs that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds() and test_pass_fds in test_subprocess
Changes by Ned Deily n...@acm.org: -- resolution: out of date - status: closed - open title: test_subprocess.test_close_fds() sporadic failures on Mac OS X Tiger - sporadic failures of test_close_fds() and test_pass_fds in test_subprocess versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12466] sporadic failures of test_close_fds and test_pass_fds in test_subprocess
Changes by Ned Deily n...@acm.org: -- title: sporadic failures of test_close_fds() and test_pass_fds in test_subprocess - sporadic failures of test_close_fds and test_pass_fds in test_subprocess ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12466 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
