[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: Thanks. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Martin Panter added the comment: Opened Issue 26762 about the leftover processes. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: Since the initial issue is fixed (test failures when using -Werror), I suggest to close this issue. > BTW the spurious background processes are still there (visible if you run > “ps” or “pstree” immediately after the test finishes), but they no longer >

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Martin Panter added the comment: Sorry, but I don’t think I have the time/motivation/expertise to look into the fix in detail. But there are no warnings or failures anymore when I use -Werror, so I am happy! BTW the spurious background processes are still there (visible if you run “ps” or

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: I'm not sure why Python 2.7 was in the Versions. I just ran "python2.7 -Werror -m test.regrtest -v test_multiprocessing" => no error. ResourceWarning doesn't exist in Python 2. Is it still something to do on multiprocessing about ResourceWarning? I didn't

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: > Lets open new issue for this. I opened the issue #26642 "Replace stdout and stderr with simple standard printers at Python exit". -- ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: > Oops, I forgot to discuss the second part :-) You also added "sys.__stderr__ > = sys.stderr" in multiprocessing. At the start of PyImport_Cleanup() sys.stdXXX is set to sys.__stdXXX__. If sys.stdXXX is not sys.__stdXXX__ and here is the last reference to

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: Serhiy: > Lets open new issue for this. I agree with PyImport_Cleanup() deserves a dedicated issue. Myself: > This change has two parts. Oops, I forgot to discuss the second part :-) You also added "sys.__stderr__ = sys.stderr" in multiprocessing. I don't

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: Lets open new issue for this. -- ___ Python tracker ___ ___

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: ASCII/backslashescape is maybe a better encoding for this printer. -- ___ Python tracker ___

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: By the way, I'm not sure that using UTF-8/backslashescape is the encoding :-/ It's common to use Latin1 on a terminal. UTF-8 is very rare on Windows for the console for example. -- ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: > I don't know. I don't know why OSError and ValueError are caught at all. Ha ha, me neither. Maybe it's time to remove them an wait for user feedback :-) > What are you think about deterministic closing standard streams in > cleanup_std_streams.patch? There

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: > What do you think? Is it better to keep stdin if stdin.close() raises an > exception? I don't know. I don't know why OSError and ValueError are caught at all. What are you think about deterministic closing standard streams in leanup_std_streams.patch?

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: > The change should be backported to Python 3.5. I'm waiting for buildbots and > I have to read again the change later. Oh, I didn't notice that 3.5 buildbots were also all red. So I backported the fix immediatly. --

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Roundup Robot added the comment: New changeset 357d1ed928fe by Victor Stinner in branch '3.5': Issue #25654: https://hg.python.org/cpython/rev/357d1ed928fe -- ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: I pushed a change based on multiprocessing_no_close_fd.patch. I changed multiprocessing.util._close_stdin() to try to replace sys.stdin with os.devnull even if stdin.close() raised an exception. Since stdin is read-only, I don't expect an exception on

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Roundup Robot added the comment: New changeset 9faa03e9cea8 by Victor Stinner in branch 'default': Issue #25654: https://hg.python.org/cpython/rev/9faa03e9cea8 -- nosy: +python-dev ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: Oh, with fixes for issues #21925 and #26637, test_multiprocessing_spawn started to fail: http://buildbot.python.org/all/builders/AMD64%20Debian%20root%203.5/builds/727/steps/test/logs/stdio

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

STINNER Victor added the comment: I marked the issue #10305 as a duplicate of this issue. -- ___ Python tracker ___

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Changes by STINNER Victor : -- nosy: +haypo ___ Python tracker ___ ___

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: Here is an alternative patch that uses closefd=False. -- Added file: http://bugs.python.org/file41075/multiprocessing_no_close_fd.patch ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: Good point! The question is whether we want to see resource warnings when override standard streams with open files (usually without closefd=False)? -- ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: You can commit your patch right now (it shouldn't make things worse), and continue to work on additional tests. -- ___ Python tracker

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: Oh, I forgot about issue21779. And there was at least yet one duplicate issue23827. Thanks for the reminder Davin. The patch in issue21779 fixes only symptoms, makes failed test more lenient. I prefer to got rid of resource warnings at all. --

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Davin Potts added the comment: Note issue21779 has been around for quite some time with its own patch specifically for multiprocessing, making its tests more robust. I am hesitant to embrace the patch suggested here. -- nosy: +davin ___ Python

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Serhiy Storchaka added the comment: sys.stdin is set to open(os.devnull) in multiprocessing.forkserer and multiprocessing.process (see issue5313). At shutdown stage sys.std* streams are restored to sys.__std*__. Here the resource warning is emitted. An example that demonstrates similar issue:

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

Martin Panter added the comment: Perhaps we don’t normally see a warning because stdin normally has closefd=False set? -- ___ Python tracker ___

[issue25654] test_multiprocessing_spawn ResourceWarning with -Werror

New submission from Martin Panter: Running the test suite with -Werror enabled causes test_multiprocessing_spawn to print out lots of unhandled errors from the garbage collector, and one of these causes a test failure. It looks like there are lots of files that should be explicitly closed