Roundup Robot added the comment:
New changeset 00d09afb57ca by Nick Coghlan in branch 'default':
Issue #19946: use public API for multiprocessing start methods
http://hg.python.org/cpython/rev/00d09afb57ca
--
___
Python tracker rep...@bugs.python.org
Nick Coghlan added the comment:
Pending a clean bill of health from the stable buildbots :)
--
status: open - pending
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
Nick Coghlan added the comment:
Now passing on all the stable buildbots (the two red Windows bots are for other
issues, such as issue 15599 for the threaded import test failure)
--
status: pending - closed
___
Python tracker rep...@bugs.python.org
Roundup Robot added the comment:
New changeset 460961e80e31 by Nick Coghlan in branch 'default':
Issue #19946: appropriately skip new multiprocessing tests
http://hg.python.org/cpython/rev/460961e80e31
--
___
Python tracker rep...@bugs.python.org
Christian Heimes added the comment:
The OpenIndiana tests are still failing. OpenIndiana doesn't support forkserver
because it doesn't implement the send handle feature. The patch skips the
forkserver tests if HAVE_SEND_HANDLE is false.
--
Added file:
Nick Coghlan added the comment:
I think that needs to be fixed on the multiprocessing side rather than just
in the tests - we shouldn't create a concrete context for a start method
that isn't going to work on that platform. Finding that kind of discrepancy
was part of my rationale for basing the
Richard Oudkerk added the comment:
On 19/12/2013 10:00 pm, Nick Coghlan wrote:
I think that needs to be fixed on the multiprocessing side rather than just
in the tests - we shouldn't create a concrete context for a start method
that isn't going to work on that platform. Finding that kind of
Nick Coghlan added the comment:
Ah, I should have looked more closely at the docs to see if there was a public
API for that before poking around in the package internals.
In that case, I suggest we change this bit in the test:
# We look inside the context module to find out which
#
Christian Heimes added the comment:
The commit broken a couple of buildbots like all Windows bots and OpenIndiana.
--
nosy: +christian.heimes
resolution: fixed -
status: closed - open
___
Python tracker rep...@bugs.python.org
Zachary Ware added the comment:
The problem on Windows at least is that the skips for the 'fork' and
'forkserver' start methods aren't firing due to setUpClass being improperly set
up in MultiProcessingCmdLineMixin: it's not decorated as a classmethod and the
'u' is lower-case instead of
Nick Coghlan added the comment:
OK, fixed test case attached. Turns out the ipython workaround test was
completely wrong and never even loaded multiprocessing, and hence always
passed, even with the workaround disabled. So I fixed that test case, and used
the same approach for the zipfile,
Changes by Nick Coghlan ncogh...@gmail.com:
Removed file:
http://bugs.python.org/file33161/test_multiprocessing_main_handling.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
Changes by Nick Coghlan ncogh...@gmail.com:
Removed file:
http://bugs.python.org/file33146/issue19946_pep_451_multiprocessing.diff
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
Roundup Robot added the comment:
New changeset b6d6f3b4b100 by Nick Coghlan in branch 'default':
Close #19946: use runpy as needed in multiprocessing
http://hg.python.org/cpython/rev/b6d6f3b4b100
--
resolution: - fixed
stage: patch review - committed/rejected
status: open - closed
Changes by Nick Coghlan ncogh...@gmail.com:
Removed file:
http://bugs.python.org/file33163/test_multiprocessing_main_handling.py
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
Richard Oudkerk added the comment:
Thanks for your hard work Nick!
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
___
Python-bugs-list
Nick Coghlan added the comment:
I created a test suite to ensure that all the various cases were handled
correctly by the eventual patch (it doesn't test some of the namespace package
related edge cases, but they devolve to normal module execution in terms of the
final state of __main__, and
Nick Coghlan added the comment:
Current work in progress patch. The existing multiprocessing tests all pass,
but the new main handling tests fail.
The fork start_method passes all the tests
The forkserver and spawn start methods fail the directory, zipfile and package
tests.
--
Added
Nick Coghlan added the comment:
Updated test that handles timeouts better.
I also realised the current test failures are due to an error in the test
design - the failing cases are ones where we deliberately *don't* rerun
__main__ because the entire __main__.py file is assumed to be inside an
Changes by Nick Coghlan ncogh...@gmail.com:
--
stage: needs patch - patch review
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
___
Olivier Grisel added the comment:
I applied issue19946_pep_451_multiprocessing_v2.diff and I confirm that it
fixes the problem that I reported initially.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
Nick Coghlan added the comment:
Issue 19700 isn't quite finished, but I believe it is finished enough to let me
get this working properly.
--
dependencies: -Update runpy for PEP 451
resolution: fixed -
___
Python tracker rep...@bugs.python.org
Richard Oudkerk added the comment:
So there are really two situations:
1) The __main__ module *should not* be imported. This is the case if you use
__main__.py in a package or if you use nose to call test_main().
This should really be detected in get_preparation_data() in the parent process
Nick Coghlan added the comment:
Bumping the priority on this, as multiprocessing is currently creating invalid
child processes by failing to set __main__.__spec__ appropriately.
The attached patch is designed to get us started down that path. It's currently
broken, but I need feedback from
Nick Coghlan added the comment:
With the restructuring in my patch, it would be easy enough to move the early
return cases from the _fixup_main_* functions to instead be don't set the
variable cases in get_preparation_data.
--
___
Python tracker
Richard Oudkerk added the comment:
I appear to be somehow getting child processes where __main__.__file__ is
set, but __main__.__spec__ is not.
That seems to be true for the __main__ module even when multiprocessing is not
involved. Running a file /tmp/foo.py containing
import sys
Nick Coghlan added the comment:
Scripts (whether in source form or precompiled) work via direct execution,
but all the other execution paths (directories, zipfiles, -m) rely on the
import system (via runpy). multiprocessing has been broken for years in
that regard, hence my old comment about the
Changes by Nick Coghlan ncogh...@gmail.com:
--
title: Have multiprocessing raise ImportError when spawning a process that
can't find the main module - Handle a non-importable __main__ in
multiprocessing
___
Python tracker rep...@bugs.python.org
Changes by Nick Coghlan ncogh...@gmail.com:
--
assignee: - ncoghlan
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19946
___
___
Python-bugs-list
29 matches
Mail list logo
