[issue32146] multiprocessing freeze_support needed outside win32

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Oliver Newman]

Change by Oliver Newman :


--
nosy: +onew

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Antoine Pitrou]

Antoine Pitrou  added the comment:

@bbayles Hopefully a Windows user could test if you give a step-by-step guide 
of what to check for.

--
nosy: +paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Ronald Oussoren]

Change by Ronald Oussoren :


--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

bbayles  added the comment:

I'm currently having difficulty getting either cx_Freeze or PyInstaller to work 
with Python 3.8, which is hindering testing. I also no longer have easy access 
to a Windows environment.

If I change the implementation to use -m instead of -c would you be able to 
test?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Antoine Pitrou]

Antoine Pitrou  added the comment:

The key here would be to replace function execution with module execution and 
command-line arguments.

So instead of:

  python -c 'from multiprocessing.forkserver import main; main(ARG1, ARG2...)'

you would do:

  python -m multiprocessing.forkserver ARG1 ARG2 

Then it should become easy to extract the concatenated command lines.

(and if that works, please do the same for semaphore_tracker ;-))

--
versions: +Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

bbayles  added the comment:

Thanks for the note. I've merged in master and fixed a conflict in the test 
file.

In an earlier rev I tried to do the argument parsing without ast.literal_eval, 
but found it awkward to support all the ways [1] can manifest. You might have a 
better idea?


[1] 
https://github.com/python/cpython/blob/3.7/Lib/multiprocessing/forkserver.py#L106-L107

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[miss-islington]

miss-islington  added the comment:


New changeset b9cd38f928f7eb4e18ad4b63e5c49c05c626c33e by Miss Islington (bot) 
in branch '3.7':
bpo-32146: Add documentation about frozen executables on Unix (GH-5850)
https://github.com/python/cpython/commit/b9cd38f928f7eb4e18ad4b63e5c49c05c626c33e


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[miss-islington]

Change by miss-islington :


--
pull_requests: +11064, 11065, 11066

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[miss-islington]

Change by miss-islington :


--
pull_requests: +11064, 11065

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[miss-islington]

Change by miss-islington :


--
pull_requests: +11064

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[STINNER Victor]

STINNER Victor  added the comment:


New changeset bab4bbb4c9cd5d25ede21a1b8c99d56e3b8dae9d by Victor Stinner (Bo 
Bayles) in branch 'master':
bpo-32146: Add documentation about frozen executables on Unix (GH-5850)
https://github.com/python/cpython/commit/bab4bbb4c9cd5d25ede21a1b8c99d56e3b8dae9d


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I'm sorry about not seeing your PR before.  Would you like to update it against 
current git master?

(note I'm not thrilled by the use of the "ast" module; I hope we can change the 
command-line args so that this isn't needed)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

Change by bbayles :


--
pull_requests: +5626

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

bbayles  added the comment:

I presume it's too late to get my pull request in for 3.7, but perhaps we could 
get a note in the documentation for this before release?

That could be as simple as: 

.. warning::

The ``'spawn'`` and ``'forkserver'`` start methods cannot currently be used 
with "frozen" executables (i.e., binaries produced by packages like 
**PyInstaller** and **cx_Freeze**) on Unix. The ``'fork'`` start method does 
work.

I will make that into a PR if it seems like a good option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

Change by bbayles :


--
keywords: +patch
pull_requests: +5049
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[bbayles]

bbayles  added the comment:

I ran into this issue, and found references to it on StackOverflow [1] and 
GitHub [2] as well. I found that the problem applies to both the 'spawn' and 
'forkserver' start methods on Linux.

I made an attempt to implement dancol's fix above. (1) and (3) are 
straightforward, but if there's an elegant way to do the "tickle" in (2) I 
wasn't able to figure it out?

My branch [3] has something that seems to work with the 'spawn' method and 
cx_Freeze. Perhaps someone could look at that and tell me if I'm on a 
reasonable track? If so, I can attempt to extend the method to cover 
'forkserver' as well.

[1] https://stackoverflow.com/q/47325297/353839

[2] https://github.com/anthony-tuininga/cx_Freeze/issues/264

[3] 
https://github.com/python/cpython/compare/master...bbayles:bpo-32146-freeze_support

--
nosy: +bbayles

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thank you for reporting this.  Would you like to submit a PR?

--
nosy: +davin, pitrou
stage:  -> needs patch
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32146] multiprocessing freeze_support needed outside win32

[Daniel Colascione]

New submission from Daniel Colascione :

multiprocessing's freeze_support makes freshly-launched subprocesses integrate 
correctly until the multiprocessing ecosystem even when the main executable is 
some application binary instead of a Python interpreter. The documentation and 
code assume that this support is needed only on win32, but it's equally 
applicable to POSIX systems using the spawn strategy in combination with 
systems like cx_freeze.

In particular:

1) The special case in context.py's version of freeze_support() for win32 
should be removed

2) The semaphore tracker should correctly tickle the freeze support

3) The documentation should be updated to be platform-neutral

--
components: Library (Lib)
messages: 307043
nosy: dancol
priority: normal
severity: normal
status: open
title: multiprocessing freeze_support needed outside win32
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com