[issue42281] Inconsistent ProcessPoolExecutor behaviour on macOS between 3.7 and 3.8/9

[Ronald Oussoren]

Ronald Oussoren  added the comment:

In 3.8 the spawn method for multiprocessing changed from "fork" to "spawn" (see 
https://docs.python.org/3/whatsnew/3.8.html#multiprocessing).

A side effect of this is that the module gets executed again in the child 
processes (the same as on Windows).

The reason for this change is that multiple higher level APIs, some of which 
used by the Python implementation, are are not safe to use with the "fork" 
spawning strategy (as in: causing crashing when using the fork strategy).

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42281] Inconsistent ProcessPoolExecutor behaviour on macOS between 3.7 and 3.8/9

[Mustafa Quraish]

New submission from Mustafa Quraish :

The code attached produces weird behaviour on macOS (Catalina at least) on 
Python3.8 / Python3.9. Issue has been reproduced on at least one other mac 
belonging to a friend. Does not occur on Linux as far as I have tested. I have 
simplified the code as much as I can, so the example attached does not actually 
need to use parallelism.

Code description: Read text from a file, change directory using `os.chdir`, 
spawn 5 processes and pass the `print` function as the handler along with the 
text.

Expectation: File contents should be printed 5 times

Result: Code tries to load (again) the file in each thread, but fails since the 
directory was changed.


Precondition: Have `file.txt` in the same directory containing some text

Output when run on Python3.7, I get the following (expected output):

```
Reading file...
Text loaded was: 'file contents'
file contents
file contents
file contents
file contents
file contents
```

When running on 3.8 / 3.9 (on a mac) I get the following:

```
Reading file...
Text loaded was: 'file contents'
Error: /Users/mustafa/dev/ptest/test_dir/file.txt does not exist.
Error: /Users/mustafa/dev/ptest/test_dir/file.txt does not exist.
Error: /Users/mustafa/dev/ptest/test_dir/file.txt does not exist.
```

It seems to me that even opened `file.txt` is loaded in the main thread and 
text is read from it, this value isn't passed into the spawned processes and 
they are calling `load()` again (which is where the error message is produced).

--
components: macOS
files: test.py
messages: 380483
nosy: bquinlan, mustafaquraish, ned.deily, pitrou, ronaldoussoren
priority: normal
severity: normal
status: open
title: Inconsistent ProcessPoolExecutor behaviour on macOS between 3.7 and 3.8/9
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49579/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com