[issue33725] Python crashes on macOS after fork with no exec

Change by Eryk Sun : -- components: +macOS -Library (Lib) nosy: +ned.deily, ronaldoussoren -ahmedsayeed1982 versions: +Python 3.8 -Python 3.7 ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg405703 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue33725] Python crashes on macOS after fork with no exec

Ahmed Sayeed added the comment: $ ./gdb -nx -q --data-directory=data-directory ~/a.out -ex "set confirm off" -ex "file -readnow ~/a.out" http://www-look-4.com/property/houses-in-france/ Reading symbols from /home/simark/a.out... Reading symbols from ~/a.out...

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: I don't think there's really anything more to do here. I'm closing the issue. Let's open a new one if needed at some future point. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed

[issue33725] Python crashes on macOS after fork with no exec

Terry J. Reedy added the comment: Since the default has been different on different systems for as long as I remember, I see no reason to break code on *nix in the name of 'consistency'. Asyncio also works different on different systems. Aside from that idea, is there anything else left

[issue33725] Python crashes on macOS after fork with no exec

Change by B. L. Alterman : -- nosy: +blalterman ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Mouse added the comment: @mark.dickinson, thank you. Following your suggestion, I've added a comment in #28965, and created a new issue https://bugs.python.org/issue40106. -- nosy: +vstinner ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Mark Dickinson added the comment: @Mouse: replace the last block of your code with something like this: # Start worker processes workers = [] for m in range(nProcesses): p = mp.Process(target=getElements, args=(q, tSleep / 1000, m)) workers.append(p)

[issue33725] Python crashes on macOS after fork with no exec

Mouse added the comment: Also, adding `p.join()` immediately after `p.start()` in my sample code showed this timing: ``` $ time python3.8 multi1.py worker 0 done, got 100 numbers worker 1 done, got 0 numbers worker 2 done, got 0 numbers worker 3 done, got 0 numbers real0m2.342s user

[issue33725] Python crashes on macOS after fork with no exec

Mouse added the comment: @mark.dickinson, the issue you referred to did not show a working sample. Could you demonstrate on my example how it should be applied? Thanks! -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Mark Dickinson added the comment: @Mouse: see #28965. The fix for the code you show is to join the child processes before the main process starts exiting. -- nosy: +mark.dickinson ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Mouse added the comment: Tried 'spawn', 'fork', 'forkserver'. - 'spawn' causes consistent `FileNotFoundError: [Errno 2] No such file or directory`; - 'fork' consistently works (tested on machines with 4 and 20 cores); - 'forkserver' causes roughly half of the processes to crash with

[issue33725] Python crashes on macOS after fork with no exec

Mouse added the comment: The fix applied for this problem actually broke multiprocessing on MacOS. The change to the new default 'spawn' from 'fork' causes program to crash in spawn.py with `FileNotFoundError: [Errno 2] No such file or directory`. I've tested this on MacOS Catalina 10.15.3

[issue33725] Python crashes on macOS after fork with no exec

Change by Ryan May : -- nosy: +Ryan May ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: As far as I can tell, the only thing left to do for this issue is to add a documentation warning to the 3.7 documents similar to what was added to 3.8 but without the change in default. A PR would be nice. -- ___

[issue33725] Python crashes on macOS after fork with no exec

miss-islington added the comment: New changeset d74438b633184bbd8d775d7118d6f12f6f825a96 by Miss Islington (bot) in branch '3.8': bpo-33725, multiprocessing doc: rephase warning against fork on macOS (GH-13841) https://github.com/python/cpython/commit/d74438b633184bbd8d775d7118d6f12f6f825a96

[issue33725] Python crashes on macOS after fork with no exec

Change by miss-islington : -- pull_requests: +13726 pull_request: https://github.com/python/cpython/pull/13849 ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: New changeset 1e77ab0a35cf95318bb4893f7253a30f73201163 by Victor Stinner in branch 'master': bpo-33725, multiprocessing doc: rephase warning against fork on macOS (GH-13841) https://github.com/python/cpython/commit/1e77ab0a35cf95318bb4893f7253a30f73201163

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: > Thanks Victor. I don’t think “reliable” is strong enough, since this will > definitely lead to core dumps under certain conditions. What about: (...) That sounds better: I wrote PR 13841. -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Change by STINNER Victor : -- pull_requests: +13718 pull_request: https://github.com/python/cpython/pull/13841 ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On Jun 4, 2019, at 08:11, STINNER Victor wrote: > Ned Deily: >> No, it has *always* been unsafe. What's new as of 10.13/14 is that macOS >> tries much harder at runtime to detect such cases and more predictably cause >> an error rather than letter than let

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: Ned Deily: > No, it has *always* been unsafe. What's new as of 10.13/14 is that macOS > tries much harder at runtime to detect such cases and more predictably cause > an error rather than letter than let the process run on and possibly fail >

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 28, 2019, at 17:38, Ned Deily wrote: > > Ned Deily added the comment: > >> To be clear, what is unsafe on macOS (as of 10.13, but even more so on >> 10.14) is calling into the Objective-C runtime between fork and exec. > > No, it has *always*

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 28, 2019, at 17:21, STINNER Victor wrote: > > > STINNER Victor added the comment: > >> To be clear, what is unsafe on macOS (as of 10.13, but even more so on >> 10.14) is calling into the Objective-C runtime between fork and exec. The >> problem

[issue33725] Python crashes on macOS after fork with no exec

Change by Ned Deily : -- Removed message: https://bugs.python.org/msg343843 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: > To be clear, what is unsafe on macOS (as of 10.13, but even more so on 10.14) > is calling into the Objective-C runtime between fork and exec. No, it has *always* been unsafe. What's new as of 10.13/14 is that macOS tries much harder at runtime to detect such

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: > To be clear, what is unsafe on macOS (as of 10.13, but even more so on 10.14) > is calling into the Objective-C runtime between fork and exec. No, it has *always* been unsafe. What's new as of 10.13/14 is that macOS tries much harder at runtime to detect such

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: > To be clear, what is unsafe on macOS (as of 10.13, but even more so on 10.14) > is calling into the Objective-C runtime between fork and exec. The problem > for Python is that it’s way too easy to do that implicitly, thus causing the > macOS to abort the

[issue33725] Python crashes on macOS after fork with no exec

Gregory P. Smith added the comment: Documentation of the issue for sure! A RuntimeWarning is a maybe if it can be made conditional on the OS version (10.13 and higher?)... when the "fork" method would be used. BUT be very cautious about warnings as they tend to crop up in front of users

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 28, 2019, at 16:43, Ned Deily wrote: > Thanks, I was just going to add that I would accept a doc change for 3.7. > But the wording should be a little clearer that fork has *always* been unsafe > on macOS, i.e. this is not a new issue for 3.7 which

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: > Can we at least document that the default start method (fork) is now unsafe > on macOS? Thanks, I was just going to add that I would accept a doc change for 3.7. But the wording should be a little clearer that fork has *always* been unsafe on macOS, i.e.

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: Ned Deily: > GPS beat me to it: this definitely should not be backported to either 3.7 or > 2.7 since it is a major user behavior change. Hum ok, but test_multiprocessing_fork is now always skipped on macOS in Python 3.7. Can we at least document that the

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: GPS beat me to it: this definitely should not be backported to either 3.7 or 2.7 since it is a major user behavior change. -- versions: -Python 2.7, Python 3.7 ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Gregory P. Smith added the comment: FWIW I am in favor of "spawn" being the default on _all_ platforms in 3.8. The safest option being the default just seems like the right thing to do. -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Gregory P. Smith added the comment: There is a multiprocessing spawn method on Python 2.7. And we should never do such a crazy huge feature backport. IMNSHO - We should not change the default in 3.7 either. That is a notable behavior change. That decision is entirely up to Ned (RM). If

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 27, 2019, at 16:28, STINNER Victor wrote: > I don't see a clear consensus to switch to spawn on *all* platforms, so I > wrote PR 13603 which is the minimum fix: switch to spawn by default, but only > on macOS. Fair enough. Let’s fix what we have

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: Ok, I pushed a minimum fix to make sure that it lands in Python 3.8 beta1. Please check if the documentation if properly updated :-) We will still have time after beta1 to revisit the question of making spawn the default on more/all platforms. I wrote PR

[issue33725] Python crashes on macOS after fork with no exec

Change by STINNER Victor : -- pull_requests: +13525 pull_request: https://github.com/python/cpython/pull/13626 ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: New changeset 17a5588740b3d126d546ad1a13bdac4e028e6d50 by Victor Stinner in branch 'master': bpo-33725: multiprocessing uses spawn by default on macOS (GH-13603) https://github.com/python/cpython/commit/17a5588740b3d126d546ad1a13bdac4e028e6d50 --

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: As soon as the "fork" start method "just works" on Linux, I'm not sure why you want to use "spawn" by default on Linux. I don't buy the "consistency" argument here. I'm not surprised that such "low level" module like multiprocessing behaves differently

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: I don't see a clear consensus to switch to spawn on *all* platforms, so I wrote PR 13603 which is the minimum fix: switch to spawn by default, but only on macOS. If this PR is merged, I understand that it should be applied to 2.7 and 3.7 branches as well.

[issue33725] Python crashes on macOS after fork with no exec

Change by STINNER Victor : -- pull_requests: +13509 pull_request: https://github.com/python/cpython/pull/13603 ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

Change by Tim Smith : -- nosy: +tdsmith ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Ned Deily : -- pull_requests: -13382 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Matthias Bussonnier : -- pull_requests: +13382 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 10, 2019, at 04:07, Josh Rosenberg wrote: > > I've seen far too many cases where Python code targeting Linux intentionally > uses the COW benefits of fork for multiprocessing to think it would be a good > idea to change the default start method

[issue33725] Python crashes on macOS after fork with no exec

Josh Rosenberg added the comment: I've seen far too many cases where Python code targeting Linux intentionally uses the COW benefits of fork for multiprocessing to think it would be a good idea to change the default start method there without *some* sort of deprecation period. --

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: I have no preference for the default on Linux, but please fix the default at least on macOS before Python 3.8 beta 1 ;-) -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On May 5, 2019, at 11:44, Davin Potts wrote: > > Victor raises an important question: should the *default* start behavior be > made consistent across platforms? Yes, I think it should. The pros of consistency and correctness outweigh the breaking

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Victor raises an important question: should the *default* start behavior be made consistent across platforms? Assuming we change it on MacOS, the default start behavior on Windows and MacOS will be spawn but the default start behavior on Linux and FreeBSD

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: I believe we must change the default behavior on MacOS to use spawn instead of fork. Encouraging people to use fork by default on MacOS is encouraging them to create something that effectively will not work. Keeping fork as the default behavior when we have

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: > And that change should be made for every platform so it will affect a much > broader group of users than are affected by *this* issue. Do you mean to stop using fork on Linux as well? Using fork is just fine on Linux, no? --

[issue33725] Python crashes on macOS after fork with no exec

Łukasz Langa added the comment: It's trivial but not safe in the sense that code that previously depended on some global state setup done in the master process right before fork will stop working. If this code is a library that is not in your control, you might not be able to "just revert"

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: Changing the default is trivial and safe. Let's do that. Additional doc wouldn't hurt. -- ___ Python tracker ___

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: As best as I can see, there is no magic bullet to help mitigate this. At a minimum, I am convinced we need to update the documentation to describe this behavior on MacOS and recommend alternatives. I continue to give serious thought to the idea of changing the

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: What's the status of this issue? As far as I know, we haven't come up with any workaround (see msg331011 above) other than disabling our tests. If we don't have anything better, at the very least we should add a warning to the documentation to avoid fork mode

[issue33725] Python crashes on macOS after fork with no exec

Change by Ned Deily : -- pull_requests: -10551 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Ned Deily : -- pull_requests: -10552 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Tal Einat : -- pull_requests: +10551, 10552 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Tal Einat : -- pull_requests: +10551 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On Dec 12, 2018, at 17:59, Ned Deily wrote: > > Ned Deily added the comment: > >> Would it be safe to run the multiprocessing tests on recent macOS with the >> OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable set? > > See Ronald's reply above in

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: > Would it be safe to run the multiprocessing tests on recent macOS with the > OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable set? See Ronald's reply above in msg331101. I believe his point is that there is nothing you can do to make this safe. And

[issue33725] Python crashes on macOS after fork with no exec

STINNER Victor added the comment: Would it be safe to run the multiprocessing tests on recent macOS with the OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable set? -- nosy: +vstinner ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: I think it make sense to disable this test; the only possible modification would be to only disable it for macOS <= 10.13. AFAIK, that's the first version where core dumps were possible. (Aside: I also saw these core dumps for a long time on 10.13 and

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: @ned.deily: Apologies, I misread what you wrote -- I would like to see the random segfaults that you were seeing on Mojave if you can still point me to a few. -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Do we really need to disable the running of test_multiprocessing_fork entirely on MacOS? My understanding so far is that not *all* of the system libraries on the mac are spinning up threads and so we should expect that there are situations where fork alone

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: Since it looks like multiprocessing_fork is not going to be fixable for macOS, the main issue remaining is how to help users avoid this trap (literally). Should we add a check and issues a warning or error at run time? Or is a doc change sufficient? In the

[issue33725] Python crashes on macOS after fork with no exec

miss-islington added the comment: New changeset df5d884defc8f1a94013ff9beb493f1428bd55b5 by Miss Islington (bot) in branch '3.6': bpo-33725: skip test_multiprocessing_fork on macOS (GH-11043) https://github.com/python/cpython/commit/df5d884defc8f1a94013ff9beb493f1428bd55b5 --

[issue33725] Python crashes on macOS after fork with no exec

miss-islington added the comment: New changeset d4bcf13e06d33b8ec66a68db20df34a029e66882 by Miss Islington (bot) in branch '3.7': bpo-33725: skip test_multiprocessing_fork on macOS (GH-11043) https://github.com/python/cpython/commit/d4bcf13e06d33b8ec66a68db20df34a029e66882 -- nosy:

[issue33725] Python crashes on macOS after fork with no exec

Change by miss-islington : -- pull_requests: +10280 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: New changeset ac218bc5dbfabbd61c76ce8a17de088611e21981 by Ned Deily in branch 'master': bpo-33725: skip test_multiprocessing_fork on macOS (GH-11043) https://github.com/python/cpython/commit/ac218bc5dbfabbd61c76ce8a17de088611e21981 --

[issue33725] Python crashes on macOS after fork with no exec

Change by miss-islington : -- pull_requests: +10281 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue33725] Python crashes on macOS after fork with no exec

Change by Ned Deily : -- keywords: +patch pull_requests: +10279 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-list

[issue33725] Python crashes on macOS after fork with no exec

Ronald Oussoren added the comment: AFAIK there is nothing you can do between after calling fork(2) to "reinitialise" the ObjC runtime. And I don't think that's the issue anyway: I suspect that the actual problem is that Apple's system frameworks use multithreading (in particular

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: I have a reliable way to call *something* in the pthread_atfork prepare handler, but I honestly don't know what to call to prevent the crash. In the Ruby thread, it seemed to say that you could just dlopen

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: On Nov 14, 2018, at 10:11, Davin Potts wrote: > > > Davin Potts added the comment: > > Barry's effort as well as comments in other links seem to all suggest that > OBJC_DISABLE_INITIALIZE_FORK_SAFETY is not comprehensive in its ability to > make other

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Given the original post mentioned 2.7.15, I wonder if it is feasible to fork near the beginning of execution, then maintain and pass around a multiprocessing.Pool to be used when needed instead of dynamically forking? Working with legacy code is almost always

[issue33725] Python crashes on macOS after fork with no exec

Ned Deily added the comment: _scproxy has been known to be problematic for some time, see for instance Issue31818. That issue also gives a simple workaround: setting urllib's "no_proxy" environment variable to "*" will prevent the calls to the System Configuration framework. --

[issue33725] Python crashes on macOS after fork with no exec

Antoine Pitrou added the comment: Legacy code is easy to migrate as long as it uses Python 3. Just call mp.set_start_method('forkserver') at the top of your code and you're done. Some use cases may fail (if sharing non-picklable types), but they're probably not very common. --

[issue33725] Python crashes on macOS after fork with no exec

Davin Potts added the comment: Barry's effort as well as comments in other links seem to all suggest that OBJC_DISABLE_INITIALIZE_FORK_SAFETY is not comprehensive in its ability to make other threads "safe" before forking. "Objective-C classes defined by the OS frameworks remain

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: FWIW, I suspect that setting the environment variable only helps if it's done before the process starts. You cannot set it before the fork and have it affect the child. -- ___ Python tracker

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: A few other things I don't understand: * Why does setting OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES only seem to work when it's set in the shell before the parent process executes? AFAICT, it does *not* work if you set that in os.environ in the parent

[issue33725] Python crashes on macOS after fork with no exec

Barry A. Warsaw added the comment: Hoo boy. I'm not sure I have the full picture, but things are starting to come into focus. After much debugging, I've narrowed down at least one crash to urllib.request.getproxies(). On macOS (darwin), this ends up calling _scproxy.get_proxies() which

[issue33725] Python crashes on macOS after fork with no exec

Change by Barry A. Warsaw : -- title: Pytho crashes on macOS after fork with no exec -> Python crashes on macOS after fork with no exec ___ Python tracker ___