[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Ronald Oussoren added the comment: I’ve filed an issue with Apple about this: FB8903019 -- ___ Python tracker ___ ___

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Ronald Oussoren added the comment: Having thought about this a little more: I agree, we shouldn’t hard code a limit. I’m not against working around misfeatures, but in this case that’s hard to do: Even if we’d limit the size of a single shared memory segment the user can create a number of

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Christian Heimes added the comment: I'm strong -1 on any kind of arbitrary memory limit. Python is used on all sorts of hardware from RPi to super computers. 1 TB sounds like a lot, but it really is not. These days you can buy workstation computers with more than 1 TB RAM. I would be ok

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Change by Vinay Sharma : -- keywords: +patch pull_requests: +21002 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21877 ___ Python tracker ___

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Ronald Oussoren added the comment: A workaround in the implementation of multiprocessing.SharedMemory is IMHO acceptable, tweaking os.ftruncate less so. Note that a Python script can already cause problems on systems by using APIs as intended (such as using shutil.rmtree on the user's home

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Vinay Sharma added the comment: I have 8GB of ram and 128 GB of hard disk. Now, creating a shared memory segment of size 10^12 (1 terabyte) somehow succeeds. Creating a shared memory segment of 10^15 (1 petabyte), mmap (not ftruncate) throws an error stating cannot allocate memory.

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Ronald Oussoren added the comment: I expect that this is more a problem with how memory management works on macOS, the freeze and crash is likely an indication of eager allocation of memory by the system. It is far from sure if implementing a workaround is feasible, the upper limit for

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Change by Ned Deily : -- components: +macOS nosy: +ned.deily, ronaldoussoren ___ Python tracker ___ ___ Python-bugs-list mailing

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Change by Vinay Sharma : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Vinay Sharma added the comment: Hi, I tried replicating this by truncating normal files but that doesn't crash. The above mentioned call of ftruncate only crashes for when the file descriptor passed points to a shared memory segment. And only, multiprocessing.shared_memory is currently

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Davin Potts added the comment: My sense is that it would be nice if we can catch this before ftruncate does something nasty. Where else is ftruncate used in CPython that this could similarly trigger a problem? How is it handled there (or not)? --

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

Change by STINNER Victor : -- title: MacOS crashes by running attached Python code -> multiprocessing.shared_memory: MacOS crashes by running attached Python code ___ Python tracker