[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Eryk Sun : -- nosy: -Alex.Willmer, ahmedsayeed1982 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Eryk Sun : -- Removed message: https://bugs.python.org/msg405712 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Ahmed Sayeed added the comment: imply that the third "n" argument is an additional boundary limit, not the destination buffer capacity (ie. the destination buffer is not implicitly SIZE_MAX), and both source and https://www.webb-dev.co.uk/computers/what-is-ssl-certificate/ destination do

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Ray Donnelly added the comment: I took the liberty of filing this: https://bugs.python.org/issue40263 Cheers. -- nosy: +Ray Donnelly ___ Python tracker ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Steve Dower added the comment: More likely there's been another change to the events that are listened to by multiprocessing, which didn't update the overall limit. File a new bug, please. -- ___ Python tracker

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Mike Hommey added the comment: This is still a problem in python 3.7 (and, I guess 3.8). When not even giving a max_workers, it fails with a ValueError exception on _winapi.WaitForMultipleObjects, with the message "need at most 63 handles, got a sequence of length 63" That happens with

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Ned Deily : -- versions: -Python 3.5, Python 3.6, Python 3.9 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Ned Deily added the comment: New changeset 8ea0fd85bc67438f679491fae29dfe0a3961900a by Ned Deily (Miss Islington (bot)) in branch '3.7': bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132) (GH-13643)

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by miss-islington : -- pull_requests: +13539 pull_request: https://github.com/python/cpython/pull/13643 ___ Python tracker ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Brian Quinlan : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by miss-islington : -- pull_requests: +13117 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Steve Dower added the comment: New changeset 39889864c09741909da4ec489459d0197ea8f1fc by Steve Dower (Brian Quinlan) in branch 'master': bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132) https://github.com/python/cpython/commit/39889864c09741909da4ec489459d0197ea8f1fc

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Brian Quinlan : -- keywords: +patch pull_requests: +13045 stage: needs patch -> patch review ___ Python tracker ___ ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Brian Quinlan added the comment: BTW, the 61 process limit comes from: 63 - - -- ___ Python tracker ___ ___ Python-bugs-list

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Change by Brian Quinlan : -- assignee: -> bquinlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Brian Quinlan added the comment: If no one has short-term plans to improve multiprocessing.connection.wait, then I'll update the docs to list this limitation, ensure that ProcessPoolExecutor never defaults to >60 processes on windows and raises a ValueError if the user explicitly passes a

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Robert Collins added the comment: This is now showing up in end user tools like black: https://github.com/ambv/black/issues/564 -- nosy: +rbcollins versions: +Python 3.7, Python 3.8, Python 3.9 ___ Python tracker

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Changes by Davin Potts : -- nosy: +davin ___ Python tracker ___ ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Changes by Steve Dower : -- stage: -> needs patch type: -> behavior versions: +Python 3.6 ___ Python tracker ___

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Steve Dower added the comment: > Some quick Googling strongly suggests there's no reasonably general way to > overcome the Windows-defined MAXIMUM_WAIT_OBJECTS=64 for implementations that > call the Windows WaitForMultipleObjects(). The recommended way to deal with this is to spin up threads

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Tim Peters added the comment: Just noting that the `multiprocessing` module can be used instead. In the example, add import multiprocessing as mp and change with concurrent.futures.ProcessPoolExecutor() as executor: to with mp.Pool() as executor: That's all it takes.

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Changes by Antoine Pitrou : -- nosy: +sbt ___ Python tracker ___ ___ Python-bugs-list mailing

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

Terry J. Reedy added the comment: The example runs fine, in about 1 second, on my 6 core (which I guess is 12 logical cores) Pentium. I am guessing that the default number of workers needs to be changed, at least on Windows, to min(#logical_cores, 60) -- nosy: +bquinlan, terry.reedy

[issue26903] ProcessPoolExecutor(max_workers=64) crashes on Windows

New submission from Diogo Pereira: I'm using Python 3.5.1 x86-64 on Windows Server 2008 R2. Trying to run the ProcessPoolExecutor example [1] generates this exception: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Program Files\Python35\lib\threading.py", line