[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

STINNER Victor added the comment: Nobody managed to find a solution in 3 years. I close the issue. -- dependencies: -multiprocessing.Pool._worker_handler(): use SIGCHLD to be notified on worker exit, multiprocessing: ApplyResult.get() hangs if the pool is terminated resolution: ->

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

Change by Srinivas Reddy Thatiparthy(శ్రీనివాస్ రెడ్డి తాటిపర్తి) : -- pull_requests: +15744 pull_request: https://github.com/python/cpython/pull/10564 ___ Python tracker ___

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

STINNER Victor added the comment: _worker_handler has two issues: * It polls the worker status every status every 100 ms: I created bpo-35493 to investigate how to avoid that * After close() or terminate() has been called, it loops until self._cache is empty. I would like to use

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

STINNER Victor added the comment: > My PR 11136 doesn't work: _maintain_pool() should be called frequently to > check when a worker completed. Polling worker exit status seems inefficient > :-( I created bpo-35493: "multiprocessing.Pool._worker_handler(): use SIGCHLD to be notified on

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

STINNER Victor added the comment: My PR 11136 doesn't work: _maintain_pool() should be called frequently to check when a worker completed. Polling worker exit status seems inefficient :-( asyncio uses SIGCHLD signal to be notified when a child process completes. SafeChildWatcher calls

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

STINNER Victor added the comment: Attached PR 11136 modify _worker_handler() loop to wait on threading.Event events, so Pool.join() completes as soon as possible. Example: --- import multiprocessing import time def the_test(): start_time = time.monotonic() pool =

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

Change by STINNER Victor : -- keywords: +patch pull_requests: +10368 stage: -> patch review ___ Python tracker ___ ___

[issue35479] multiprocessing.Pool.join() always takes at least 100 ms

New submission from STINNER Victor : The join() method of multiprocessing.Pool calls self._worker_handler.join(): it's a thread running _handle_workers(). The core of this thread function is: while thread._state == RUN or (pool._cache and thread._state != TERMINATE):