Dan O'Reilly added the comment:
It seems like everyone agrees that this functionality is useful, so I'm
reviving this in hopes of getting a patch pushed through. I've updated Andreas'
patch so that it applies cleanly against the latest tree, and tweaked the
handling of exceptions in
Andreas van Cranenburgh added the comment:
Giampaolo, this patch is for ProcessPoolExecutor as well.
About keyboard interrupts, if my tests are correct, they work
in Python 3.3+ with both multiprocessing and concurrent.futures.
(Although for the latter I have to hit ctrl-c twice).
--
Giampaolo Rodola' added the comment:
Related:
https://groups.google.com/forum/#!topic/dev-python/ytbYwHXKC6o
I'm not sure how what is proposed here would be useful for ThreadPoolExecutor
but it would definitely be helpful being able to set an initializer for
ProcessPoolExecutor because right
Changes by Martin Dengler mar...@martindengler.com:
--
nosy: +mdengler
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21423
___
___
Mark Dickinson added the comment:
BTW, I think there's a design mistake in the EnhancedThreadPoolExecutor that's
worth avoiding in any std. lib. implementation: the initialiser and
uninitialiser for the EnhancedThreadPoolExecutor accept no arguments. In
retrospect, it would have been better
Andreas van Cranenburgh added the comment:
Here's a patch. I have added initializer and initargs keywords to both
ThreadPoolExecutor and ProcessPoolExecutor, with the same semantics as
multiprocessing.Pool.
I couldn't figure out what to do if the initializer fails with a
ProcessPoolExecutor:
Andreas van Cranenburgh added the comment:
Here's a version with tests. Detecting an execption in the initializer works
with ProcessPoolExecutor, but not with ThreadPoolExecutor.
--
Added file: http://bugs.python.org/file35181/pool_initializer_tests.patch
Changes by Ned Deily n...@acm.org:
--
nosy: +bquinlan
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue21423
___
___
Python-bugs-list mailing list
Josh Rosenberg added the comment:
Do you mean ProcessPoolExecutor? Thread based pools don't involve serialization.
It would be good for ThreadPoolExecutor to have it as well, to make it easier
to switch between executors, but only ProcessPoolExecutor is suffering from
serialization overhead.
Andreas van Cranenburgh added the comment:
Yes I did mean ProcessPoolExecutor, but indeed, it's good to have for threads
as well.
I could try to make a patch if it is likely that it would be accepted.
--
___
Python tracker rep...@bugs.python.org
Josh Rosenberg added the comment:
I'm not a core developer, but writing the patch is usually considered helpful.
Two notes:
1. Make sure to write unit tests for any new behavior
2. I'd suggest making any such argument keyword-only; if we move closer to the
Java executor model, that means
New submission from Andreas van Cranenburgh:
It would be useful if concurrent.futures.ThreadPoolExecutor took an initializer
argument, like multiprocessing.Pool.
This is useful for example to load a large dataset once upon initialization of
each worker process, without have to pass the
12 matches
Mail list logo
