Is your concern something like the following?
with Pool(8) as p:
gen = p.imap_unordered(func, ls)
first_elem = next(gen)
p.apply_async(long_func, x)
remaining_elems = [elem for elem in gen]
...here, if we store "func" on each worker Process as a global, and execute
this pattern ab
On Sun, Oct 14, 2018 at 12:15 PM Jeff Allen wrote:
>
>
> On 10/10/2018 00:06, Steven D'Aprano wrote:
>
> On Tue, Oct 09, 2018 at 09:37:48AM -0700, Jeff Hardy wrote:
>
> ...
>
> From an alternative implementation point of view, CPython's behaviour
> *is* the spec. Practicality beats purity and all
Would this change the other pool method behavior in some way if the user,
for whatever reason, mixed techniques?
imap_unordered will only block when nexting the generator. If the user
mingles nexting that generator with, say, apply_async, could the change
you're proposing have some side-effect?
O
@Nataniel this is what I am suggesting as well. No cacheing - just storing
the `fn` on each worker, rather than pickling it for each item in our
iterable.
As long as we store the `fn` post-fork on the worker process (perhaps as
global), subsequent calls to Pool.map shouldn't be effected (referenci
