On Sun, 21 Oct 2018 19:58:05 +0200
Vladimir Filipović <hemf...@gmail.com>
wrote:
>
> To anticipate a couple more possible questions:
>
> - What would this proposal do about multiple producers/consumers
> needing to jointly decide _when_ to close the queue?
>
> Explicitly nothing.
>
> The queue's state is either closed or not, and it doesn't care who
> closed it. It needs to interact correctly with multiple consumers and
> multiple producers, but once any one piece of code closes it, the
> correct interaction is acting like a closed queue for everybody.
Ah. This is the one statement that makes me favorable to this idea.
When there is a single consumer, it's easy enough to send a sentinel.
But when there are multiple consumers, suddenly you must send exactly
the right number of sentinels (which means you also have to careful
keep track of their number, which isn't always easy). There's some
delicate code doing exactly that in concurrent.futures.
> - Should multiprocessing.Queue do the same thing?
>
> I think so, though I'm not proposing it.
>
> It already has a close() method, whose meaning is very similar but not
> identical to (a subset of) the proposed threading.Queue.close's
> meaning (with resource-management effects not relevant to
> threading.Queue either way).
Not really similar, unfortunately. mp.Queue.close() isn't a logical
thing, but releases the queue's internal resources. It doesn't signal
consumers that the producers has finished with the queue.
Perhaps if you renamed close() to something else ("put_eof" or
"put_end" perhaps?) that would allow porting it to mp.Queue?
Regards
Antoine.
_______________________________________________
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
