Michael Hrivnak <[email protected]> added the comment:
My project also has a use case for this, very similar to the others described.
Here's what we want:
with ThreadPoolExecutor(queue_size=500) as executor:
for item in parse_a_long_list_of_work(somefile.xml):
executor.submit(Job(item))
I do not want to parse the entire list of work items and load them into memory
at once. It is preferable for the main thread running the above code to block
on submit() when the queue size is above some threshold.
It's a classic case of the producer and consumer operating at different speeds.
In the past, a Queue object has been the way to connect such a producer and
consumer. The various Executor classes do not provide an easy way to consume
from a provided Queue object, so giving them that capability would be a
reasonable alternative to having the submit() method block.
----------
nosy: +mhrivnak
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue14119>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
