On Sep 4, 2019, at 04:21, Chris Simmons <chris.simmon...@hotmail.com> wrote:
I have seen deployed servers that wrap an Executor with a Semaphore to add this functionality (which is mildly silly, but not when the “better” alternative is to subclass the Executor and use knowledge of its implementation intervals…). Which implies that this feature would be helpful in real life code. But not quite as described: > It might be a good idea to add a "block" option to Executor.submit > (https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.submit) > that allows the caller to block until a worker is free to handle the > request. I believe that this would be trivial to implement by passing the > "block" option through to the Executor's internal Queue.put call > (https://github.com/python/cpython/blob/242c26f53edb965e9808dd918089e664c0223407/Lib/concurrent/futures/thread.py#L176). No, that won’t work. Queue.put _already_ defaults to blocking. The reason it doesn’t block here is that Executor creates an unbounded Queue (or, actually, a SimpleQueue), so it’s never full, so it can never block. More generally, if you want blocking queue behavior, you inherently need to to specify a maximum length somewhere. The Executor can’t guess what maximum length you might want (since usually you don’t want _any_ max), so you’d need to add that to the constructor, say an optional max_queued_jobs param. If passed, it creates a Queue(max_queued_jobs) instead of a SimpleQueue(). And once you do that, submit automatically blocks when the Queue is full, so you’re done. Also, the block option is not a choice between blocking vs. ignoring the bounds and succeeding immediately, it’s a choice between blocking and _failing_ immediately. I don’t think that choice is likely to be as useful for executors as for queues, so I don’t think you need (or want) to add anything to the submit API. If you _did_ want to add something anyway, that would be a problem. The submit method takes *args, **kw and forwards them to fn. If you add any additional param, you lose the ability to submit functions that have a param with the same name. Worse, there are common cases where you build further forwarding functions around the submit method. So you might accidentally create situations where code mysteriously starts blocking in 3.9 when it worked as expected in 3.8, and it’s not even clear why. But you could solve all of these problems by just adding a second submit_nowait method instead of adding a block=True parameter to submit. A timeout might be more useful than plain nowait. But I suspect you’d want to always use the same timeout for a single executor, so if that is worth adding, maybe that should be another constructor param, not another submit variant. But anyway, without a compelling use case, or anyone asking for it, I think YAGNI wins here. We don’t have to add timeouts just because we’re adding blocking. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PVHWVYS6LG3UDWOML6G6N7BERDPVIBMP/ Code of Conduct: http://python.org/psf/codeofconduct/
