On Jan 2, 2020, at 20:40, Miguel Ángel Prosper <miguelangel.pros...@gmail.com> wrote: > > I think it would be very helpful to have an additional argument (cancel for > example) added to Executor.shutdown that cancels all pending futures > submitted to the executor.
> Then context manager would gain the ability to abort all futures incase of a > exception, additionally this would also implement the missing cuterpart of > the multiprocessing module terminate, currently we only have close. But terminate kills the process abruptly—skipping with/finally and atexit and internal interpreter cleanup, orphaning any children, etc. This is rarely what you want, especially not if you’re sharing a queue and a lock with the process. And I think you can’t cleanly abort existing futures once you’ve done that; you have to mark them as being in an unknown state, not call any callbacks, and probably raise if the parent does anything but discard them uninspected. Also, there’s no way to do the same thing for threads—and if you could, it would leave the process in a corrupted state. Having a way to clear the queue and then shutdown once existing jobs are done is a lot more manageable. But it’s not terminate; there’s still a probably short but potentially unbounded wait for shutdown to finish. (Or, if you haven’t designed your jobs to be appropriately small, a probably _long_ wait.) So the only clean way to do this is cooperative: flush the queue, send some kind of message to all children telling them to finish as quickly as possible, then wait for them to finish. If you’re looking for a way to do a “graceful shutdown if possible but after N seconds just go down hard” the way, e.g., Apache or nginx does, I don’t think that could be safely done killing child threads in Python. (In C, you can write code to handle any emergency situation up to even having a trashed heap and stack if you’re careful, but in Python, you can’t do that; you can’t interpret a single Python instruction inside such an emergency.) But since you’re shutting down as quickly as possible after the N seconds are up, you can just daemonize the whole pool, do your emergency shutdown code, and exit or abort, after which the daemon threads get safely killed immediately. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SZSQS63IGHBATXBCS6L6GZQ56YN3GNZP/ Code of Conduct: http://python.org/psf/codeofconduct/
