dlesco commented on issue #22191: URL: https://github.com/apache/airflow/issues/22191#issuecomment-1083342338
This looks like a better approach than what I was thinking. One comment on the `_kill_process` change. self._process is an instance of multiprocessing.Process. An alternative to using `os.waitpid` is to use [self._process.join(optional_timeout)](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.Process.join), which is multiprocessing's higher-level interface to the OS waitpid code. (Python versions >=3.7 also support a kill() method on the multiprocessing Process, but probably os.kill was used in order to support older versions of Python). The question is whether to use a timeout or not for the join. Because SIGKILL was used, doing a join afterwards *should* always work, because KILL should always work (unlike SIGTERM, SIGTERM might not end the process). I don't know if the community wants to be extra cautious, and have a timeout on the join, and if a timeout occurs, abnormally end the whole process with an exception, because the OS did something unexpected. Abending would be better than hanging forever. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
