phoerious commented on a change in pull request #16658:
URL: https://github.com/apache/beam/pull/16658#discussion_r811967341
##########
File path: sdks/python/apache_beam/runners/worker/worker_pool_main.py
##########
@@ -51,6 +51,27 @@
_LOGGER = logging.getLogger(__name__)
+def kill_process_gracefully(proc, timeout=10):
+ """
+ Kill a worker process gracefully by sending a SIGTERM and waiting for it to
finish.
+ A SIGKILL will be sent if the process has not finished after ``timeout``
seconds.
+ """
+ def _kill():
+ proc.terminate()
+ t = time.time()
+ while time.time() < t + timeout:
+ time.sleep(0.01)
+ if proc.poll() is not None:
+ return
+ _LOGGER.warning('Worker process did not respond, killing it.')
+ proc.kill()
+ proc.wait() # Avoid zombies
+
+ kill_thread = threading.Thread(target=_kill)
+ kill_thread.start()
+ kill_thread.join()
Review comment:
If we can guarantee that the worker pool is only ever using the `boot`
binary, then we can even reduce this to a simple `SIGTERM`, because `boot`
gives the same treatment to its Python subprocesses and should never hang
indefinitely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
