mxm commented on a change in pull request #11084: [BEAM-9474] Improve
robustness of BundleFactory and ProcessEnvironment
URL: https://github.com/apache/beam/pull/11084#discussion_r391247459
##########
File path:
runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ProcessManager.java
##########
@@ -186,31 +186,28 @@ private void stopProcess(String id, Process process) {
LOG.debug("Attempting to stop process with id {}", id);
// first try to kill gracefully
process.destroy();
- long maxTimeToWait = 2000;
- if (waitForProcessToDie(process, maxTimeToWait)) {
- LOG.debug("Process for worker {} shut down gracefully.", id);
- } else {
- LOG.info("Process for worker {} still running. Killing.", id);
- process.destroyForcibly();
+ long maxTimeToWait = 500;
Review comment:
It's not always shutting down gracefully but that's what the change is
about: removing processes and ensuring a quick recovery time. It's a trade-off.
Ideally we would want to allow more time but if we wait 2 seconds with an SDK
parallelism of 16, that's already more than half a minute waiting time. We
really want to do the process removal in parallel. I'll look into this.
I'm not sure the ProcessManager is a good place to document the shutdown
behavior. If you have any suggestions though, I'll add them here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
