scwhittle commented on code in PR #35120:
URL: https://github.com/apache/beam/pull/35120#discussion_r2226180643
##########
sdks/java/harness/src/main/java/org/apache/beam/fn/harness/FnHarness.java:
##########
@@ -279,7 +279,15 @@ public static void main(
ExecutorService executorService =
options.as(ExecutorOptions.class).getScheduledExecutorService();
ExecutionStateSampler executionStateSampler =
- new ExecutionStateSampler(options, System::currentTimeMillis);
+ new ExecutionStateSampler(
+ options,
+ System::currentTimeMillis,
+ message -> {
+ System.err.println(
+ "\n*** FATAL ERROR: Timeout occurred! Exiting JVM with
status 1. ***");
+ System.err.println("Details: " + message);
+ System.exit(1);
Review Comment:
I was thinking to still have the consumer function, in this case the
FnHarness can use it to notify the CompletableFuture (which otherwise is never
completed).
So all together something like:
```
CompletableFuture<Void> samplerTerminationFuture = new CompletableFuture();
ExecutionStateSampler executionStateSampler =
new ExecutionStateSampler(
options,
System::currentTimeMillis,
message -> {
samplerTerminationFuture.completeExceptionally(...message...);
});
and then
if (options.as(SdkHarnessOptions.class).getEnableLogViaFnApi()) {
CompletableFuture.anyOf(control.terminationFuture(),
logging.terminationFuture(), samplerTerminationFuture).get();
} else {
CompletableFuture.anyOf(control.terminationFuture(),
samplerTerminationFuture).get();
}
```
> Does the normal shutdown path allow to restart the pipeline job?
Yes we restart the process in dataflow if it stops for any reason. The
cleaner shutdown may allow other things to complete first or flush logs better
etc but the unclean shutdown should still be recovered from (and those do occur
in production with ooms etc).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
