chris-fast commented on code in PR #17909:
URL:
https://github.com/apache/dolphinscheduler/pull/17909#discussion_r2741115851
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkParameters.java:
##########
@@ -114,6 +114,21 @@ public class FlinkParameters extends AbstractParameters {
*/
private String rawScript;
+ /**
+ * Shutdown on attached exit (-sae parameter)
+ *
+ * <p>When enabled, Flink CLI will attempt to shutdown the cluster when
the CLI
+ * terminates abruptly. This is only suitable for attached mode
(CLUSTER/LOCAL).
+ *
+ * <p>For APPLICATION mode, this should typically be disabled as the job
runs
+ * in detached mode on YARN.
+ *
+ * <p>Default: false (disabled for safety)
+ *
+ * @see FlinkArgsUtils#buildRunCommandLine
+ */
+ private Boolean shutdownOnAttachedExit;
Review Comment:
I every **agree with you on the compatibility point**，**keeping the default
as TRUE** is definitely the safest move.
And I want to clarify a few things regarding the underlying mechanism,
though:
1.**It's not really a "Flink bug"**，-sae parameter was designed to prevent
resource leaks (zombie clusters), not to handle duplicate submissions.
2.Relying on -sae=true to prevent "double runs" is actually pretty
unreliable. If a worker hits a hard crash (like an OOM or power outage), the
CLI dies instantly and never gets a chance to send the shutdown signal to the
cluster. So, the job keeps running, and a retry will still cause a duplicate.
3.**The better way to handle idempotency is via YARN Application Tags (e.g.,
using the ProcessInstanceId) and checking if that tag exists before
submitting.**
I think that "idempotency check" deserves to be a future optimization
feature on its own. It’s probably better to keep it out of this current PR so
we don't block the merge.
Thanks a lot for the feedback—I actually learned a ton digging into this!
I’d be more than happy to help contribute code for that future optimization
feature, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
