Admaing commented on issue #16789:
URL:
https://github.com/apache/dolphinscheduler/issues/16789#issuecomment-5887806430
I'd like to work on this. What I'll change:
1. Cancel the Flink job through the Flink CLI when the task is stopped,
instead of only killing the
client process tree.
- batch `FlinkTask` currently never calls `flink cancel`;
- `FlinkStreamTask` calls it, but with an unresolved `${FLINK_HOME}`
(`ProcessBuilder` does not
expand shell variables), so it always throws `Cannot run program
"${FLINK_HOME}/bin/flink"`.
2. Resolve `${FLINK_HOME}` for the cancel and savepoint commands (task
params first, then the
environment variable).
3. Keep the current behaviour as a fallback: kill the process tree and
cancel the YARN/K8s
application when the task was submitted to a resource manager, when there
is no JobID, or when the
CLI fails.
4. Wait for the cancel command to finish, and add unit tests for the command
resolution and the
cancel/fallback logic.
Verification: on current `dev` a batch task (`deployMode=cluster`, detached
submit) never issues
`flink cancel` and a stream task always throws; with the change both run
`flink cancel <jobId>`. I
verified this by driving the real classes in a Linux container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
