hanghangliu commented on code in PR #3546:
URL: https://github.com/apache/gobblin/pull/3546#discussion_r960100702
##########
gobblin-cluster/src/main/java/org/apache/gobblin/cluster/GobblinHelixJobScheduler.java:
##########
@@ -431,7 +431,7 @@ private void
cancelJobIfRequired(DeleteJobConfigArrivalEvent deleteJobArrival) t
if (jobNameToWorkflowIdMap.containsKey(deleteJobArrival.getJobName())) {
String workflowId =
jobNameToWorkflowIdMap.get(deleteJobArrival.getJobName());
TaskDriver taskDriver = new TaskDriver(this.jobHelixManager);
- taskDriver.waitToStop(workflowId, this.helixJobStopTimeoutMillis);
+ taskDriver.stop(workflowId);
Review Comment:
I think the issue actually resides in our codebase, but won't be a
strait-forward fix. The jobRunningMap is parsed to GobblinHelixJobLauncher,
which will constantly pull the helix workflow status and will update the map
based on it. I find current behavior is whenever we call waitToStop, the
workflow goes to stopping state, and
[HelixUtils.waitJobCompletion](https://github.com/apache/gobblin/blob/8c9c8a84ed23c0215c4d80125ac532e97085d76f/gobblin-cluster/src/main/java/org/apache/gobblin/cluster/HelixUtils.java#L278)
will then delete it, so it never really goes to stopped state.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
