alirezazamani commented on a change in pull request #1468:
URL: https://github.com/apache/helix/pull/1468#discussion_r513075635
##########
File path: helix-core/src/main/java/org/apache/helix/task/JobDispatcher.java
##########
@@ -478,4 +482,61 @@ private TaskAssignmentCalculator
getAssignmentCalculator(JobConfig jobConfig,
}
return new FixedTargetTaskAssignmentCalculator(assignableInstanceManager);
}
+
+ /**
+ * Find the tasks that have been removed from job config, add them to
tasksToDrop. If task's
+ * currentState and pending message have been removed, delete the task from
job context.
+ * @param jobName
+ * @param jobConfig
+ * @param jobContext
+ * @param currentInstanceToTaskAssignments
+ * @param tasksToDrop
+ * @param currStateOutput
+ * @param allPartitions
+ */
+ private void handleDeletedTasks(String jobName, JobConfig jobConfig,
JobContext jobContext,
+ Map<String, SortedSet<Integer>> currentInstanceToTaskAssignments,
+ Map<String, Set<Integer>> tasksToDrop, CurrentStateOutput
currStateOutput,
+ Set<Integer> allPartitions) {
+ if (TaskUtil.isGenericTaskJob(jobConfig)) {
+ // Get all partitions existed in the context
+ Set<Integer> contextPartitions = jobContext.getPartitionSet();
+ // Check whether the tasks have been deleted from jobConfig
+ for (Integer partition : contextPartitions) {
+ String partitionID = jobContext.getTaskIdForPartition(partition);
+ if (!jobConfig.getTaskConfigMap().containsKey(partitionID)) {
+ boolean hasCurrentState = false;
+ for (Map.Entry<String, SortedSet<Integer>> instanceToPartitions :
currentInstanceToTaskAssignments
Review comment:
I think that would make sense. However, I changed the structure of the
code. If we assume that there won't be many removed tasks, then my new code
would make more sense. In the new structure, if there is no task being removed,
we don't loop at all. So overall, if not much task is being removed (which is
mostly the case), the current implementation would be faster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
