snleee commented on a change in pull request #6094:
URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r513753755
##########
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/TaskGeneratorUtils.java
##########
@@ -68,4 +73,30 @@
}
return runningSegments;
}
+
+ public static Map<String, List<String>> getScheduledSegmentsMap(@Nonnull
String taskType,
+ @Nonnull ClusterInfoProvider clusterInfoProvider) {
+ Map<String, List<String>> scheduledSegments = new HashMap<>();
+ Map<String, TaskState> taskStates =
clusterInfoProvider.getTaskStates(taskType);
+ for (Map.Entry<String, TaskState> entry : taskStates.entrySet()) {
+ // Skip COMPLETED tasks
+ if (entry.getValue() == TaskState.COMPLETED) {
+ continue;
+ }
Review comment:
If the task got stuck, we are cleaning up the segment lineage data after
one day (the retention manager runs the clean up for segment lineage).
Once the segment lineage is updated, the segments from the old task will be
rescheduled. If we reschedule the segment here before the segment lineage gets
cleaned up, the minion task will anyway fail because it won't be able to update
segment lineage because we don't allow the same segment appear on the
`segmentsFrom` field twice.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org
For additional commands, e-mail: commits-h...@pinot.apache.org
