zhangyue19921010 commented on a change in pull request #10676:
URL: https://github.com/apache/druid/pull/10676#discussion_r544091878
##########
File path:
indexing-service/src/main/java/org/apache/druid/indexing/common/task/batch/parallel/ParallelIndexSupervisorTask.java
##########
@@ -491,6 +497,31 @@ private boolean isParallelMode()
return isParallelMode(baseInputSource, ingestionSchema.getTuningConfig());
}
+ /**
+ * Attempt to wait for indexed segments to become available on the cluster.
+ * @param reportsMap Map containing information with published segments that
we are going to wait for.
+ */
+ private void waitForSegmentAvailability(Map<String, PushedSegmentsReport>
reportsMap)
+ {
+ ArrayList<DataSegment> segmentsToWaitFor = new ArrayList<>();
+ reportsMap.values()
+ .forEach(report -> {
+ segmentsToWaitFor.addAll(report.getNewSegments());
+ });
+ ExecutorService availabilityExec =
Execs.singleThreaded("ParallelTaskAvailabilityWaitExec");
+ try {
+ segmentAvailabilityConfirmationCompleted = waitForSegmentAvailability(
+ toolbox,
+ availabilityExec,
+ segmentsToWaitFor,
+
ingestionSchema.getTuningConfig().getAwaitSegmentAvailabilityTimeoutMillis()
Review comment:
nit: Could we extract
`ingestionSchema.getTuningConfig().getAwaitSegmentAvailabilityTimeoutMillis()`
at class level if possible?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
