jihoonson commented on a change in pull request #8257: Add support for parallel 
native indexing with shuffle for perfect rollup
URL: https://github.com/apache/incubator-druid/pull/8257#discussion_r314482462
 
 

 ##########
 File path: 
indexing-service/src/main/java/org/apache/druid/indexing/common/task/AbstractBatchIndexTask.java
 ##########
 @@ -377,6 +403,44 @@ static Granularity 
findGranularityFromSegments(List<DataSegment> segments)
     }
   }
 
+  /**
+   * Creates shard specs based on the given configurations. The return value 
is a map between intervals created
+   * based on the segment granularity and the shard specs to be created.
+   * Note that the shard specs to be created is a pair of {@link 
ShardSpecFactory} and number of segments per interval
+   * and filled only when {@link #isGuaranteedRollup} = true. Otherwise, the 
return value contains only the set of
+   * intervals generated based on the segment granularity.
+   */
+  protected static Map<Interval, Pair<ShardSpecFactory, Integer>> 
createShardSpecWithoutInputScan(
+      GranularitySpec granularitySpec,
+      IndexIOConfig ioConfig,
+      IndexTuningConfig tuningConfig,
+      PartitionsSpec nonNullPartitionsSpec
+  )
+  {
+    final Map<Interval, Pair<ShardSpecFactory, Integer>> allocateSpec = new 
HashMap<>();
+    final SortedSet<Interval> intervals = 
granularitySpec.bucketIntervals().get();
+
+    if (isGuaranteedRollup(ioConfig, tuningConfig)) {
+      // Overwrite mode, guaranteed rollup: shardSpecs must be known in 
advance.
+      assert nonNullPartitionsSpec instanceof HashedPartitionsSpec;
 
 Review comment:
   Added a comment.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org
For additional commands, e-mail: commits-h...@druid.apache.org

Reply via email to