AmatyaAvadhanula commented on code in PR #17420:
URL: https://github.com/apache/druid/pull/17420#discussion_r1821514854
##########
server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java:
##########
@@ -2900,6 +2929,70 @@ public int removeDataSourceMetadataOlderThan(long
timestamp, @NotNull Set<String
);
}
+ @VisibleForTesting
+ Set<DataSegment> retrieveUsedSegmentsForAllocation(
+ final Handle handle,
+ final String dataSource,
+ final Interval interval
+ )
+ {
+ final Set<SegmentId> overlappingSegmentIds =
SqlSegmentsMetadataQuery.forHandle(handle, connector, dbTables, jsonMapper)
+
.retrieveUsedSegmentIds(
+
dataSource,
+
interval
+ );
+ // Map from version -> interval -> segmentId with the smallest partitionNum
+ Map<String, Map<Interval, SegmentId>> versionIntervalToSmallestSegmentId =
new HashMap<>();
+ for (SegmentId segmentId : overlappingSegmentIds) {
+ final Map<Interval, SegmentId> map
+ =
versionIntervalToSmallestSegmentId.computeIfAbsent(segmentId.getVersion(), v ->
new HashMap<>());
+ final SegmentId value = map.get(segmentId.getInterval());
+ if (value == null || value.getPartitionNum() >
segmentId.getPartitionNum()) {
+ map.put(interval, segmentId);
+ }
+ }
+
+ // Retrieve the segments for the ids stored in the map to get the
numCorePartitions
+ final Set<String> segmentIdsToRetrieve = new HashSet<>();
+ for (Map<Interval, SegmentId> itvlMap :
versionIntervalToSmallestSegmentId.values()) {
Review Comment:
The idea was so that we get a consistent result irrespective of the order in
which the metadata store returns results
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
