findingrish commented on code in PR #17025:
URL: https://github.com/apache/druid/pull/17025#discussion_r1753172217
##########
server/src/main/java/org/apache/druid/segment/metadata/AbstractSegmentMetadataCache.java:
##########
@@ -732,18 +718,35 @@ public Set<SegmentId> refreshSegmentsForDataSource(final
String dataSource, fina
log.debug("Refreshing metadata for datasource[%s].", dataSource);
+ final Set<SegmentId> retVal = new HashSet<>();
+
+ ConcurrentSkipListMap<SegmentId, AvailableSegmentMetadata>
datasourceSegments = segmentMetadataInfo.get(dataSource);
+ // this datasource no longer exists, skip refresh
+ if (datasourceSegments == null) {
+ return retVal;
+ }
+
+ // Skip refreshing tombstone segments. These segments lack data or column
information.
+ // Additionally, segment metadata queries, which are not yet implemented
for tombstone segments
+ // (see: https://github.com/apache/druid/pull/12137) do not provide
metadata for tombstones,
+ // leading to indefinite refresh attempts for these segments.
+ Set<SegmentId> segmentsWithoutTombstone =
Review Comment:
I have refactored to ensure that the stream is not materialized.
For `count` I have added a terminal operation but that wouldn't materialize
the entire stream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
