kfaraz commented on PR #20027: URL: https://github.com/apache/druid/pull/20027#issuecomment-5312256964
Thanks for the changes, @cecemei ! I was under the impression that the code already ignores segments that cross the skip interval boundaries. https://github.com/apache/druid/blob/cea0d9950482b565da93351f887016022cf1d760/server/src/main/java/org/apache/druid/server/compaction/DataSourceCompactibleSegmentIterator.java#L418-L424 The above snippet is from the method `DataSourceCompactibleSegmentIterator.findInitialSearchInterval()`. This method uses the `SegmentTimeline` (which is already aligned to the target segment granularity) to look up segments that need to be compacted. If any segment does not lie fully within the lookup interval, that segment will not be picked for compaction. But since the lookup interval has no overlap with any of the skip intervals (due to the `filterSkipIntervals` step performed earlier), we can be sure that none of the segments that fully lie in the lookup interval have any overlap with any of the skip intervals. @cecemei , did you encounter/write up a test case which currently fails due to misaligned granularities? I think it would be worth checking that out first. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
