sakshichitnis27 opened a new pull request, #20813:
URL: https://github.com/apache/druid/pull/20813
Fixes #20752.
### Description
`HeapMemoryDatasourceSegmentCache.markSegmentsWithinIntervalAsUnused`
selected segments with an overlap lookup, while
`SqlSegmentsMetadataQuery.markSegmentsUnused` only touches used segments that
are fully contained in the requested interval. When the requested interval was
smaller than a segment (for example one hour inside a DAY segment), the
metadata store left the segment as used and returned `numChangedSegments: 0`,
but the cache marked it unused. With the incremental cache enabled, the segment
then went missing from the Coordinator's view until the next restart.
#### Only mark fully contained segments as unused in the cache
The cache now skips segments that are not fully contained in the requested
interval, which matches the SQL behaviour.
Added
`testMarkSegmentsWithinIntervalAsUnused_ignoresPartiallyOverlappingSegments` to
`HeapMemoryDatasourceSegmentCacheTest`. It covers an interval inside a segment,
an interval straddling two segments, and an interval fully containing a
segment. The test fails on master (returns 1 instead of 0 for the first case)
and passes with this change.
#### Release note
Fixed an issue where marking segments unused with an interval smaller than a
segment could remove that segment from the incremental segment metadata cache
even though it stayed used in the metadata store.
<hr>
##### Key changed/added classes in this PR
* `HeapMemoryDatasourceSegmentCache`
* `HeapMemoryDatasourceSegmentCacheTest`
<hr>
This PR has:
- [x] been self-reviewed.
- [x] a release note entry in the PR description.
- [x] added unit tests or modified existing tests to cover new code paths,
ensuring the threshold for [code
coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md)
is met.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
