Yafeng Lu created CASSSIDECAR-484:
-------------------------------------
Summary: Fix the spurious oldest segment age in
CdcRawDirectorySpaceCleaner
Key: CASSSIDECAR-484
URL: https://issues.apache.org/jira/browse/CASSSIDECAR-484
Project: Sidecar for Apache Cassandra
Issue Type: Improvement
Reporter: Yafeng Lu
{{CdcRawDirectorySpaceCleaner}} re-reads segment metadata via
{{File.lastModified()}} well after the initial {{listFiles()}} scan. If
Cassandra (or any other actor) reclaims a segment in the interim, the
{{File.lastModified()}} will return {{{}0{}}}, and {{nowInMillis - 0}}
collapses to wall-clock milliseconds. This produced three subtle failure modes
on a hot cluster:
# {{oldestSegmentAge}} gauge reported a nonsensical ~50-year age. Because
{{segmentFiles}} is sorted ascending by {{segmentId}} and Cassandra reclaims
oldest-first, the race hits precisely the file whose age is being reported.
# Critical/low CDC-buffer alerts ({{{}criticalCdcRawSpace{}}},
{{{}lowCdcRawSpace{}}}) were silently suppressed. With {{ageMillis}} collapsed
to ~10¹² ms, both {{< criticalMillis}} and {{< lowMillis}} conditions were
always false.
# {{deletedSegment}} metric was over-attributed to the cleaner. When Cassandra
reclaimed a segment first, the cleaner still called {{deleteSegment(...)}} (a
no-op via {{{}Files.deleteIfExists{}}}), logged a misleading "Deleting Cdc
segment lastModified=0" line, and bumped the metric with the cached file size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
