freemandealer commented on PR #64186: URL: https://github.com/apache/doris/pull/64186#issuecomment-4661617340
Added a follow-up safeguard for TTL cache directories. Context: the main fix keeps logical TTL changes separate from the on-disk storage expiration, but a defensive cleanup path is still useful for historical leftovers, abnormal interruption, or rare races that leave multiple hash_<storage_expiration> directories for the same hash. Approach: add a low-frequency TTL repair checker with a 12-hour default interval. It scans only the filesystem directory layer to find hashes with duplicate storage-expiration directories, then briefly takes the cache lock only for those suspect hashes to read the in-memory storage binding. Stable hashes enqueue stale directory removal through the existing background GC queue; unbound hashes, downloading/unstable hashes, mixed storage bindings, disabled checker config, and existing cleanup backlog are skipped. This avoids full in-memory traversal under cache_lock and avoids synchronous bulk directory deletion in the checker thread. Validation: ran targeted ASAN BE UTs covering duplicate stale-dir repair, unbound skip, disable-config skip, downloading skip, cleanup-backlog skip, max-repairs-per-round, plus the existing TTL finalize/lazy-load/remove regressions. All 9 BlockFileCacheTest cases passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
