freemandealer opened a new pull request, #64186: URL: https://github.com/apache/doris/pull/64186
## Motivation In branch-4.0, File Cache still uses the same TTL expiration value for two different purposes: - logical lifecycle state, such as TTL queue membership and TTL GC decisions; - physical storage location, because disk directories are named as `hash_expiration`. That coupling lets multiple paths mutate the same metadata. A normal read with a stale or different TTL context can rename or recreate cache directories for an already cached hash, while tablet meta sync and TTL GC can also update the expiration. When the same hash is represented by multiple `hash_*` directories, the cache may miss blocks that already exist on disk, create another directory for new holes, and fail to remove unloaded sibling directories. Directory-level rename is also fragile when the destination directory already exists and is non-empty. ## Changes - Split File Cache metadata into logical expiration/type and storage expiration/type. - Keep read paths read-only for existing TTL metadata; a query context mismatch no longer updates an already managed hash. - Make new holes inherit the canonical storage expiration from the existing hash, so stale read contexts do not create more `hash_*` directories. - Change `modify_expiration_time` and TTL GC to update logical metadata and queues only, without requiring a parent directory rename. - Add same-hash `hash_*` directory discovery for startup loading and lazy-load fallback, so existing blocks in older TTL directories can still be found. - Add hash-level cleanup to remove all disk directories for a cache hash, including directories that were never loaded into memory. - Keep compatibility with the old `_ttl` filename suffix. - Treat LRU restore entries as hints only; disk scan corrects real storage paths or drops unbound restored entries. ## Tests - `git diff --check` - `DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON ENABLE_INJECTION_POINT=ON ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh run-be-ut.sh --run --filter=BlockFileCacheTest.ttl_modify:BlockFileCacheTest.ttl_modify_memory_storage:BlockFileCacheTest.ttl_change_to_normal:BlockFileCacheTest.ttl_change_to_normal_memory_storage:BlockFileCacheTest.ttl_change_expiration_time:BlockFileCacheTest.ttl_change_expiration_time_memory_storage:BlockFileCacheTest.change_cache_type1:BlockFileCacheTest.change_cache_type2:BlockFileCacheTest.ttl_read_path_mismatch_does_not_update_existing_hash:BlockFileCacheTest.ttl_new_hole_inherits_existing_storage_expiration:BlockFileCacheTest.ttl_modify_expiration_updates_logical_only:BlockFileCacheTest.ttl_lazy_load_misses_exact_dir_and_loads_old_hash_dirs:BlockFileCacheTest.remove_if_cached_removes_unloaded_hash_dirs:BlockFileCacheTest.test_lru_duplicate_queue_entry_restore` The BE UT filter above passed 14 `BlockFileCacheTest` cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
