freemandealer opened a new pull request, #64381:
URL: https://github.com/apache/doris/pull/64381
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: File cache LRU log replay needs tighter default replay
latency, bounded in-memory queues, and observability for both pending block LRU
updates and LRU log replay. This change lowers the replay interval default to 1
ms, adds hard caps for the pending block update queue and per-type LRU log
queues, preserves existing LRU log backlog when tail recording is disabled, and
exposes queue length plus produce/consume/idle metrics. Tests that depended on
background replay timing now use a deterministic single-replay helper with a
high background interval.
### Release note
Add bounded file cache LRU recorder queues and replay/update queue
observability metrics.
### Check List (For Author)
- Test: Unit Test
- `JAVA_HOME=/mnt/disk1/zhangzhengyu/build-dep/jdk-17.0.2/
CUSTOM_MVN=/data/data4/zhangzhengyu/project/doris-pick/.tooling/apache-maven-3.9.11/bin/mvn
DORIS_TOOLCHAIN=clang DISABLE_BE_JAVA_EXTENSIONS=ON ENABLE_INJECTION_POINT=ON
ENABLE_CACHE_LOCK_DEBUG=0 ENABLE_PCH=0 sh run-be-ut.sh --run
--filter=BlockFileCacheTest.test_lru_log_record_replay_dump_restore:BlockFileCacheTest.test_lru_duplicate_queue_entry_restore:BlockFileCacheTest.need_update_lru_blocks_hard_cap:BlockFileCacheTest.version3_add_remove_restart:CacheLRUDumperTest.*:NeedUpdateLRUBlocksTest.*`
- `PATH=/mnt/disk1/zhangzhengyu/build-dep/ldb_toolchain.back/bin:$PATH
build-support/check-format.sh`
- Behavior changed: Yes. File cache LRU replay defaults to 1 ms, LRU log and
block update queues are capped at 500000 by default, and disabling tail record
enqueueing no longer clears existing replay backlog.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
