deardeng opened a new pull request, #63789:
URL: https://github.com/apache/doris/pull/63789
Problem Summary: FSFileCacheStorage used a single mutex and unordered map to
track in-flight local file cache writers. Cache miss read paths write back
remote or peer-read blocks into local file cache, and different block writers
had to serialize on the same lock while creating cache directories and
temporary files. This change shards the writer map into 1024 buckets, each with
its own mutex, so unrelated cache blocks no longer contend on one global
writer-map lock. The current master file cache storage interface only has
append and finalize, so the cherry-pick was adapted to those two critical
sections while preserving the v3 cache path and metadata-store logic. A BE unit
test now keeps multiple writers in flight before finalizing them, verifies they
are tracked across multiple writer shards, and checks the finalized block data
can be read back.
### Release note
None
### Check List (For Author)
- Test: Unit Test / Manual test
- Added BE UT
BlockFileCacheTest.sharded_writer_map_tracks_multiple_inflight_writers
- Tried ./run-be-ut.sh --run
--filter=BlockFileCacheTest.sharded_writer_map_tracks_multiple_inflight_writers
-j 8, but CMake configuration failed before compiling tests because OpenMP_C
was not found while configuring contrib/openblas
- Ran git show --check on the amended commit
- Ran git diff --check
- Tried build-support/clang-format.sh, but the environment clang-format
version is not 16
- Behavior changed: No
- Does this need documentation: No
(cherry picked from commit e84ecd60ed5c3af23e260f7f2fc25431bda04e32)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
