bobhan1 opened a new pull request, #66414:
URL: https://github.com/apache/doris/pull/66414
### What problem does this PR solve?
Issue Number: N/A
Related PR: #65021
Problem Summary:
Backport #65021 to `branch-4.1`. This adds an option for TopN lazy
materialization phase 2 to read remote cache misses without populating File
Cache, and exposes the corresponding profile metrics.
The cherry-pick conflicts were adapted to the `branch-4.1` layouts:
- Keep the existing adjacent-file batching and compile checks in
`RowIDFetcher`, while propagating `file_cache_miss_policy` to Doris-format
reads.
- Keep the single-row `Segment` API and preserve the copied `IOContext` when
setting reader type, stats, and cache-miss policy.
- Use the `branch-4.1` `VariableMgr.VarAttr` annotation for the new session
variable.
### Release note
Add `enable_topn_lazy_mat_phase2_no_write_file_cache` to skip File Cache
population for remote misses during TopN lazy materialization phase 2.
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
Validation:
- `./build.sh --fe -j100`
- `./build.sh --be --clean -j100`
- `./run-be-ut.sh --run
--filter='MaterializationSharedStateTest.*:BlockFileCacheTest.get_downloaded_blocks_if_fully_covered_is_read_only:BlockFileCacheTest.cached_remote_file_reader_remote_only_on_miss:BlockFileCacheTest.fd_cache_remove:BlockFileCacheTest.fd_cache_evict'
-j100` (9 tests passed)
- `./run-regression-test.sh --run -d cloud_p0/cache/topn_lazy_file_cache -s
test_topn_lazy_mat_phase2_no_write_file_cache -g docker -runMode=cloud
-dockerSuiteParallel 1` (1 suite passed)
- Behavior changed:
- [ ] No.
- [x] Yes. When enabled, TopN lazy materialization phase 2 serves remote
misses without writing the fetched data into File Cache.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
