This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new de3862f910d [refactor](file-cache) refactor statistics collection
logic in VFileScanner (#53385)
de3862f910d is described below
commit de3862f910dac5fd64f92f9c5a4d6564ace0f7fe
Author: Wen Zhenghu <[email protected]>
AuthorDate: Fri Aug 29 17:10:16 2025 +0800
[refactor](file-cache) refactor statistics collection logic in VFileScanner
(#53385)
### What problem does this PR solve?
Add query statistics collection for scan bytes from local and remote
storage when enable file cache.
This change enhances the file cache statistics collection mechanism and
provides better visibility into storage access patterns during file
scanning operations.
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [x] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
be/src/vec/exec/scan/vfile_scanner.cpp | 16 ++++++++++++----
1 file changed, 12 insertions(+), 4 deletions(-)
diff --git a/be/src/vec/exec/scan/vfile_scanner.cpp
b/be/src/vec/exec/scan/vfile_scanner.cpp
index 7cbe6116b63..d6059347f02 100644
--- a/be/src/vec/exec/scan/vfile_scanner.cpp
+++ b/be/src/vec/exec/scan/vfile_scanner.cpp
@@ -1186,10 +1186,18 @@ void VFileScanner::try_stop() {
void VFileScanner::_collect_profile_before_close() {
VScanner::_collect_profile_before_close();
- if (config::enable_file_cache && _state->query_options().enable_file_cache
&&
- _profile != nullptr) {
- io::FileCacheProfileReporter cache_profile(_profile);
- cache_profile.update(_file_cache_statistics.get());
+ if (config::enable_file_cache &&
_state->query_options().enable_file_cache) {
+ if (_profile != nullptr) {
+ io::FileCacheProfileReporter cache_profile(_profile);
+ cache_profile.update(_file_cache_statistics.get());
+ }
+
+ if (_query_statistics) {
+ _query_statistics->add_scan_bytes_from_local_storage(
+ _file_cache_statistics->bytes_read_from_local);
+ _query_statistics->add_scan_bytes_from_remote_storage(
+ _file_cache_statistics->bytes_read_from_remote);
+ }
}
if (_cur_reader != nullptr) {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
