This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new f47e1626238 branch-4.0: [opt](scanner) set number of file scanner to
max_scanners_concurrency(#59622) (#59696)
f47e1626238 is described below
commit f47e162623843cb71eb5efbb5e40b6fb7918af07
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Jan 9 12:03:35 2026 +0800
branch-4.0: [opt](scanner) set number of file scanner to
max_scanners_concurrency(#59622) (#59696)
bp #59622
---
be/src/pipeline/exec/file_scan_operator.cpp | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/be/src/pipeline/exec/file_scan_operator.cpp
b/be/src/pipeline/exec/file_scan_operator.cpp
index 993023790c7..b05638b7471 100644
--- a/be/src/pipeline/exec/file_scan_operator.cpp
+++ b/be/src/pipeline/exec/file_scan_operator.cpp
@@ -111,9 +111,13 @@ void FileScanLocalState::set_scan_ranges(RuntimeState*
state,
auto calc_max_scanners = [&](int parallel_instance_num) -> int {
int max_scanners =
vectorized::ScannerScheduler::default_remote_scan_thread_num() /
parallel_instance_num;
- if (should_run_serial()) {
- max_scanners = 1;
- }
+ // For external tables, each scanner is not bound to specific splits.
+ // Instead, when a scanner is scheduled, it dynamically fetches the
next scan range
+ // from a unified split source for scanning.
+ // Therefore, the number of scanners only needs to match
"max_scanners_concurrency"
+ // to ensure full-speed execution.
+ // For 32 core node, the default "max_scanners_concurrency" should be
16
+ max_scanners = std::min(max_scanners, max_scanners_concurrency(state));
return max_scanners;
};
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
