kaka11chen opened a new pull request, #44635:
URL: https://github.com/apache/doris/pull/44635
### What problem does this PR solve?
Problem Summary:
There's only one scan range for each backend in batch split mode. Each
backend only starts up one ScanNode instance. However, when calculating the
concurrency of scanners in the scan operator in batch split mode, it is not
divided by 1, but by ·query_parallel_instance_num·, resulting in poor
performance of batch split mode.
### Release note
There's only one scan range for each backend in batch split mode. Each
backend only starts up one ScanNode instance. However, when calculating the
concurrency of scanners in the scan operator in batch split mode, it is not
divided by 1, but by ·query_parallel_instance_num·, resulting in poor
performance of batch split mode.
Fix calculation of maximum number of scanner threads in batch split mode.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [x] Manual test (add detailed scripts or steps below)
1. Create a hive partition table which contains many hive partitions(>
1024);
2. Execute `select * from par_table` in doris.
3. Open the profile to view the number of file scan operators and the
number of `MaxScannerThreadNum`.
You should be able to see that each be node of the scan operator is 1,
and there will be more `MaxScannerThreadNum` in the scan operator.
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
