This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new fc6f8ea1b95 [fix](s3-client) Fix s3-client max-connection default
value which cause s3 to be slow and unable to handle high concurrency in
many-files scenarios. (#58511) (#58626)
fc6f8ea1b95 is described below
commit fc6f8ea1b952a75585eb5f52538e83d10f53c45c
Author: Qi Chen <[email protected]>
AuthorDate: Wed Dec 3 09:53:38 2025 +0800
[fix](s3-client) Fix s3-client max-connection default value which cause s3
to be slow and unable to handle high concurrency in many-files scenarios.
(#58511) (#58626)
### What problem does this PR solve?
Related PR: #51825, #58514
Problem Summary:
### Release note
Cherry-pick #58511
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
be/src/util/s3_util.cpp | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/be/src/util/s3_util.cpp b/be/src/util/s3_util.cpp
index a0e87febf6a..bd0629ab3ef 100644
--- a/be/src/util/s3_util.cpp
+++ b/be/src/util/s3_util.cpp
@@ -369,8 +369,7 @@ std::shared_ptr<io::ObjStorageClient>
S3ClientFactory::_create_s3_client(
if (s3_conf.max_connections > 0) {
aws_config.maxConnections = s3_conf.max_connections;
} else {
- // AWS SDK max concurrent tcp connections for a single http client to
use. Default 25.
- aws_config.maxConnections =
std::max(config::doris_scanner_thread_pool_thread_num, 25);
+ aws_config.maxConnections = 102400;
}
aws_config.requestTimeoutMs = 30000;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
