This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch tpc_preview3
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/tpc_preview3 by this push:
new 85af0ca8c79 [fix](s3-client) Fix s3-client max-connection default
value which causes s3 to be slow and unable to handle high concurrency in
multi-file scenarios. (#58512)
85af0ca8c79 is described below
commit 85af0ca8c7916ca9714fe6d583d314557ddddc3b
Author: Qi Chen <[email protected]>
AuthorDate: Fri Nov 28 17:41:59 2025 +0800
[fix](s3-client) Fix s3-client max-connection default value which causes s3
to be slow and unable to handle high concurrency in multi-file scenarios.
(#58512)
### What problem does this PR solve?
Problem Summary:
### Release note
Cherry-pick #58511
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
---
be/src/util/s3_util.cpp | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/be/src/util/s3_util.cpp b/be/src/util/s3_util.cpp
index 172f5f85419..55b48e64f6d 100644
--- a/be/src/util/s3_util.cpp
+++ b/be/src/util/s3_util.cpp
@@ -369,8 +369,7 @@ std::shared_ptr<io::ObjStorageClient>
S3ClientFactory::_create_s3_client(
if (s3_conf.max_connections > 0) {
aws_config.maxConnections = s3_conf.max_connections;
} else {
- // AWS SDK max concurrent tcp connections for a single http client to
use. Default 25.
- aws_config.maxConnections =
std::max(config::doris_scanner_thread_pool_thread_num, 25);
+ aws_config.maxConnections = 102400;
}
aws_config.requestTimeoutMs = 30000;
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
