This is an automated email from the ASF dual-hosted git repository.
JNSimba pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 1d629e40636 [improvement](streaming-job) Adjust CDC request timeout
and split batch defaults (#67559)
1d629e40636 is described below
commit 1d629e40636308d768042a75bd3f111352212083
Author: wudi <[email protected]>
AuthorDate: Wed Sep 9 09:49:44 2026 +0800
[improvement](streaming-job) Adjust CDC request timeout and split batch
defaults (#67559)
### What problem does this PR solve?
Slow CDC control requests, including snapshot split discovery for
unevenly distributed keys, can exceed the default 60-second BE HTTP
timeout. Retrying split discovery can repeat expensive work before the
FE receives progress.
This PR changes only two defaults:
- Increase `request_cdc_client_timeout_ms` from `60000` to `120000` (120
seconds).
- Reduce `streaming_cdc_fetch_splits_batch_size` from `100` to `16` to
return smaller batches on batched split-discovery paths.
---
be/src/common/config.cpp | 2 +-
fe/fe-common/src/main/java/org/apache/doris/common/Config.java | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index faec885e0ef..068ff5dff7c 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -714,7 +714,7 @@ DEFINE_mBool(enable_stream_load_commit_txn_on_be, "false");
DEFINE_Int64(stream_tvf_buffer_size, "1048576"); // 1MB
// request cdc client timeout
-DEFINE_mInt32(request_cdc_client_timeout_ms, "60000");
+DEFINE_mInt32(request_cdc_client_timeout_ms, "120000");
// OlapTableSink sender's send interval, should be less than the real response
time of a tablet writer rpc.
// You may need to lower the speed when the sink receiver bes are too busy.
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index de5d152a009..54a16d46685 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -1224,7 +1224,7 @@ public class Config extends ConfigBase {
public static int streaming_pg_max_identifier_length = 63;
@ConfField(mutable = true, masterOnly = true)
- public static int streaming_cdc_fetch_splits_batch_size = 100;
+ public static int streaming_cdc_fetch_splits_batch_size = 16;
/**
* the max timeout of get kafka meta.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
