This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 8aa4aee3e99 docs: clarify Doris Kafka Connector batch tuning (#3793)
8aa4aee3e99 is described below
commit 8aa4aee3e99e2272da02713a58e89d717a417d36
Author: lsy3993 <[email protected]>
AuthorDate: Sat Oct 17 05:40:06 2026 +0800
docs: clarify Doris Kafka Connector batch tuning (#3793)
## Summary
Clarify batch tuning for Doris Kafka Connector tasks running on Kafka
Connect or Confluent. The old FAQ treated an explicitly configured
10,000-record example as the Connector default. The revised dev and 4.x
English/Chinese entries distinguish those values, provide a
50,000-record / 100 MiB example, and explain flush-time, task-memory,
and latency tradeoffs.
## Validation
- Checked the official Connector configuration and DorisWriter
implementation; current defaults are 50,000 records and 100 MiB.
- Checked all four dev/4.x English/Chinese pages and their Connector
guide links.
- Targeted static governance checks and git diff --check passed with no
new blocking findings.
- Independently reviewed by a second agent.
## Versions and languages
- [x] dev
- [x] 4.x
- [x] English
- [x] Chinese
---------
Co-authored-by: Claude Opus 4.7 <[email protected]>
Co-authored-by: morningman <[email protected]>
---
docs/faq/load-faq.md | 11 +++++++++++
.../docusaurus-plugin-content-docs/current/faq/load-faq.md | 11 +++++++++++
.../version-4.x/faq/load-faq.md | 11 +++++++++++
versioned_docs/version-4.x/faq/load-faq.md | 11 +++++++++++
4 files changed, 44 insertions(+)
diff --git a/docs/faq/load-faq.md b/docs/faq/load-faq.md
index 517dae1392d..3e486b955b9 100644
--- a/docs/faq/load-faq.md
+++ b/docs/faq/load-faq.md
@@ -72,6 +72,17 @@ curl --location-trusted -u root:"" \
http://127.0.0.1:8030/api/db/loadtest/_stream_load
```
+### Small batches when loading through Kafka Connect or Confluent
+
+When a Doris Kafka Connector task sends frequent, small Stream Load requests,
check the task's buffer settings. For example, if an existing configuration
sets `buffer.count.records=10000` and `buffer.size.bytes=5000000`, consider
testing larger batches:
+
+```properties
+buffer.count.records=50000
+buffer.size.bytes=104857600
+```
+
+These are Doris Kafka Connector options, rather than general Confluent
settings. They control the record-count and byte-size thresholds for a batch. A
larger buffer can reduce the number of load requests, but requires more Kafka
Connect task memory and can increase latency. Also review `buffer.flush.time`,
which can flush a batch before either size threshold is reached. Check the
options and defaults for the Connector version you use in [Doris Kafka
Connector](../connection-integration/d [...]
+
## Routine Load
### Major Bug Fixes
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/load-faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/load-faq.md
index 8d384565cc1..64345212324 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/load-faq.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/faq/load-faq.md
@@ -72,6 +72,17 @@ curl --location-trusted -u root:"" \
http://127.0.0.1:8030/api/db/loadtest/_stream_load
```
+### 通过 Kafka Connect 或 Confluent 导入时批次过小
+
+Doris Kafka Connector 任务频繁发送较小的 Stream Load 请求时，应检查任务的缓冲配置。例如，已有配置显式设置了
`buffer.count.records=10000` 和 `buffer.size.bytes=5000000` 时，可测试更大的批次：
+
+```properties
+buffer.count.records=50000
+buffer.size.bytes=104857600
+```
+
+这些参数属于 Doris Kafka Connector，并非 Confluent
的通用配置，分别控制一个批次的记录数和字节数阈值。增大缓冲区可以减少导入请求数，但会增加 Kafka Connect 任务的内存需求，也可能增加延迟。同时检查
`buffer.flush.time`，它可能在两个大小阈值到达前触发刷新。所用 Connector 版本的参数及默认值请参阅 [Doris Kafka
Connector](../connection-integration/data-integration/doris-kafka-connector.md)。
+
## Routine Load
### 较严重的 Bug 修复
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/faq/load-faq.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/faq/load-faq.md
index 8d384565cc1..64345212324 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/faq/load-faq.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/faq/load-faq.md
@@ -72,6 +72,17 @@ curl --location-trusted -u root:"" \
http://127.0.0.1:8030/api/db/loadtest/_stream_load
```
+### 通过 Kafka Connect 或 Confluent 导入时批次过小
+
+Doris Kafka Connector 任务频繁发送较小的 Stream Load 请求时，应检查任务的缓冲配置。例如，已有配置显式设置了
`buffer.count.records=10000` 和 `buffer.size.bytes=5000000` 时，可测试更大的批次：
+
+```properties
+buffer.count.records=50000
+buffer.size.bytes=104857600
+```
+
+这些参数属于 Doris Kafka Connector，并非 Confluent
的通用配置，分别控制一个批次的记录数和字节数阈值。增大缓冲区可以减少导入请求数，但会增加 Kafka Connect 任务的内存需求，也可能增加延迟。同时检查
`buffer.flush.time`，它可能在两个大小阈值到达前触发刷新。所用 Connector 版本的参数及默认值请参阅 [Doris Kafka
Connector](../connection-integration/data-integration/doris-kafka-connector.md)。
+
## Routine Load
### 较严重的 Bug 修复
diff --git a/versioned_docs/version-4.x/faq/load-faq.md
b/versioned_docs/version-4.x/faq/load-faq.md
index 517dae1392d..3e486b955b9 100644
--- a/versioned_docs/version-4.x/faq/load-faq.md
+++ b/versioned_docs/version-4.x/faq/load-faq.md
@@ -72,6 +72,17 @@ curl --location-trusted -u root:"" \
http://127.0.0.1:8030/api/db/loadtest/_stream_load
```
+### Small batches when loading through Kafka Connect or Confluent
+
+When a Doris Kafka Connector task sends frequent, small Stream Load requests,
check the task's buffer settings. For example, if an existing configuration
sets `buffer.count.records=10000` and `buffer.size.bytes=5000000`, consider
testing larger batches:
+
+```properties
+buffer.count.records=50000
+buffer.size.bytes=104857600
+```
+
+These are Doris Kafka Connector options, rather than general Confluent
settings. They control the record-count and byte-size thresholds for a batch. A
larger buffer can reduce the number of load requests, but requires more Kafka
Connect task memory and can increase latency. Also review `buffer.flush.time`,
which can flush a batch before either size threshold is reached. Check the
options and defaults for the Connector version you use in [Doris Kafka
Connector](../connection-integration/d [...]
+
## Routine Load
### Major Bug Fixes
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
