This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 970319aef810 [docs](metrics) Explain Routine Load received-byte
accounting (#4622)
970319aef810 is described below
commit 970319aef8105bf7a5eda771034c086911b0e351
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Fri Nov 13 08:03:31 2026 +0800
[docs](metrics) Explain Routine Load received-byte accounting (#4622)
The Routine Load metric reference does not explain whether
`doris_fe_routine_load_receive_bytes` measures source input or data
written after a `WHERE` clause. Clarify that it counts received
source-message payload bytes before filtering, including data excluded
by `WHERE`, rather than table-write bytes.
Fixes #2254.
The same flow was checked on Doris master and branches 3.0, 3.1, 4.0 and
4.1:
- The consumer subtracts `msg->len()` from its batch byte budget and
reports the resulting `receive_bytes`.
- `StreamLoadExecutor` transfers that count into `receivedBytes` in the
Routine Load transaction attachment.
- `RoutineLoadJob.updateNumOfData` increments the FE counter with that
value, independently of the filtered/unselected row counts.
Source example: [3.0
consumer](https://github.com/apache/doris/blob/4090f6c40d2238b7aff73185bcada48f96fbd6be/be/src/runtime/routine_load/data_consumer_group.cpp),
[executor](https://github.com/apache/doris/blob/4090f6c40d2238b7aff73185bcada48f96fbd6be/be/src/runtime/stream_load/stream_load_executor.cpp),
[FE
accounting](https://github.com/apache/doris/blob/4090f6c40d2238b7aff73185bcada48f96fbd6be/fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java).
## Review and validation
| Checkpoint | Conclusion |
| --- | --- |
| Goal | Distinguishes source payload from post-filter table writes. |
| Scope | One metric-description cell in each counterpart. |
| Versions/locales | Dev, 4.x, 3.x EN/ZH updated together; behavior
checked in corresponding source branches. Archives unchanged. |
| Links/navigation | No links, headings, routes or sidebars changed. |
| Framework | Not affected. |
| Validation | Reviewed consumer-to-FE accounting; checked six Markdown
row shapes, JSON front matter and `git diff --check`. No cluster test or
site build. |
| Other issues | No introduced issue found in self-review. Does not
claim the FE metric counts unreported/in-flight task bytes. |
## Versions
- [x] dev
- [x] 4.x
- [x] 3.x
- [ ] 2.1 or older
## Languages
- [x] Chinese
- [x] English
## Docs Checklist
- [x] Checked by AI
- [ ] Test Cases Built
- [x] Updated required version and language counterparts
- [x] Updated both languages
Co-authored-by: morningman <[email protected]>
---
docs/admin-manual/maint-monitor/metrics.md | 2 +-
.../current/admin-manual/maint-monitor/metrics.md | 2 +-
.../version-3.x/admin-manual/maint-monitor/metrics.md | 2 +-
.../version-4.x/admin-manual/maint-monitor/metrics.md | 2 +-
versioned_docs/version-3.x/admin-manual/maint-monitor/metrics.md | 2 +-
versioned_docs/version-4.x/admin-manual/maint-monitor/metrics.md | 2 +-
6 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/docs/admin-manual/maint-monitor/metrics.md
b/docs/admin-manual/maint-monitor/metrics.md
index af214e17a466..c429ad716f09 100644
--- a/docs/admin-manual/maint-monitor/metrics.md
+++ b/docs/admin-manual/maint-monitor/metrics.md
@@ -138,7 +138,7 @@ The process's `bvar_max_dump_multi_dimension_metric_num`
setting limits multidim
| `doris_fe_report_queue_size` | | Num | Length of the queue on the FE side
for various periodic report tasks from BE | This value reflects the degree of
blocking of report tasks on the Master FE node. The larger the value, the more
limited the FE's processing capacity | P0 |
| `doris_fe_request_total` | | Num | All operation requests (including
queries and other statements) received through the MySQL port | | |
| `doris_fe_routine_load_error_rows` | | Num | Total number of error rows
across all Routine Load jobs in the cluster | | |
-| `doris_fe_routine_load_receive_bytes` | | Bytes | Total volume of data
received by all Routine Load jobs in the cluster | | |
+| `doris_fe_routine_load_receive_bytes` | | Bytes | Cumulative source-message
payload bytes received by Routine Load jobs, measured before WHERE filtering.
Includes data excluded by WHERE; does not measure bytes written to the table. |
| |
| `doris_fe_routine_load_rows` | | Num | Total number of data rows received
by all Routine Load jobs in the cluster | | |
| `doris_fe_routine_load_get_meta_latency` | | Milliseconds | Latency of
metadata retrieval for all Routine Load Jobs in the cluster | | |
| `doris_fe_routine_load_get_meta_count` | | Num | Number of metadata
retrieval operations for all Routine Load Jobs in the cluster | | |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metrics.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metrics.md
index 7334d89b440b..cb43a162e16d 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metrics.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/maint-monitor/metrics.md
@@ -138,7 +138,7 @@ curl http://cloud_host:brpc_listen_port/brpc_metrics
| `doris_fe_report_queue_size` | | Num | BE 的各种定期汇报任务在 FE 端的队列长度 | 该值反映了汇报任务在
Master FE 节点上的阻塞程度，数值越大，表示 FE 处理能力不足 | P0 |
| `doris_fe_request_total` | | Num | 所有通过 MySQL 端口接收的操作请求（包括查询和其他语句） | | |
| `doris_fe_routine_load_error_rows` | | Num | 统计集群内所有 Routine Load 作业的错误行数总和
| | |
-| `doris_fe_routine_load_receive_bytes` | | 字节 | 统计集群内所有 Routine Load
作业接收的数据量大小 | | |
+| `doris_fe_routine_load_receive_bytes` | | 字节 | 统计 Routine Load
作业接收的源消息正文的累计字节数，在 WHERE 过滤之前计量。包含被 WHERE 条件过滤掉的数据，不代表实际写入表的数据量。 | | |
| `doris_fe_routine_load_rows` | | Num | 统计集群内所有 Routine Load 作业接收的数据行数 | |
|
| `doris_fe_routine_load_get_meta_latency` | | 毫秒 | 统计集群内所有 Routine Load Job
获取元数据的延迟 | | |
| `doris_fe_routine_load_get_meta_count` | | Num | 统计集群内所有 Routine Load Job
获取元数据的操作次数 | | |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/admin-manual/maint-monitor/metrics.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/admin-manual/maint-monitor/metrics.md
index 1390b569ee67..be580247f19a 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/admin-manual/maint-monitor/metrics.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-3.x/admin-manual/maint-monitor/metrics.md
@@ -86,7 +86,7 @@ curl http://be_host:webserver_port/metrics?type=json
| `doris_fe_report_queue_size` |
| Num | BE 的各种定期汇报任务在 FE 端的队列长度
| 该值反映了汇报任务在 Master FE
节点上的阻塞程度，数值越大，表示 FE 处理能力不足 | P0 |
| `doris_fe_request_total` |
| Num | 所有通过 MySQL 端口接收的操作请求（包括查询和其他语句）
|
|
| `doris_fe_routine_load_error_rows` |
| Num | 统计集群内所有 Routine Load 作业的错误行数总和
|
|
-| `doris_fe_routine_load_receive_bytes` |
| 字节 | 统计集群内所有 Routine Load 作业接收的数据量大小
|
|
+| `doris_fe_routine_load_receive_bytes` |
| 字节 | 统计 Routine Load 作业接收的源消息正文的累计字节数，在 WHERE 过滤之前计量。包含被 WHERE
条件过滤掉的数据，不代表实际写入表的数据量。 |
|
| `doris_fe_routine_load_rows` |
| Num | 统计集群内所有 Routine Load 作业接收的数据行数
|
|
| `doris_fe_routine_load_get_meta_latency` |
| 毫秒 | 统计集群内所有 Routine Load Job 获取元数据的延迟
| | |
| `doris_fe_routine_load_get_meta_count` |
| Num | 统计集群内所有 Routine Load Job 获取元数据的操次数 |
| |
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/maint-monitor/metrics.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/maint-monitor/metrics.md
index 4960bdc017b7..7d60458b66c6 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/maint-monitor/metrics.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-4.x/admin-manual/maint-monitor/metrics.md
@@ -138,7 +138,7 @@ curl http://cloud_host:brpc_listen_port/brpc_metrics
| `doris_fe_report_queue_size` | | Num | BE 的各种定期汇报任务在 FE 端的队列长度 | 该值反映了汇报任务在
Master FE 节点上的阻塞程度，数值越大，表示 FE 处理能力不足 | P0 |
| `doris_fe_request_total` | | Num | 所有通过 MySQL 端口接收的操作请求（包括查询和其他语句） | | |
| `doris_fe_routine_load_error_rows` | | Num | 统计集群内所有 Routine Load 作业的错误行数总和
| | |
-| `doris_fe_routine_load_receive_bytes` | | 字节 | 统计集群内所有 Routine Load
作业接收的数据量大小 | | |
+| `doris_fe_routine_load_receive_bytes` | | 字节 | 统计 Routine Load
作业接收的源消息正文的累计字节数，在 WHERE 过滤之前计量。包含被 WHERE 条件过滤掉的数据，不代表实际写入表的数据量。 | | |
| `doris_fe_routine_load_rows` | | Num | 统计集群内所有 Routine Load 作业接收的数据行数 | |
|
| `doris_fe_routine_load_get_meta_latency` | | 毫秒 | 统计集群内所有 Routine Load Job
获取元数据的延迟 | | |
| `doris_fe_routine_load_get_meta_count` | | Num | 统计集群内所有 Routine Load Job
获取元数据的操作次数 | | |
diff --git a/versioned_docs/version-3.x/admin-manual/maint-monitor/metrics.md
b/versioned_docs/version-3.x/admin-manual/maint-monitor/metrics.md
index f845ec879787..2df3ae64b4c4 100644
--- a/versioned_docs/version-3.x/admin-manual/maint-monitor/metrics.md
+++ b/versioned_docs/version-3.x/admin-manual/maint-monitor/metrics.md
@@ -85,7 +85,7 @@ The query error rate can be obtained by calculating the slope
of `doris_fe_query
| doris_fe_report_queue_size |
| Num | The queue length of various periodic reporting tasks of BE on
the FE side | This value reflects the blocking degree of the reporting task on
the Master FE node. The larger the value, the lower the processing capacity of
the FE . | P0 |
| doris_fe_request_total |
| Num | All operation requests received through the MySQL port
(including queries and other statements ) |
| |
| doris_fe_routine_load_error_rows |
| Num | Count the total number of error rows for all Routine Load jobs
in the cluster | |
|
-| doris_fe_routine_load_receive_bytes |
| byte | the amount of data received by all Routine Load jobs in the
cluster | |
|
+| doris_fe_routine_load_receive_bytes |
| byte | Cumulative source-message payload bytes received by Routine
Load jobs, measured before WHERE filtering. Includes data excluded by WHERE;
does not measure bytes written to the table. |
| |
| doris_fe_routine_load_rows |
| Num | Count the number of data rows received by all Routine Load
jobs in the cluster |
| |
| doris_fe_routine_load_get_meta_latency |
| millisecond | Total latency for retrieving metadata for all Routine Load
Jobs across the cluster |
| |
| doris_fe_routine_load_get_meta_count |
| Num | Total number of metadata retrieval operations for all Routine
Load Jobs across the cluster |
| |
diff --git a/versioned_docs/version-4.x/admin-manual/maint-monitor/metrics.md
b/versioned_docs/version-4.x/admin-manual/maint-monitor/metrics.md
index af214e17a466..c429ad716f09 100644
--- a/versioned_docs/version-4.x/admin-manual/maint-monitor/metrics.md
+++ b/versioned_docs/version-4.x/admin-manual/maint-monitor/metrics.md
@@ -138,7 +138,7 @@ The process's `bvar_max_dump_multi_dimension_metric_num`
setting limits multidim
| `doris_fe_report_queue_size` | | Num | Length of the queue on the FE side
for various periodic report tasks from BE | This value reflects the degree of
blocking of report tasks on the Master FE node. The larger the value, the more
limited the FE's processing capacity | P0 |
| `doris_fe_request_total` | | Num | All operation requests (including
queries and other statements) received through the MySQL port | | |
| `doris_fe_routine_load_error_rows` | | Num | Total number of error rows
across all Routine Load jobs in the cluster | | |
-| `doris_fe_routine_load_receive_bytes` | | Bytes | Total volume of data
received by all Routine Load jobs in the cluster | | |
+| `doris_fe_routine_load_receive_bytes` | | Bytes | Cumulative source-message
payload bytes received by Routine Load jobs, measured before WHERE filtering.
Includes data excluded by WHERE; does not measure bytes written to the table. |
| |
| `doris_fe_routine_load_rows` | | Num | Total number of data rows received
by all Routine Load jobs in the cluster | | |
| `doris_fe_routine_load_get_meta_latency` | | Milliseconds | Latency of
metadata retrieval for all Routine Load Jobs in the cluster | | |
| `doris_fe_routine_load_get_meta_count` | | Num | Number of metadata
retrieval operations for all Routine Load Jobs in the cluster | | |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
