This is an automated email from the ASF dual-hosted git repository. lzljs3620320 pushed a commit to branch release-0.6 in repository https://gitbox.apache.org/repos/asf/incubator-paimon.git
commit a1ca84f5105299e5b786df488dc843eedbc1c1a9 Author: GuojunLi <[email protected]> AuthorDate: Mon Dec 11 13:22:02 2023 +0800 [doc] Add write and write buffer metrics (#2474) --- docs/content/maintenance/metrics.md | 80 ++++++++++++++++++++++++++++++++++++- 1 file changed, 78 insertions(+), 2 deletions(-) diff --git a/docs/content/maintenance/metrics.md b/docs/content/maintenance/metrics.md index 156b84785..437fc6dc5 100644 --- a/docs/content/maintenance/metrics.md +++ b/docs/content/maintenance/metrics.md @@ -35,11 +35,11 @@ There are three types of metrics provided in the Paimon metric system, `Gauge`, - `Counter`: Used to count values by incrementing and decrementing. - `Histogram`: Measure the statistical distribution of a set of values including the min, max, mean, standard deviation and percentile. -Paimon has supported built-in metrics to measure operations of **commits**, **scans** and **compactions**, which can be bridged to any computing engine that supports, like Flink, Spark etc. +Paimon has supported built-in metrics to measure operations of **commits**, **scans**, **writes** and **compactions**, which can be bridged to any computing engine that supports, like Flink, Spark etc. ## Metrics List -Below is lists of Paimon built-in metrics. They are summarized into three types of metrics, scan metrics, commit metrics and compaction metrics. +Below is lists of Paimon built-in metrics. They are summarized into types of scan metrics, commit metrics, write metrics, write buffer metrics and compaction metrics. ### Scan Metrics @@ -215,6 +215,72 @@ Below is lists of Paimon built-in metrics. They are summarized into three types </tbody> </table> +### Write Metrics + +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 225pt">Metrics Name</th> + <th class="text-left" style="width: 65pt">Level</th> + <th class="text-left" style="width: 70pt">Type</th> + <th class="text-left" style="width: 300pt">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>writeRecordCount</td> + <td>Bucket</td> + <td>Counter</td> + <td>Total number of records written into the bucket.</td> + </tr> + <tr> + <td>flushCostMillis</td> + <td>Bucket</td> + <td>Histogram</td> + <td>Distributions of the time taken by the last few write buffer flushing.</td> + </tr> + <tr> + <td>prepareCommitCostMillis</td> + <td>Bucket</td> + <td>Histogram</td> + <td>Distributions of the time taken by the last few call of `prepareCommit`.</td> + </tr> + </tbody> +</table> + +### Write Buffer Metrics + +<table class="table table-bordered"> + <thead> + <tr> + <th class="text-left" style="width: 225pt">Metrics Name</th> + <th class="text-left" style="width: 65pt">Level</th> + <th class="text-left" style="width: 70pt">Type</th> + <th class="text-left" style="width: 300pt">Description</th> + </tr> + </thead> + <tbody> + <tr> + <td>bufferPreemptCount</td> + <td>Table</td> + <td>Gauge</td> + <td>The total number of memory preempted.</td> + </tr> + <tr> + <td>usedWriteBufferSizeByte</td> + <td>Table</td> + <td>Gauge</td> + <td>Current used write buffer size in byte.</td> + </tr> + <tr> + <td>totalWriteBufferSizeByte</td> + <td>Table</td> + <td>Gauge</td> + <td>The total write buffer size configured in byte.</td> + </tr> + </tbody> +</table> + ### Compaction Metrics <table class="table table-bordered"> @@ -316,6 +382,16 @@ From Flink Web-UI, go to the committer operator's metrics, it's shown as: <td><host>.taskmanager.<tm_id>.<job_name>.<committer_operator_name>.<subtask_index></td> <td>paimon.table.<table_name>.commit</td> </tr> + <tr> + <td>Write Metrics</td> + <td><host>.taskmanager.<tm_id>.<job_name>.<writer_operator_name>.<subtask_index></td> + <td>paimon.table.<table_name>.partition.<partition_string>.bucket.<bucket_index>.writer</td> + </tr> + <tr> + <td>Write Buffer Metrics</td> + <td><host>.taskmanager.<tm_id>.<job_name>.<writer_operator_name>.<subtask_index></td> + <td>paimon.table.<table_name>.writeBuffer</td> + </tr> <tr> <td>Compaction Metrics</td> <td><host>.taskmanager.<tm_id>.<job_name>.<writer_operator_name>.<subtask_index></td>
