[
https://issues.apache.org/jira/browse/FLINK-34558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17907779#comment-17907779
]
Jufang He edited comment on FLINK-34558 at 12/23/24 9:25 AM:
-------------------------------------------------------------
[~masteryhx]
Thanks for your suggestion, and I submitted a PR that implements the changes
similar to the state-latency track(sampling and disabled by default). Maybe you
could take a look at it, and I'll start the benchmark test if that's okay.
was (Author: JIRAUSER302059):
[~masteryhx]
Thanks for your suggestion, and I submitted a PR that implements the changes
similar to the state-latency track. Maybe you could take a look at it, and I'll
start the benchmark test if that's okay.
> Add RocksDB key/value size metrics
> ----------------------------------
>
> Key: FLINK-34558
> URL: https://issues.apache.org/jira/browse/FLINK-34558
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.19.0
> Reporter: Jufang He
> Priority: Major
> Labels: pull-request-available
>
> In some scenarios, the poor performance of RocksDB may be caused by too large
> key/value size, but now there is a lack of metrics for key/value size. By
> adding these metrics, we can conveniently calculate the distribution of
> key/value size, such as the average size and p99 size. To reduce the
> negative impact of adding metric on RocksDB performance, we could reduce the
> impact by supporting sampling.
> The possible implementation is as follows: After the RocksDB key/value
> serialization, we could obtain the byte array and report the size of the
> array through histogram metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
