[
https://issues.apache.org/jira/browse/FLINK-34386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jufang He updated FLINK-34386:
------------------------------
Description:
In our production environment, with RocksDB bloom filter enabled, the
performance optimization effect on task state reading is obvious. However,
there is a lack of usage metrics for bloom filter, If these Metrics are
reported via Metrics reporter, it is easy to monitor the effectiveness of bloom
filter optimization.
And these metrics are available from RocksDB Statistics:
BLOOM_FILTER_USEFUL: times bloom filter has avoided file reads.
BLOOM_FILTER_FULL_POSITIVE: times bloom FullFilter has not avoided the reads.
BLOOM_FILTER_FULL_TRUE_POSITIVE: times bloom FullFilter has not avoided the
reads and data actually exist.
was:
In our production environment, with RocksDB bloom filter enabled, the
performance optimization effect on task state reading is obvious. However,
there is a lack of usage metrics for bloom filter, If these Metrics are
reported via Metrics reporter, it is easy to monitor the effectiveness of bloom
filter optimization.
And these metrics are available from rocksdb Statistics:
BLOOM_FILTER_USEFUL: times bloom filter has avoided file reads.
BLOOM_FILTER_FULL_POSITIVE: times bloom FullFilter has not avoided the reads.
BLOOM_FILTER_FULL_TRUE_POSITIVE: times bloom FullFilter has not avoided the
reads and data actually exist.
> Add RocksDB bloom filter metrics
> --------------------------------
>
> Key: FLINK-34386
> URL: https://issues.apache.org/jira/browse/FLINK-34386
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.18.0
> Reporter: Jufang He
> Priority: Major
>
> In our production environment, with RocksDB bloom filter enabled, the
> performance optimization effect on task state reading is obvious. However,
> there is a lack of usage metrics for bloom filter, If these Metrics are
> reported via Metrics reporter, it is easy to monitor the effectiveness of
> bloom filter optimization.
> And these metrics are available from RocksDB Statistics:
> BLOOM_FILTER_USEFUL: times bloom filter has avoided file reads.
> BLOOM_FILTER_FULL_POSITIVE: times bloom FullFilter has not avoided the reads.
> BLOOM_FILTER_FULL_TRUE_POSITIVE: times bloom FullFilter has not avoided the
> reads and data actually exist.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
