Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
pnowojski merged PR #24274: URL: https://github.com/apache/flink/pull/24274 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1953405068 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on code in PR #24274:
URL: https://github.com/apache/flink/pull/24274#discussion_r1494487106
##
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBNativeMetricOptions.java:
##
@@ -262,6 +262,27 @@ public class RocksDBNativeMetricOptions implements
Serializable {
.withDescription(
"Monitor the total count of block cache misses in
RocksDB (BLOCK_CACHE_MISS == BLOCK_CACHE_INDEX_MISS + BLOCK_CACHE_FILTER_MISS +
BLOCK_CACHE_DATA_MISS).");
+public static final ConfigOption MONITOR_BLOOM_FILTER_USEFUL =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-useful")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom filter has
avoided file reads.");
+
+public static final ConfigOption
MONITOR_BLOOM_FILTER_FULL_POSITIVE =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-full-positive")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom FullFilter has
not avoided the reads.");
+
+public static final ConfigOption
MONITOR_BLOOM_FILTER_FULL_TRUE_POSITIVE =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-full-true-positive")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom FullFilter has
not avoided the reads and data actually exist.");
Review Comment:
Thanks for your advice, I’ve refined this description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
Zakelly commented on code in PR #24274:
URL: https://github.com/apache/flink/pull/24274#discussion_r1494398734
##
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBNativeMetricOptions.java:
##
@@ -262,6 +262,27 @@ public class RocksDBNativeMetricOptions implements
Serializable {
.withDescription(
"Monitor the total count of block cache misses in
RocksDB (BLOCK_CACHE_MISS == BLOCK_CACHE_INDEX_MISS + BLOCK_CACHE_FILTER_MISS +
BLOCK_CACHE_DATA_MISS).");
+public static final ConfigOption MONITOR_BLOOM_FILTER_USEFUL =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-useful")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom filter has
avoided file reads.");
+
+public static final ConfigOption
MONITOR_BLOOM_FILTER_FULL_POSITIVE =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-full-positive")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom FullFilter has
not avoided the reads.");
+
+public static final ConfigOption
MONITOR_BLOOM_FILTER_FULL_TRUE_POSITIVE =
+
ConfigOptions.key("state.backend.rocksdb.metrics.bloom-filter-full-true-positive")
+.booleanType()
+.defaultValue(false)
+.withDescription(
+"Monitor the total count of bloom FullFilter has
not avoided the reads and data actually exist.");
Review Comment:
I would suggest a refine, such as "Monitor the count of reads avoided by
full filter while the data actually exists in RocksDB".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1942857042 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1941505603 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1940793381 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1940364419 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
JingGe commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1939890045 @hejufang the issue should be fixed if you rebase your branch -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
JingGe commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1935899938 There some issue while building the hadoop image: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57409=logs=ef799394-2d67-5ff4-b2e5-410b80c9c0af=9e5768bc-daae-5f5f-1861-e58617922c7a=10543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1935521846 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1935352424 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1934485237 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
pnowojski commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1934175386 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1933419178 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1931766026 @pnowojski Thank you for your reminding. I have fixed the test failure, please review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1931429706 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
flinkbot commented on PR #24274: URL: https://github.com/apache/flink/pull/24274#issuecomment-1929093462 ## CI report: * 7ab0a0e0b8eeb7459e5dbaee55804fa72e7c4837 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [FLINK-34386][state] Add RocksDB bloom filter metrics [flink]
hejufang opened a new pull request, #24274: URL: https://github.com/apache/flink/pull/24274 ## What is the purpose of the change Add RocksDB bloom filter metrics. ## Brief change log Get RocksDB bloom filter metrics via RocksDB Statistics, and report it via Metrics reporter. ## Verifying this change Please make sure both new and modified tests in this PR follows the conventions defined in our code quality guide: https://flink.apache.org/contributing/code-style-and-quality-common.html#testing This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): no - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no - The serializers: no - The runtime per-record code paths (performance sensitive): no - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no - The S3 file system connector: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? not documented -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
