[
https://issues.apache.org/jira/browse/KAFKA-18495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Swikar Patel updated KAFKA-18495:
---------------------------------
Description:
The `numberOfOpenFiles` metric in the Kafka Streams RocksDB state store
implementation is currently invalid and should be removed.
This metric was intended to report the approximate number of open files in the
RocksDB instance. It was previously calculated using the RocksDB internal
metric `NO_FILE_CLOSES`, which was removed in RocksDB version 7.9.3. Due to
this change, the `numberOfOpenFiles` metric no longer provides accurate
information. The current workaround is to have it always return -1, which is
misleading.
The `numberOfOpenFiles` metric is: * *{*}Invalid:{*}* It does not reflect the
actual number of open files in RocksDB. * *{*}Misleading:{*}* Returning -1 can
confuse users and monitoring systems. * *{*}Unnecessary:{*}* It provides no
useful information in its current state.
h4. Proposed Solution
This Jira issue is to implement the changes proposed in KIP-1125:
[[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]].
The solution is to completely remove the `opennumberfiles` metric, including:
* Removing the related code in `RocksDBMetricsRecorder`
(`numberOfOpenFilesSensor`, recording logic). * Removing the
`numberOfOpenFilesSensor` method in `RocksDBMetrics` * Updating the Kafka
Streams documentation to reflect the metric's removal.
h4. KIP
KIP-1125:
[[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]]
was:
h4. Description
The `numberOfOpenFiles` metric in the Kafka Streams RocksDB state store
implementation is currently invalid and should be removed.
h4. Background
This metric was intended to report the approximate number of open files in the
RocksDB instance. It was previously calculated using the RocksDB internal
metric `NO_FILE_CLOSES`, which was removed in RocksDB version 7.9.3. Due to
this change, the `numberOfOpenFiles` metric no longer provides accurate
information. The current workaround is to have it always return -1, which is
misleading.
h4. Problem
The `numberOfOpenFiles` metric is: * **Invalid:** It does not reflect the
actual number of open files in RocksDB. * **Misleading:** Returning -1 can
confuse users and monitoring systems. * **Unnecessary:** It provides no useful
information in its current state.
h4. Proposed Solution
This Jira issue is to implement the changes proposed in KIP-1125:
[[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]].
The solution is to completely remove the `opennumberfiles` metric, including:
* Removing the related code in `RocksDBMetricsRecorder`
(`numberOfOpenFilesSensor`, recording logic). * Removing the
`numberOfOpenFilesSensor` method in `RocksDBMetrics` * Updating the Kafka
Streams documentation to reflect the metric's removal.
h4. KIP
KIP-1125:
[[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]]
> Remove Invalid 'numberOfOpenFiles' Metric from RocksDB State Store
> -------------------------------------------------------------------
>
> Key: KAFKA-18495
> URL: https://issues.apache.org/jira/browse/KAFKA-18495
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Swikar Patel
> Assignee: Swikar Patel
> Priority: Major
>
> The `numberOfOpenFiles` metric in the Kafka Streams RocksDB state store
> implementation is currently invalid and should be removed.
> This metric was intended to report the approximate number of open files in
> the RocksDB instance. It was previously calculated using the RocksDB internal
> metric `NO_FILE_CLOSES`, which was removed in RocksDB version 7.9.3. Due to
> this change, the `numberOfOpenFiles` metric no longer provides accurate
> information. The current workaround is to have it always return -1, which is
> misleading.
> The `numberOfOpenFiles` metric is: * *{*}Invalid:{*}* It does not reflect the
> actual number of open files in RocksDB. * *{*}Misleading:{*}* Returning -1
> can confuse users and monitoring systems. * *{*}Unnecessary:{*}* It provides
> no useful information in its current state.
> h4. Proposed Solution
> This Jira issue is to implement the changes proposed in KIP-1125:
> [[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]].
> The solution is to completely remove the `opennumberfiles` metric,
> including: * Removing the related code in `RocksDBMetricsRecorder`
> (`numberOfOpenFilesSensor`, recording logic). * Removing the
> `numberOfOpenFilesSensor` method in `RocksDBMetrics` * Updating the Kafka
> Streams documentation to reflect the metric's removal.
> h4. KIP
> KIP-1125:
> [[https://cwiki.apache.org/confluence/pages/resumedraft.action?draftId=337677055&draftShareId=6b7ff8cd-436c-4038-a7fd-de376429c1b2&;]]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
