This is an automated email from the ASF dual-hosted git repository. guozhang pushed a commit to branch 2.4 in repository https://gitbox.apache.org/repos/asf/kafka.git
The following commit(s) were added to refs/heads/2.4 by this push: new 2667773 KAFKA-8942: Document RocksDB metrics 2667773 is described below commit 26677731f87116154f74da421d4f440b80599da1 Author: Bruno Cadonna <br...@confluent.io> AuthorDate: Tue Oct 15 11:10:42 2019 -0700 KAFKA-8942: Document RocksDB metrics Author: Bruno Cadonna <br...@confluent.io> Reviewers: Guozhang Wang <wangg...@gmail.com> Closes #7490 from cadonna/AK8942-docs-rocksdb_metrics Minor comments (cherry picked from commit 9c80a06466dbfade96308ef26c20c6555612191e) Signed-off-by: Guozhang Wang <wangg...@gmail.com> --- docs/ops.html | 102 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/docs/ops.html b/docs/ops.html index 05d7de1..d87d16a 100644 --- a/docs/ops.html +++ b/docs/ops.html @@ -1979,6 +1979,108 @@ All the following metrics have a recording level of <code>debug</code>: </tbody> </table> + <h5><a id="kafka_streams_rocksdb_monitoring" href="#kafka_streams_rocksdb_monitoring">RocksDB Metrics</a></h5> + All the following metrics have a recording level of <code>debug</code>. + The metrics are collected every minute from the RocksDB state stores. + If a state store consists of multiple RocksDB instances as it is the case for aggregations over time and session windows, + each metric reports an aggregation over the RocksDB instances of the state store. + Note that the <code>store-scope</code> for built-in RocksDB state stores are currently the following: + <ul> + <li><code>rocksdb-state</code> (for RocksDB backed key-value store)</li> + <li><code>rocksdb-window-state</code> (for RocksDB backed window store)</li> + <li><code>rocksdb-session-state</code> (for RocksDB backed session store)</li> + </ul> + + <table class="data-table"> + <tbody> + <tr> + <th>Metric/Attribute name</th> + <th>Description</th> + <th>Mbean name</th> + </tr> + <tr> + <td>bytes-written-rate</td> + <td>The average number of bytes written per second to the RocksDB state store.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>bytes-written-total</td> + <td>The total number of bytes written to the RocksDB state store.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>bytes-read-rate</td> + <td>The average number of bytes read per second from the RocksDB state store.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>bytes-read-total</td> + <td>The total number of bytes read from the RocksDB state store.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>memtable-bytes-flushed-rate</td> + <td>The average number of bytes flushed per second from the memtable to disk.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>memtable-bytes-flushed-total</td> + <td>The total number of bytes flushed from the memtable to disk.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>memtable-hit-ratio</td> + <td>The ratio of memtable hits relative to all lookups to the memtable.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>block-cache-data-hit-ratio</td> + <td>The ratio of block cache hits for data blocks relative to all lookups for data blocks to the block cache.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>block-cache-index-hit-ratio</td> + <td>The ratio of block cache hits for index blocks relative to all lookups for index blocks to the block cache.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>block-cache-filter-hit-ratio</td> + <td>The ratio of block cache hits for filter blocks relative to all lookups for filter blocks to the block cache.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>write-stall-duration-avg</td> + <td>The average duration of write stalls in ms.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>write-stall-duration-total</td> + <td>The total duration of write stalls in ms.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>bytes-read-compaction-rate</td> + <td>The average number of bytes read per second during compaction.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>bytes-written-compaction-rate</td> + <td>The average number of bytes written per second during compaction.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>number-open-files</td> + <td>The number of current open files.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + <tr> + <td>number-file-errors-total</td> + <td>The total number of file errors occurred.</td> + <td>kafka.streams:type=stream-state-metrics,client-id=([-.\w]+),task-id=([-.\w]+),[store-scope]-id=([-.\w]+)</td> + </tr> + </tbody> + </table> + <h5><a id="kafka_streams_cache_monitoring" href="#kafka_streams_cache_monitoring">Record Cache Metrics</a></h5> All the following metrics have a recording level of <code>debug</code>: