Jennifer88huang commented on a change in pull request #4910: [doc] Add a
documentation page for metrics reference
URL: https://github.com/apache/pulsar/pull/4910#discussion_r313788304
##########
File path: site2/docs/reference-metrics.md
##########
@@ -0,0 +1,243 @@
+---
+id: reference-metrics
+title: Pulsar Metrics
+sidebar_label: Pulsar Metrics
+---
+
+<style type="text/css">
+ table{
+ font-size: 80%;
+ }
+</style>
+
+Pulsar exposes metrics in Prometheus format that can be collected and used for
monitoring the health of the cluster.
+
+* [ZooKeeper](#zookeeper)
+* [BookKeeper](#bookkeeper)
+* [Broker](#broker)
+
+## Overview
+
+The metrics exposed by Pulsar are in Prometheus format. The types of metrics
are:
+
+- [Counter](https://prometheus.io/docs/concepts/metric_types/#counter): a
cumulative metric that represents a single monotonically increasing counter
whose value can only increase or be reset to zero on restart.
+- [Gauge](https://prometheus.io/docs/concepts/metric_types/#gauge): a *gauge*
is a metric that represents a single numerical value that can arbitrarily go up
and down.
+- [Histogram](https://prometheus.io/docs/concepts/metric_types/#histogram): a
histogram samples observations (usually things like request durations or
response sizes) and counts them in configurable buckets.
+- [Summary](https://prometheus.io/docs/concepts/metric_types/#summary):
similar to a histogram, a summary samples observations (usually things like
request durations and response sizes). While it also provides a total count of
observations and a sum of all observed values, it calculates configurable
quantiles over a sliding time window.
+
+## ZooKeeper
+
+The zookeeper metrics are exposed under "/metrics" at port 8000. You can
change the port by configuring a system
+property `stats_server_port` to use a different port.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_znode_count | Gauge | Number of z-nodes stored. |
+| zookeeper_server_data_size_bytes | Gauge | The total size of all of z-nodes
stored. |
+| zookeeper_server_connections | Gauge | Number of currently opened
connections. |
+| zookeeper_server_watches_count | Gauge | Number of watchers registered. |
+| zookeeper_server_ephemerals_count | Gauge | Number of ephemeral z-nodes. |
+
+### Request metrics
+
+| Name | Type | Description |
+|---|---|---|
+| zookeeper_server_requests | Counter | The total number of requests received
by a particular server. |
+| zookeeper_server_requests_latency_ms | Summary | The requests latency
calculated in milliseconds. <br> Available labels: *type* (write, read). <br>
<ul><li>*write*: the requests that write data to zookeeper.</li><li>*read*: the
requests that read data from zookeeper.</li></ul>|
+
+## BookKeeper
+
+The bookkeeper metrics are exposed under "/metrics" at port 8000. You can
change the port by updating `prometheusStatsHttpPort`
+in `bookkeeper.conf` configuration file.
+
+### Server metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_SERVER_STATUS | Gauge | The server status for bookie server.
<br><ul><li>1: the bookie is running in writable mode.</li><li>0: the bookie is
running in readonly mode.</li></ul> |
+| bookkeeper_server_ADD_ENTRY_count | Counter | The total number of ADD_ENTRY
requests received at the bookie. Label `success` used to distinguish successes
and failures. |
+| bookkeeper_server_READ_ENTRY_count | Counter | The total number of
READ_ENTRY requests received at the bookie. Label `success` used to distinguish
successes and failures. |
+| bookie_WRITE_BYTES | Counter | The total number of bytes written to the
bookie. |
+| bookie_READ_BYTES | Counter | The total number of bytes read from the
bookie. |
+| bookkeeper_server_ADD_ENTRY_REQUEST | Histogram | The histogram of request
latency of ADD_ENTRY requests at the bookie. Label `success` used to
distinguish successes and failures. |
+| bookkeeper_server_READ_ENTRY_REQUEST | Histogram | The histogram of request
latency of READ_ENTRY requests at the bookie. Label `success` used to
distinguish successes and failures. |
+
+### Journal metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_journal_JOURNAL_SYNC_count | Counter | The total number of journal
fsync operations happening at the bookie. Label `success` used to distinguish
successes and failures. |
+| bookie_journal_JOURNAL_QUEUE_SIZE | Gauge | The total number of requests
pending in the journal queue. |
+| bookie_journal_JOURNAL_FORCE_WRITE_QUEUE_SIZE | Gauge | The total number of
force write (fsync) requests pending in the force-write queue. |
+| bookie_journal_JOURNAL_CB_QUEUE_SIZE | Gauge | The total number of callbacks
pending in the callback queue. |
+| bookie_journal_JOURNAL_ADD_ENTRY | Histogram | The histogram of request
latency of adding entries to the journal. |
+| bookie_journal_JOURNAL_SYNC | Histogram | The histogram of fsync latency of
syncing data to the journal disk. |
+
+### Storage metrics
+
+| Name | Type | Description |
+|---|---|---|
+| bookie_ledgers_count | Gauge | The total number of ledgers stored in the
bookie. |
+| bookie_entries_count | Gauge | The total number of entries stored in the
bookie. |
+| bookie_write_cache_size | Gauge | The bookie write cache size (in bytes). |
+| bookie_read_cache_size | Gauge | The bookie read cache size (in bytes). |
+| bookie_DELETED_LEDGER_COUNT | Counter | The total number of ledgers deleted
since the bookie starts. |
+| bookie_ledger_writable_dirs | Gauge | The number of writable directories in
the bookie. |
+
+## Broker
+
+The broker metrics are exposed under "/metrics" at port 8080. You can change
the port by updating `webServicePort` to a different port
+in `broker.conf` configuration file.
+
+All the metrics exposed by a broker are labelled with
`cluster=${pulsar_cluster}`. The value of `${pulsar_cluster}` is the pulsar
cluster
+name you configured in `broker.conf`.
+
+Broker has the following kinds of metrics:
+
+* [Namespace metrics](#namespace-metrics)
+ * [Replication metrics](#replication-metrics)
+* [Topic metrics](#topic-metrics)
+ * [Replication metrics](#replication-metrics-1)
+* [Subscription metrics](#subscription-metrics)
+* [Consumer metrics](#consumer-metrics)
+
+### Namespace metrics
+
+> Namespace metrics are only exposed when
`exposeTopicLevelMetricsInPrometheus` is set to false.
Review comment:
```suggestion
> Namespace metrics are only exposed when
`exposeTopicLevelMetricsInPrometheus` is set to `false`.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
