dongjoon-hyun commented on a change in pull request #27526: [SPARK-30775][DOC]
Improve the description of executor metrics in the monitoring documentation.
URL: https://github.com/apache/spark/pull/27526#discussion_r399915414
##########
File path: docs/monitoring.md
##########
@@ -657,16 +657,60 @@ A list of the available metrics, with a short
description:
### Executor Metrics
-Executor-level metrics are sent from each executor to the driver as part of
the Heartbeat to describe the performance metrics of Executor itself like JVM
heap memory, GC information.
-Executor metric values and their measured peak values per executor are exposed
via the REST API at the end point `/applications/[app-id]/executors`.
-In addition, aggregated per-stage peak values of the executor metrics are
written to the event log if `spark.eventLog.logStageExecutorMetrics` is true.
-Executor metrics are also exposed via the Spark metrics system based on the
Dropwizard metrics library.
-A list of the available metrics, with a short description:
+Executor-level metrics are sent from each executor to the driver as part of
the Heartbeat to describe
+the performance metrics of Executor itself like JVM heap memory, GC
information.
+Executor metric values and their measured memory peak values per executor are
exposed via the REST API in JSON and Prometheus format.
+The JSON end point is exposed at: `/applications/[app-id]/executors`, the
Prometheus end point at: `/metrics/executors/prometheus`.
+The Prometheus end point is conditional to a configuration parameter:
`spark.ui.prometheus.enabled=true` (the default is `false`).
+In addition, aggregated per-stage peak values of the executor memory metrics
are written to the event log if
+`spark.eventLog.logStageExecutorMetrics` is true.
+Executor memory metrics are also exposed via the Spark metrics system based on
the Dropwizard metrics library.
+A list of the available executor metrics, with a short description:
<table class="table">
<tr><th>Executor Level Metric name</th>
<th>Short description</th>
</tr>
+ <tr>
+ <td>rddBlocks</td>
+ <td>RDD blocks in the block manager of this executor.</td>
+ </tr>
+ <tr>
+ <td>memoryUsed</td>
+ <td>Storage memory used by this executor.</td>
+ </tr>
+ <tr>
+ <td>diskUsed</td>
+ <td>Disk space used for RDD storage by this executor.</tr>
Review comment:
Missing `</td>`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
