Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22167#discussion_r215315657
--- Diff: docs/monitoring.md ---
@@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and
`spark.ui.retainedStages` that
Note that the garbage collection takes place on playback: it is possible
to retrieve
more entries by increasing these values and restarting the history server.
+### Executor Task Metrics
+
+The REST API exposes the values of the Task Metrics collected by Spark
executors at the
+task execution level. The metrics can be used for performance
troubleshooting.
+A list of the available metrics with a short description:
+
+<table class="table">
+ <tr><th>Spark Executor Task Metric name</th>
+ <th>Short description</th>
+ </tr>
+ <tr>
+ <td>executorRunTime</td>
+ <td>Time the executor spent running this task. This includes time
fetching shuffle data.
+ The value is expressed in milliseconds.</td>
+ </tr>
+ <tr>
+ <td>executorCpuTime
+ <td>CPU Time the executor spent running this task. This includes time
fetching shuffle data.
+ The value is expressed in nanoseconds.
+ </tr>
+ <tr>
+ <td>executorDeserializeTime</td>
+ <td>Time taken on the executor to deserialize this task.
+ The value is expressed in milliseconds.</td>
+ </tr>
+ <tr>
+ <td>executorDeserializeCpuTime</td>
+ <td>CPU Time taken on the executor to deserialize this task.
+ The value is expressed in nanoseconds.</td>
+ </tr>
+ <tr>
+ <td>resultSize</td>
+ <td>The number of bytes this task transmitted back to the driver as
the TaskResult.</td>
+ </tr>
+ <tr>
+ <td>jvmGCTime</td>
+ <td>Amount of time the JVM spent in garbage collection while executing
this task.
--- End diff --
Why do we start with `amount of` while the above parameters start `Time` or
`CPU Time`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
