tgravescs commented on a change in pull request #29247:
URL: https://github.com/apache/spark/pull/29247#discussion_r523029232
##########
File path:
core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -52,6 +53,25 @@ private[v1] class AbstractApplicationResource extends
BaseAppResource {
@Path("executors")
def executorList(): Seq[ExecutorSummary] = withUI(_.store.executorList(true))
+ @GET
+ @Path("executorSummary")
Review comment:
should be executorMetricsSummary or perhaps this should be
executormetricsDistribution or Quantiles or something. The metrics themselves
are in the /executors path in ExecutorSummary, correct?
##########
File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala
##########
@@ -366,6 +366,18 @@ private[spark] class AppStatusStore(
Some(computedQuantiles)
}
+ /**
+ * Calculates a summary of the executor metrics for executors, returning the
+ * requested quantiles for the recorded metrics.
+ */
+ def executorMetricSummary(activeOnly: Boolean,
+ unsortedQuantiles: Array[Double])
Review comment:
indentation si off
##########
File path: docs/monitoring.md
##########
@@ -518,6 +518,15 @@ can be identified by their `[attempt-id]`. In the API
listed below, when running
<td><code>/applications/[app-id]/allexecutors</code></td>
<td>A list of all(active and dead) executors for the given
application.</td>
</tr>
+ <tr>
+ <td><code>/applications/[app-id]/executorSummary</code></td>
+ <td>
+ Summary peak memory metrics of executors.
Review comment:
this is misleading the peak memory metrics are in the executors rest api
already right?
This seems just to add in the ability to get them as quantiles?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
