ron8hu commented on a change in pull request #29247:
URL: https://github.com/apache/spark/pull/29247#discussion_r551608679
##########
File path:
core/src/main/scala/org/apache/spark/status/api/v1/OneApplicationResource.scala
##########
@@ -52,6 +53,25 @@ private[v1] class AbstractApplicationResource extends
BaseAppResource {
@Path("executors")
def executorList(): Seq[ExecutorSummary] = withUI(_.store.executorList(true))
+ @GET
+ @Path("executorMetricsDistribution")
+ def executorSummary(
+ @QueryParam("activeOnly") @DefaultValue("true") activeOnly: Boolean,
+ @DefaultValue("0.05,0.25,0.5,0.75,0.95") @QueryParam("quantiles")
quantileString: String)
Review comment:
FYI. In the corresponding web UI, the quantiles of Min/Max are
displayed in the table "Summary Metrics for Completed Tasks" for a given stage
page. In a parallel system, the duration of a stage is often determined by the
slowest task/executor. To monitor/debug a skew issue, the maximal value (or
100% percentile value) is more useful than the 95% percentile value. On the
other hand, 95% percentile value has been used in the past. One wise man once
told me: Consistency means to repeat yesterday's mistake.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
