AngersZhuuuu opened a new pull request #31001:
URL: https://github.com/apache/spark/pull/31001
### What changes were proposed in this pull request?
Add restful api for user to get stage level executor peak metrics
distribution.
- **/applications/<application_id>/stages/<stage_id>/< stage_attempt_id
>/executorMetricsDistribution** : distribution of peak values of executor
metrics for each executor for the stage, followed by peak values of executor
metrics for the stage
- **/applications/<application_id>/stages/<stage_id>/< stage_attempt_id
>/executorMetricsDistribution?quantiles=0.25,0.5,0.75** : summarize the
metrics with the given quantiles.
Example: <code>?quantiles=0.01,0.5,0.99</code>
### Why are the changes needed?
It can help Spark users debug/monitor a bottleneck of a stage
### Does this PR introduce _any_ user-facing change?
Usage as First section.
<tr>
<td><code>/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/executorMetricsDistribution</code></td>
<td>
Summary peak executor metrics of all executors in the given stage
attempt.
<br><code>?quantiles</code> summarize the metrics with the given
quantiles.
<br>Example: <code>?quantiles=0.01,0.5,0.99</code>
</td>
</tr>
### How was this patch tested?
Added UT
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
