thejdeep opened a new pull request, #41017:
URL: https://github.com/apache/spark/pull/41017
When we calculate the quantile information from the peak executor metrics
values for the distribution, there is a possibility of running into an
`ArrayIndexOutOfBounds` exception when the metric values are empty. This PR
addresses that and fixes it by returning an empty array if the values are empty.
### Why are the changes needed?
Without these changes, when the withDetails query parameter is used to
query the stages REST API, we encounter a partial JSON response since the peak
executor metrics distribution cannot be serialized due to the above index error.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Added a unit test to test this behavior
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
