[
https://issues.apache.org/jira/browse/SPARK-43334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718704#comment-17718704
]
Hudson commented on SPARK-43334:
--------------------------------
User 'thejdeep' has created a pull request for this issue:
https://github.com/apache/spark/pull/41017
> Error while serializing ExecutorPeakMetricsDistributions into API JSON
> response
> -------------------------------------------------------------------------------
>
> Key: SPARK-43334
> URL: https://issues.apache.org/jira/browse/SPARK-43334
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 3.3.3
> Reporter: Thejdeep
> Priority: Major
>
> When we try to get the ExecutorPeakMetricsDistributions [through the
> API|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/status/api/v1/api.scala#L463]
> (/stages), there is a possibility of encountering an issue while serializing
> the StagesData into a JSON if the executor metrics are empty.
>
> The following error is thrown :
> {code:java}
> Caused by: com.fasterxml.jackson.databind.JsonMappingException: -1 (through
> reference chain: scala.c
> ollection.immutable.$colon$colon[0]->org.apache.spark.status.api.v1.StageData["executorMetricsDistri
> butions"]->org.apache.spark.status.api.v1.ExecutorMetricsDistributions["peakMemoryMetrics"])
> at
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.jav
> a:390)
> at
> com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.jav
> a:349) {code}
> This happens because the indices for the quartiles are populated incorrectly
> as -1 since the metrics itself are empty and this leads to this exception
> being thrown.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
