AngersZhuuuu commented on a change in pull request #30573:
URL: https://github.com/apache/spark/pull/30573#discussion_r535906072
##########
File path: core/src/main/scala/org/apache/spark/status/LiveEntity.scala
##########
@@ -386,7 +386,7 @@ private class LiveExecutorStageSummary(
metrics.memoryBytesSpilled,
metrics.diskBytesSpilled,
isExcluded,
- Some(peakExecutorMetrics).filter(_.isSet),
+ Some(peakExecutorMetrics),
Review comment:
> I think it's better to resolve in the read side. Otherwise, there is
no need to make this field optional as well
The previous PR make this value as Option.
Since currently, the way got executor summary data change
```
stageExecutorSummaryInfoKeys.forEach(function (columnKeyIndex) {
var executorSummary =
responseBody.executorSummary[columnKeyIndex];
var executorDetail =
executorDetailsMap[columnKeyIndex.toString()];
executorSummary.id = columnKeyIndex;
executorSummary.executorLogs = {};
executorSummary.hostPort = "CANNOT FIND ADDRESS";
if (executorDetail) {
if (executorDetail["executorLogs"]) {
responseBody.executorSummary[columnKeyIndex].executorLogs =
executorDetail["executorLogs"];
}
if (executorDetail["hostPort"]) {
responseBody.executorSummary[columnKeyIndex].hostPort =
executorDetail["hostPort"];
}
}
executorSummaryTable.push(responseBody.executorSummary[columnKeyIndex]);
});
```
If not change this, throw exception since no `peakMemoryMetrics` value
passed to restful api response
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
