Github user tgravescs commented on a diff in the pull request:
https://github.com/apache/spark/pull/21688#discussion_r224850397
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -341,7 +341,9 @@ private class LiveExecutorStageSummary(
metrics.shuffleWriteMetrics.recordsWritten,
metrics.memoryBytesSpilled,
metrics.diskBytesSpilled,
- isBlacklisted)
+ isBlacklisted,
--- End diff --
@vanzin ideas on how to better handle this?
I don't see a real clean way to populate these fields from the
AppstatusListener before being written. For context, in this PR these are
currently being populated in the AppStatusStore.executorSummary call before
going back to user.
We could potentially split into separate api or on UI side query both that
and the executor info and join but seems like a lot more data.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
