Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/20940#discussion_r179796802
--- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala ---
@@ -268,6 +268,9 @@ private class LiveExecutor(val executorId: String,
_addTime: Long) extends LiveE
def hasMemoryInfo: Boolean = totalOnHeap >= 0L
+ // peak values for executor level metrics
+ var peakExecutorMetrics = new PeakExecutorMetrics
--- End diff --
I think this can be a `val` -- its mutated, not replaced
Also I don't see any tests that this is updated correctly, both in a live
app and when replayed from the limited logs? OTOH, maybe you should just
remove this from this change, and wait till SPARK-23431 as there you'll need to
add something else to expose this for each executor / stage, which may be more
important than just the overall total for the executor.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
