weixiuli commented on code in PR #36162:
URL: https://github.com/apache/spark/pull/36162#discussion_r862447219
##########
core/src/main/scala/org/apache/spark/SparkStatusTracker.scala:
##########
@@ -120,4 +120,8 @@ class SparkStatusTracker private[spark] (sc: SparkContext,
store: AppStatusStore
exec.memoryMetrics.map(_.totalOnHeapStorageMemory).getOrElse(0L))
}.toArray
}
+
+ def getAppStatusStore: AppStatusStore = {
+ store
+ }
Review Comment:
It might not be able to fetch the stage/task state (metrics) from
TaskSetManager directly. Because the tasks are running in the executors, and
the executors report (heartbeat) their task metrics to the driver, the driver
gets the Heartbeat and posts a `SparkListenerExecutorMetricsUpdate` message
to the listenerBus, and AppStatusListener.onExecutorMetricsUpdate will get the
ExecutorMetrics and update them. So we may get the task Metrics from
AppStatusStore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
