weixiuli commented on code in PR #36162:
URL: https://github.com/apache/spark/pull/36162#discussion_r867336650
##########
core/src/main/scala/org/apache/spark/SparkStatusTracker.scala:
##########
@@ -120,4 +120,8 @@ class SparkStatusTracker private[spark] (sc: SparkContext,
store: AppStatusStore
exec.memoryMetrics.map(_.totalOnHeapStorageMemory).getOrElse(0L))
}.toArray
}
+
+ def getAppStatusStore: AppStatusStore = {
+ store
+ }
Review Comment:
@mridulm @Ngone51 Thanks for your suggestions, I submitted a new commit with
a small change in the AppStatusListener to improve tracking metrics for
successful and in-progress tasks, which might be cheaper to evaluate the
efficiency of task processing.
Why do we use an existing AppStatusListener instead of creating a new
Listener? Because using the AppStatusListener can reuse some data structures
and execution logic for us.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
