linhongliu-db commented on code in PR #35856:
URL: https://github.com/apache/spark/pull/35856#discussion_r844630790
##########
sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala:
##########
@@ -118,6 +119,12 @@ private[sql] class SharedState(
statusStore
}
+ sparkContext.statusStore.diskStore.foreach { kvStore =>
+ sparkContext.listenerBus.addToQueue(
+ new DiagnosticListener(conf, kvStore.asInstanceOf[ElementTrackingStore]),
Review Comment:
@dongjoon-hyun
I used the same way that the in-memory kvStore used. which is:
1. define a shared kvStore in SparkContext
2. share the kvStore with the listeners that need to update the store (Jobs
UI Listener, SQL UI Listener)
3. share the kvStore with the components that need to read the store (web
UI, rest API)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
