Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19981#discussion_r157153841
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala ---
@@ -82,6 +82,19 @@ private[sql] class SharedState(val sparkContext:
SparkContext) extends Logging {
*/
val cacheManager: CacheManager = new CacheManager
+ /**
+ * A status store to query SQL status/metrics of this Spark application,
based on SQL-specific
+ * [[org.apache.spark.scheduler.SparkListenerEvent]]s.
+ */
+ val statusStore: SQLAppStatusStore = {
+ val kvStore = sparkContext.statusStore.store
+ val listener = new SQLAppStatusListener(sparkContext.conf, kvStore,
live = true)
+ sparkContext.listenerBus.addToStatusQueue(listener)
--- End diff --
There seems a global sql listener at `SparkSession` before. Now we create
new sql listener for each `SharedState`? I think `SharedState` should be
reused, but not sure if any case we create more than one `SharedState`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
