Github user merlintang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21504#discussion_r193911087
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/StreamingQueryManager.scala
---
@@ -55,6 +56,11 @@ class StreamingQueryManager private[sql] (sparkSession:
SparkSession) extends Lo
@GuardedBy("awaitTerminationLock")
private var lastTerminatedQuery: StreamingQuery = null
+ sparkSession.sparkContext.conf.get(STREAMING_QUERY_LISTENERS).foreach {
classNames =>
+ Utils.loadExtensions(classOf[StreamingQueryListener], classNames,
+ sparkSession.sparkContext.conf).foreach(addListener)
+ }
+
--- End diff --
two comments here:
1. we need to log the registration here
2. we need to use try catch for this, it is possible that register fail.
this would break the job.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
