Github user marmbrus commented on a diff in the pull request:
https://github.com/apache/spark/pull/13335#discussion_r64960190
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/util/ContinuousQueryListener.scala
---
@@ -65,11 +65,33 @@ object ContinuousQueryListener {
trait Event
/** Event representing the start of a query */
- class QueryStarted private[sql](val query: ContinuousQuery) extends Event
+ class QueryStarted private[sql](val queryInfo: ContinuousQueryInfo)
extends Event
/** Event representing any progress updates in a query */
- class QueryProgress private[sql](val query: ContinuousQuery) extends
Event
+ class QueryProgress private[sql](val queryInfo: ContinuousQueryInfo)
extends Event
- /** Event representing that termination of a query */
- class QueryTerminated private[sql](val query: ContinuousQuery) extends
Event
+ /**
+ * Event representing that termination of a query
+ *
+ * @param queryInfo
+ * @param exception The excpetion information of the [[ContinuousQuery]]
if any. Otherwise, it
+ * will be `None`.
+ */
+ class QueryTerminated private[sql](
+ val queryInfo: ContinuousQueryInfo,
+ val exception: Option[String]) extends Event
}
+
+/**
+ * :: Experimental ::
+ * A class that contains information about [[ContinuousQuery]].
+ *
+ * @param name The [[ContinuousQuery]] name
+ * @param sourceStatuses The current statuses of the [[ContinuousQuery]]'s
sources.
+ * @param sinkStatus The current status of the [[ContinuousQuery]]'s sink.
+ */
+@Experimental
+class ContinuousQueryInfo private[sql](
--- End diff --
A few more questions:
- Is there a reason this isn't in the object with the other listener
messages? At a minimum it should probably be defined near `SourceStatus`
(which is currently defined at the top level). We should find a place to
colocate all of them.
- Is `name` enough to uniquely identify a streaming query?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
