uncleGen commented on a change in pull request #28781:
URL: https://github.com/apache/spark/pull/28781#discussion_r437985761
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingQueryListenerBus.scala
##########
@@ -110,7 +116,15 @@ class StreamingQueryListenerBus(sparkListenerBus:
LiveListenerBus)
listener: StreamingQueryListener,
event: StreamingQueryListener.Event): Unit = {
def shouldReport(runId: UUID): Boolean = {
- activeQueryRunIds.synchronized { activeQueryRunIds.contains(runId) }
+ // When loaded by Spark History Server, we should process all event
coming from replay
+ // listener bus.
+ if (sparkListenerBus.isEmpty) {
Review comment:
In live ui, when one streaming query event posted.
`StreamingQueryListenerBus` will manage the query status. But in history ui,
there is no need to manage these status, and we should process all replayed
events.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
