panbingkun commented on code in PR #39494:
URL: https://github.com/apache/spark/pull/39494#discussion_r1066758271
##########
sql/core/src/test/scala/org/apache/spark/sql/streaming/ui/StreamingQueryStatusListenerSuite.scala:
##########
@@ -181,9 +181,15 @@ class StreamingQueryStatusListenerSuite extends StreamTest
{
val terminateEvent1 = new StreamingQueryListener.QueryTerminatedEvent(id1,
runId1, None)
listener.onQueryTerminated(terminateEvent1)
checkInactiveQueryStatus(1, Seq(id1))
+ // SPARK-41972: having a short sleep here to make sure the end time of
query 2 is larger than
+ // query 1.
+ Thread.sleep(20)
Review Comment:
When I run it 100 times, I reproduce issue.
RUN1 - id: 017350a8-fb82-475a-86d5-c39e25a3410e, runId:
b09905df-70d5-4659-94f5-7a2f88385317
RUN2 - id: fabdb4cc-7849-4a46-aef0-92fcc5879a2c, runId:
524ecd20-b351-40ae-9f7b-8d6aa6032f42
RUN3 - id: 331a07fa-8195-44a6-a4ce-12036d63c19f, runId:
fec57472-c1be-4552-8d31-aa1982675322
delete runId: List(524ecd20-b351-40ae-9f7b-8d6aa6032f42),
inactiveQueries:
524ecd20-b351-40ae-9f7b-8d6aa6032f42, fabdb4cc-7849-4a46-aef0-92fcc5879a2c,
Some(_1673428448998_)
b09905df-70d5-4659-94f5-7a2f88385317, 017350a8-fb82-475a-86d5-c39e25a3410e,
Some(_1673428448998_)
fec57472-c1be-4552-8d31-aa1982675322, 331a07fa-8195-44a6-a4ce-12036d63c19f,
Some(_1673428448998_)
When `endTimestamp` have equal values, will delete a random value
<img width="470" alt="image"
src="https://user-images.githubusercontent.com/15246973/211768949-8a3606e3-4a92-41c2-8ec0-5b0969218948.png";>
In fact, we only need to sleep for 1 millisecond to fix the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/15246973/211768949-8a3606e3-4a92-41c2-8ec0-5b0969218948.png"](https://user-images.githubusercontent.com/15246973/211768949-8a3606e3-4a92-41c2-8ec0-5b0969218948.png"){kind=link}