HeartSaVioR commented on code in PR #37213:
URL: https://github.com/apache/spark/pull/37213#discussion_r924027152
##########
sql/core/src/test/scala/org/apache/spark/sql/streaming/EventTimeWatermarkSuite.scala:
##########
@@ -261,28 +263,24 @@ class EventTimeWatermarkSuite extends StreamTest with
BeforeAndAfter with Matche
// Offset log should have watermark recorded as 5.
*/
- StartStream(Trigger.Once),
+ StartStream(Trigger.AvailableNow),
awaitTermination(),
AddData(inputData, 25),
- StartStream(Trigger.Once, checkpointLocation =
checkpointDir.getAbsolutePath),
+ StartStream(Trigger.AvailableNow, checkpointLocation =
checkpointDir.getAbsolutePath),
awaitTermination(),
- CheckNewAnswer(),
- assertEventStats(min = 25, max = 25, avg = 25, wtrmark = 5),
- // watermark should be updated to 25 - 10 = 15
+ CheckNewAnswer((10, 3)), // watermark should be updated to 25 - 10 = 15
Review Comment:
Here the change is actually based on the good thing being provided by
Trigger.AvailableNow - Trigger.AvailableNow deals with no-data batch as well,
hence the output is being emitted earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
