HeartSaVioR commented on a change in pull request #26610: [SPARK-29973][SS]
Make `processedRowsPerSecond` calculated more accurately and meaningfully
URL: https://github.com/apache/spark/pull/26610#discussion_r348915783
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryStatusAndProgressSuite.scala
##########
@@ -215,6 +218,60 @@ class StreamingQueryStatusAndProgressSuite extends
StreamTest with Eventually {
}
}
+ test("SPARK-29973: Make `processedRowsPerSecond` calculated more accurately
and meaningfully") {
Review comment:
The CI build fails here, and in fact this UT doesn't test the actual change.
Wouldn't it pass the test with current master branch?
Actually it can't be tested unless modifying clock to support
sub-milliseconds (say, microseconds), but it would bring too many change
compared to the actual change we do. (Still be able to test with 0
milliseconds, but that's slightly different than what we intended.)
Personally I'm OK to skip adding the UT in this case, but may need to hear
others' voices as well.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
