HeartSaVioR opened a new pull request #25725: [SPARK-24663][STREAMING][TESTS] StreamingContextSuite: Wait until slow receiver has been initialized, but with hard timeout URL: https://github.com/apache/spark/pull/25725 ### What changes were proposed in this pull request? This patch fixes the flaky test failure from StreamingContextSuite "stop slow receiver gracefully", via putting flag whether initializing slow receiver is completed, and wait for such flag to be true. As receiver should be submitted via job and initialized in executor, 500ms might not be enough for covering all cases. ### Why are the changes needed? We got some reports for test failure on this test. Please refer [SPARK-24663](https://issues.apache.org/jira/browse/SPARK-24663) ### Does this PR introduce any user-facing change? No ### How was this patch tested? Modified UT. I've artificially made delay on handling job submission via adding below code in `DAGScheduler.submitJob`: ``` if (rdd != null && rdd.name != null && rdd.name.startsWith("Receiver")) { println(s"Receiver Job! rdd name: ${rdd.name}") Thread.sleep(5000) } ``` and the test "stop slow receiver gracefully" failed on current master and passed on the patch.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
