attilapiros commented on a change in pull request #24303:
[SPARK-27394][WebUI]Flush LiveEntity if necessary when receiving
SparkListenerExecutorMetricsUpdate
URL: https://github.com/apache/spark/pull/24303#discussion_r272782834
##########
File path: core/src/test/scala/org/apache/spark/ui/UISeleniumSuite.scala
##########
@@ -725,6 +729,30 @@ class UISeleniumSuite extends SparkFunSuite with
WebBrowser with Matchers with B
}
}
+ test("Staleness of Spark UI should not last minutes or hours") {
+ withSpark(newSparkContext(
+ master = "local[2]",
+ // Set a small heart beat interval to make the test fast
+ additionalConfs = Map(
+ EXECUTOR_HEARTBEAT_INTERVAL.key -> "10ms",
+ LIVE_ENTITY_UPDATE_STALENESS_LIMIT.key -> "10ms"))) { sc =>
+ val f = sc.parallelize(1 to 1000, 1000).foreachAsync { _ =>
+ // Make the task never finish so there won't be any task start/end
events after the first 2
+ // tasks start.
+ Thread.sleep(300000)
Review comment:
I have checked and the thread leaks are gone.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
