uros-b commented on code in PR #56790:
URL: https://github.com/apache/spark/pull/56790#discussion_r3479752397
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala:
##########
@@ -1119,8 +1120,13 @@ class SQLAppStatusListenerMemoryLeakSuite extends
SparkFunSuite {
val statusStore = spark.sharedState.statusStore
assert(statusStore.executionsCount() <= 50)
assert(statusStore.planGraphCount() <= 50)
- // No live data should be left behind after all executions end.
- assert(statusStore.listener.get.noLiveData())
+ // No live data should be left behind after all executions end. The
cleanup of live
+ // executions/stage metrics is finalized when the metrics aggregation
triggered by the
+ // SQLExecutionEnd event completes, so wait for the listener to drain
rather than asserting
+ // immediately to avoid a timing race.
+ eventually(timeout(10.seconds), interval(10.milliseconds)) {
Review Comment:
Note: stacking a 10s eventually on top of the existing 10s waitUntilEmpty()
doubles worst-case hang to ~20s on a genuinely broken system.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
