uros-b commented on code in PR #56790:
URL: https://github.com/apache/spark/pull/56790#discussion_r3479749881
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala:
##########
@@ -1119,8 +1120,13 @@ class SQLAppStatusListenerMemoryLeakSuite extends
SparkFunSuite {
val statusStore = spark.sharedState.statusStore
assert(statusStore.executionsCount() <= 50)
assert(statusStore.planGraphCount() <= 50)
- // No live data should be left behind after all executions end.
- assert(statusStore.listener.get.noLiveData())
+ // No live data should be left behind after all executions end. The
cleanup of live
Review Comment:
The added comment / PR body pin the flake on the kvstore.doAsync cleanup
completing after the bus drains, but the test sets
ASYNC_TRACKING_ENABLED=false, which makes doAsync run synchronously on the
listener thread (sameThreadExecutor); so waitUntilEmpty() should already cover
that path. The stated root cause likely doesn't hold under this config; the
real residual (e.g. stageMetrics cleanup) isn't identified. The fix is still
safe, but please clarify the actual race (or confirm the CI env doesn't apply
the config) so this isn't masking an undiagnosed condition. Tighten the inline
comment accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
