Github user carsonwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/9991#discussion_r46014967
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala
---
@@ -343,6 +343,8 @@ class SQLListenerMemoryLeakSuite extends SparkFunSuite {
.set("spark.sql.ui.retainedExecutions", "50") // Set it to 50 to run
this test quickly
val sc = new SparkContext(conf)
try {
+ // Clear the sql listener created by a previous test suite.
+ SQLContext.clearSqlListener()
--- End diff --
I think we can add a SparkContext stop hook. When SparkContext is being
stopped, clear the reference. The user doesn't have to call a method to clear
the sqlListener reference. The sqlListener is added to SparkContext and will
only be garbage collected when SparkContext is stopped.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
