Juliusz Sompolski created SPARK-44730: -----------------------------------------
Summary: Spark Connect: Cleaner thread not stopped when SparkSession stops Key: SPARK-44730 URL: https://issues.apache.org/jira/browse/SPARK-44730 Project: Spark Issue Type: Bug Components: Connect Affects Versions: 3.5.0, 4.0.0 Reporter: Juliusz Sompolski Spark Connect scala client SparkSession has a cleaner, which starts a daemon thread to clean up Closeable objects after GC. This daemon thread is never stopped, and every SparkSession creates a new one. Cleaner implements a stop() function, but no-one ever calls it. Possibly because even after SparkSession.stop(), the cleaner may still be needed when remaining references are GCed... For this reason it seems that the Cleaner should rather be a global singleton than within a session. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org