[
https://issues.apache.org/jira/browse/SPARK-44730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Juliusz Sompolski resolved SPARK-44730.
---------------------------------------
Resolution: Not A Problem
> Spark Connect: Cleaner thread not stopped when SparkSession stops
> -----------------------------------------------------------------
>
> Key: SPARK-44730
> URL: https://issues.apache.org/jira/browse/SPARK-44730
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: 3.5.0, 4.0.0
> Reporter: Juliusz Sompolski
> Priority: Minor
>
> Spark Connect scala client SparkSession has a cleaner, which starts a daemon
> thread to clean up Closeable objects after GC. This daemon thread is never
> stopped, and every SparkSession creates a new one.
> Cleaner implements a stop() function, but no-one ever calls it. Possibly
> because even after SparkSession.stop(), the cleaner may still be needed when
> remaining references are GCed... For this reason it seems that the Cleaner
> should rather be a global singleton than within a session.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
