Andrew Or created SPARK-6132:
--------------------------------
Summary: Context cleaner thread lives across SparkContexts
Key: SPARK-6132
URL: https://issues.apache.org/jira/browse/SPARK-6132
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.3.0
Reporter: Andrew Or
Assignee: Andrew Or
The context cleaner thread is not stopped properly. If a SparkContext is
started immediately after one stops, the context cleaner of the former can
clean variables in the latter.
This is because the cleaner.stop() just sets a flag and expects the thread to
terminate asynchronously, but the code to clean broadcasts goes through
`SparkEnv.get.blockManager`, which could belong to a different SparkContext.
The right behavior is to wait until all currently running clean up tasks have
finished.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
