[
https://issues.apache.org/jira/browse/SPARK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-4194.
------------------------------
Resolution: Fixed
Fix Version/s: 1.4.0
Issue resolved by pull request 5335
[https://github.com/apache/spark/pull/5335]
> Exceptions thrown during SparkContext or SparkEnv construction might lead to
> resource leaks or corrupted global state
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4194
> URL: https://issues.apache.org/jira/browse/SPARK-4194
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Josh Rosen
> Priority: Critical
> Fix For: 1.4.0
>
>
> The SparkContext and SparkEnv constructors instantiate a bunch of objects
> that may need to be cleaned up after they're no longer needed. If an
> exception is thrown during SparkContext or SparkEnv construction (e.g. due to
> a bad configuration setting), then objects created earlier in the constructor
> may not be properly cleaned up.
> This is unlikely to cause problems for batch jobs submitted through
> {{spark-submit}}, since failure to construct SparkContext will probably cause
> the JVM to exit, but it is a potentially serious issue in interactive
> environments where a user might attempt to create SparkContext with some
> configuration, fail due to an error, and re-attempt the creation with new
> settings. In this case, resources from the previous creation attempt might
> not have been cleaned up and could lead to confusing errors (especially if
> the old, leaked resources share global state with the new SparkContext).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
