[
https://issues.apache.org/jira/browse/SPARK-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-4194:
-----------------------------
Target Version/s: (was: 1.2.0)
Affects Version/s: 1.2.0
1.3.0
Assignee: Marcelo Vanzin
> Exceptions thrown during SparkContext or SparkEnv construction might lead to
> resource leaks or corrupted global state
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-4194
> URL: https://issues.apache.org/jira/browse/SPARK-4194
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.0, 1.3.0
> Reporter: Josh Rosen
> Assignee: Marcelo Vanzin
> Priority: Critical
> Fix For: 1.4.0
>
>
> The SparkContext and SparkEnv constructors instantiate a bunch of objects
> that may need to be cleaned up after they're no longer needed. If an
> exception is thrown during SparkContext or SparkEnv construction (e.g. due to
> a bad configuration setting), then objects created earlier in the constructor
> may not be properly cleaned up.
> This is unlikely to cause problems for batch jobs submitted through
> {{spark-submit}}, since failure to construct SparkContext will probably cause
> the JVM to exit, but it is a potentially serious issue in interactive
> environments where a user might attempt to create SparkContext with some
> configuration, fail due to an error, and re-attempt the creation with new
> settings. In this case, resources from the previous creation attempt might
> not have been cleaned up and could lead to confusing errors (especially if
> the old, leaked resources share global state with the new SparkContext).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
