[
https://issues.apache.org/jira/browse/SPARK-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244478#comment-15244478
]
Josh Rosen commented on SPARK-14686:
------------------------------------
The idea of making these properties non-inheritable has come up in the past
because of other problems that this inheritance has caused. Here are a few
relevant JIRA tickets:
- https://issues.apache.org/jira/browse/SPARK-12964
- https://issues.apache.org/jira/browse/SPARK-10407
- https://issues.apache.org/jira/browse/SPARK-10563
- https://issues.apache.org/jira/browse/SPARK-6629
We should try to summarize all of these to figure out whether we can just
remove the inheritance.
> Implement a non-inheritable localProperty facility
> --------------------------------------------------
>
> Key: SPARK-14686
> URL: https://issues.apache.org/jira/browse/SPARK-14686
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Marcin Tustin
> Priority: Minor
>
> As discussed here:
> http://mail-archives.apache.org/mod_mbox/spark-dev/201604.mbox/%3CCANXtaKA4ZdpiUbZPnzDBN8ZL7_RwKSMuz6n45ixnHVEBE1hgjg%40mail.gmail.com%3E
> Spark localProperties are always inherited by spawned threads. There are
> situations in which this is undesirable (notably spark.sql.execution.id and
> any other localProperty that should always be cleaned up). This is a ticket
> to implement a non-inheritable mechanism for localProperties.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
