[
https://issues.apache.org/jira/browse/SPARK-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375576#comment-14375576
]
Littlestar commented on SPARK-1480:
-----------------------------------
same as https://issues.apache.org/jira/browse/SPARK-6461
"run-example SparkPi" can reproduce this bug.
> Choose classloader consistently inside of Spark codebase
> --------------------------------------------------------
>
> Key: SPARK-1480
> URL: https://issues.apache.org/jira/browse/SPARK-1480
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Patrick Wendell
> Assignee: Patrick Wendell
> Priority: Blocker
> Fix For: 1.0.0
>
>
> The Spark codebase is not always consistent on which class loader it uses
> when classlaoders are explicitly passed to things like serializers. This
> caused SPARK-1403 and also causes a bug where when the driver has a modified
> context class loader it is not translated correctly in local mode to the
> (local) executor.
> In most cases what we want is the following behavior:
> 1. If there is a context classloader on the thread, use that.
> 2. Otherwise use the classloader that loaded Spark.
> We should just have a utility function for this and call that function
> whenever we need to get a classloader.
> Note that SPARK-1403 is a workaround for this exact problem (it sets the
> context class loader because downstream code assumes it is set). Once this
> gets fixed in a more general way SPARK-1403 can be reverted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
