[
https://issues.apache.org/jira/browse/SPARK-5358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-5358.
------------------------------
Resolution: Duplicate
OK as I understand this was covered by SPARK-2996
> spark.files.userClassPathFirst doesn't work correctly
> -----------------------------------------------------
>
> Key: SPARK-5358
> URL: https://issues.apache.org/jira/browse/SPARK-5358
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.1.0, 1.2.0
> Reporter: Matthew Whelan
>
> org.apache.spark.executor.ChildExecutorURLClassLoader#findClass delegates to
> two different classloaders: a parent-less URL classloader, and the parent
> classloader, as a fallback. The delegation is broken such that calling
> loadClass twice in succession with the same parameters will fail the second
> time.
> The delegation to the userClassLoader calls findClass directly, which
> bypasses the classloader's cache. So userClassLoader will attempt to define
> the same class multiple times, throwing LinkageErrors after the first time.
> The canonical way to change the default delegation scheme is to override
> loadClass, rather than just findClass. It also might be sufficient to have
> userClassLoader.findClass call super.loadClass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
