[
https://issues.apache.org/jira/browse/SPARK-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375572#comment-14375572
]
Littlestar commented on SPARK-1480:
-----------------------------------
I meet this bug on spark 1.3.0 + mesos 0.21.1....
100%..
I0323 16:32:18.933440 14504 fetcher.cpp:64] Extracted resource
'/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S4/frameworks/20150323-152848-1214949568-5050-21134-0009/executors/20150323-100710-1214949568-5050-3453-S4/runs/3d8f22f5-7fed-44ed-b5f9-98a219133911/spark-1.3.0-bin-2.4.0.tar.gz'
into
'/home/mesos/work_dir/slaves/20150323-100710-1214949568-5050-3453-S4/frameworks/20150323-152848-1214949568-5050-21134-0009/executors/20150323-100710-1214949568-5050-3453-S4/runs/3d8f22f5-7fed-44ed-b5f9-98a219133911'
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/executor/MesosExecutorBackend
Caused by: java.lang.ClassNotFoundException:
org.apache.spark.executor.MesosExecutorBackend
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: org.apache.spark.executor.MesosExecutorBackend
> Choose classloader consistently inside of Spark codebase
> --------------------------------------------------------
>
> Key: SPARK-1480
> URL: https://issues.apache.org/jira/browse/SPARK-1480
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Patrick Wendell
> Assignee: Patrick Wendell
> Priority: Blocker
> Fix For: 1.0.0
>
>
> The Spark codebase is not always consistent on which class loader it uses
> when classlaoders are explicitly passed to things like serializers. This
> caused SPARK-1403 and also causes a bug where when the driver has a modified
> context class loader it is not translated correctly in local mode to the
> (local) executor.
> In most cases what we want is the following behavior:
> 1. If there is a context classloader on the thread, use that.
> 2. Otherwise use the classloader that loaded Spark.
> We should just have a utility function for this and call that function
> whenever we need to get a classloader.
> Note that SPARK-1403 is a workaround for this exact problem (it sets the
> context class loader because downstream code assumes it is set). Once this
> gets fixed in a more general way SPARK-1403 can be reverted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
