[
https://issues.apache.org/jira/browse/SPARK-6890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493447#comment-14493447
]
Marcelo Vanzin commented on SPARK-6890:
---------------------------------------
Also, another possible way to fix this is to pass the location of the assembly
jar to the {{Main}} class, instead of the current code. That was my original
suggestion when Nishkam was working on this. It makes the code a little uglier
(to allow for plumbing that path through the code), but it would allow
maintaining the behavior added by that patch while probably fixing this issue.
Let me know if you're working on this, Andrew, otherwise I can do that.
> Local cluster mode is broken
> ----------------------------
>
> Key: SPARK-6890
> URL: https://issues.apache.org/jira/browse/SPARK-6890
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.4.0
> Reporter: Davies Liu
> Assignee: Andrew Or
> Priority: Critical
>
> In master, local cluster mode is broken. If I run `bin/spark-submit --master
> local-cluster[2,1,512]`, my executors keep failing with class not found
> exception. It appears that the assembly jar is not added to the executors'
> class paths. I suspect that this is caused by
> https://github.com/apache/spark/pull/5085.
> {code}
> Exception in thread "main" java.lang.NoClassDefFoundError: scala/Option
> at java.lang.Class.getDeclaredMethods0(Native Method)
> at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
> at java.lang.Class.getMethod0(Class.java:2774)
> at java.lang.Class.getMethod(Class.java:1663)
> at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
> Caused by: java.lang.ClassNotFoundException: scala.Option
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
