[
https://issues.apache.org/jira/browse/SPARK-1035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sandy Ryza resolved SPARK-1035.
-------------------------------
Resolution: Won't Fix
When I originally filed this, I didn't realize that jars could be added at
runtime. In light of this, I don't think we can do much better than the
current state of things.
> Use a single mechanism for distributing jars on Yarn
> ----------------------------------------------------
>
> Key: SPARK-1035
> URL: https://issues.apache.org/jira/browse/SPARK-1035
> Project: Spark
> Issue Type: Bug
> Components: YARN
> Affects Versions: 0.9.0
> Reporter: Sandy Pérez González
>
> When running Spark on Yarn, the app jar is distributed through a different
> mechanism than additional added jars. The app jar gets to every worker node
> as a Yarn local resource. Additional jars only get to the app master, and the
> app master serves them to workers with the HTTP file server. Strangeness
> comes when an application addJar's the app jar, which is a natural thing to
> do in mesos or standalone mode, but in Yarn mode, will try to distribute the
> same jar through a different mechanism. Using the same mechanism for both
> would eliminate this issue, as well as greatly simplify debugging
> ClassNotFoundExceptions in workers.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
