Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/303#issuecomment-39491279
From my understanding local: is a spark thing that says the file is on each
worker. This is a bit of an odd usage on yarn. Generally Yarn recommends
sending things with the application so that you can easily upgrade or use
new/different versions of applications. I can see it maybe being used for the
spark jar but it doesn't really make sense to me for the application jar or the
logging config which I would expect to be very different per user. If you put
those in the public distributed cache, they will be there on all nodes for most
applications after the first app downloads them.
I am ok with making it work though since seems to be a standard Spark thing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
