GitHub user growse opened a pull request:
https://github.com/apache/spark/pull/4509
[SPARK-5655] Don't chmod700 application files if running in YARN
[Was previously PR4507]
As per SPARK-5655, recently committed code chmod 700s all application files
created on the local fs by a spark executor. This is both unnecessary and
broken on YARN, where files created in the nodemanager's working directory are
already owned by the user running the job and the 'yarn' group. Group read
permission is also needed for the auxiliary shuffle service to be able to read
the files, as this is running as the 'yarn' user.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/growse/spark master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4509.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4509
----
commit f57ce6b45d4d668d339f37592126a8d4d5624635
Author: Andrew Rowson <[email protected]>
Date: 2015-02-10T15:55:26Z
[SPARK-5655] Don't chmod700 application files if running in a YARN container
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
