[
https://issues.apache.org/jira/browse/YARN-742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-742:
----------------------------
Attachment: YARN-742.patch
Patch to walk back up the app log dir path to check for the existence of a
directory before blindly proceeding to create it. It gets out early if it
finds a path that exists.
In addition to the unit test, I manually tested this on a single-node cluster
verifying the directories are created iff they are missing and permissions are
set iff necessary due to a too-restrictive umask.
> Log aggregation causes a lot of redundant setPermission calls
> -------------------------------------------------------------
>
> Key: YARN-742
> URL: https://issues.apache.org/jira/browse/YARN-742
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 0.23.7, 2.0.4-alpha
> Reporter: Kihwal Lee
> Assignee: Jason Lowe
> Attachments: YARN-742.patch
>
>
> In one of our clusters, namenode RPC is spending 45% of its time on serving
> setPermission calls. Further investigation has revealed that most calls are
> redundantly made on /mapred/logs/<user>/logs. Also mkdirs calls are made
> before this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
