[
https://issues.apache.org/jira/browse/YARN-5765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603320#comment-15603320
]
Haibo Chen commented on YARN-5765:
----------------------------------
I believe this is broken by YARN-5287.
"chmod clears the set-group-ID bit of a regular file if the file's group ID
does not match the user's effective group ID or one of the user's supplementary
group IDs, unless the user has appropriate privileges. " According to linux man
page. This is inline with the reproduction setup I had.
Walking through the container-executor.c code, {nm_root}/usercache/{userName}
is created with correct permission with the group owner being that of the nm
process and Setgid set. However, in create_validate_dir(), "mkdir(npath, perm)
!= 0" returns false on directory {nm_root}/usercache/{userName}/appcache, so
chmod(npath, perm) is executed on the directory, clearing the Setgid Bits.
Consequentially, all directories/files created under the appcache directory
have the wrong group owner.
The container working directory is also created with the same code, therefore,
having wrong group owner as well.
> LinuxContainerExecutor creates appcache/{appId} with wrong group owner.
> -----------------------------------------------------------------------
>
> Key: YARN-5765
> URL: https://issues.apache.org/jira/browse/YARN-5765
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 3.0.0-alpha1
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Blocker
>
> LinuxContainerExecutor creates usercache/\{userId\}/appcache/\{appId\} with
> wrong group owner, causing Log aggregation and ShuffleHandler to fail because
> node manager process does not have permission to read the files under the
> directory.
> This can be easily reproduced by enabling LCE and submitting a MR example
> job.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
