[
https://issues.apache.org/jira/browse/HDFS-6962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-6962:
--------------------------------
Target Version/s: 2.7.0 (was: 2.4.1)
Hello, [~Alexandre LINTE]. Thank you for filing this issue. I tested the same
scenario against a Linux local file system, and I confirmed that HDFS is
showing different behavior, just like you described.
I also confirmed that this is a divergence from the POSIX ACL specs. Here is a
quote of the relevant section:
{quote}
The permissions of inherited access ACLs are further modified by the mode
parameter that each system call creating file system objects has. The mode
parameter contains nine permission bits that stand for the permissions of the
owner, group, and other class permissions. The effective permissions of each
class are set to the intersection of the permissions defined for this class in
the ACL and specified in the mode parameter.
If the parent directory has no default ACL, the permissions of the new file are
determined as defined in POSIX.1. The effective permissions are set to the
permissions defined in the mode parameter, minus the permissions set in the
current umask.
The umask has no effect if a default ACL exists.
{quote}
Changing this behavior is going to be somewhat challenging. Note the
distinction made in the spec between mode and umask. When creating a new child
(file or directory) of a directory with a default ACL, the mode influences the
inherited access ACL entries, but the umask has no effect. Unfortunately, our
current implementation intersects mode and umask on the client side before
passing them to the NameNode in the RPC. This happens in {{DFSClient#mkdirs}}
and {{DFSClient#create}}:
{code}
public boolean mkdirs(String src, FsPermission permission,
boolean createParent) throws IOException {
if (permission == null) {
permission = FsPermission.getDefault();
}
FsPermission masked = permission.applyUMask(dfsClientConf.uMask);
{code}
{code}
public DFSOutputStream create(String src,
FsPermission permission,
EnumSet<CreateFlag> flag,
boolean createParent,
short replication,
long blockSize,
Progressable progress,
int buffersize,
ChecksumOpt checksumOpt,
InetSocketAddress[] favoredNodes) throws
IOException {
checkOpen();
if (permission == null) {
permission = FsPermission.getFileDefault();
}
FsPermission masked = permission.applyUMask(dfsClientConf.uMask);
{code}
On the NameNode side, when it copies the default ACL from parent to child,
we've lost the information. We just have a single piece of permissions data,
with no knowledge of what was the mode vs. the umask on the client side.
A potential solution is to push both mode and umask explicitly to the NameNode
in the RPC requests for {{MkdirsRequestProto}} and {{CreateRequestProto}}.
Those messages already contain an instance of {{FsPermissionProto}}. We could
add a second optional instance. If both instances are defined, then the
NameNode would interpret one as being mode and the other as being umask. There
would still be a possibility of an older client still passing just one
instance, and in that case, we'd have to fall back to the current behavior.
It's a bit messy, but it could work.
We also have one additional problem specific to the shell for files (not
directories). The implementation of copyFromLocal breaks down into 2 separate
RPCs: creating the file, followed by a separate chmod call. The NameNode has
no way of knowing if that chmod call is part of a copyFromLocal or not though.
It's too late to enforce the mode vs. umask distinction.
I'm tentatively targeting this to 2.7.0. I think this will need more
investigation to make sure there are no compatibility issues with the solution.
If there is an unavoidable compatibility problem, then it might require
pushing out to 3.x. We won't know for sure until someone starts coding.
Thank you again for the very detailed bug report.
> ACLs inheritance conflict with umaskmode
> ----------------------------------------
>
> Key: HDFS-6962
> URL: https://issues.apache.org/jira/browse/HDFS-6962
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
> Affects Versions: 2.4.1
> Environment: CentOS release 6.5 (Final)
> Reporter: LINTE
> Labels: hadoop, security
>
> In hdfs-site.xml
> <property>
> <name>dfs.umaskmode</name>
> <value>027</value>
> </property>
> 1/ Create a directory as superuser
> bash# hdfs dfs -mkdir /tmp/ACLS
> 2/ set default ACLs on this directory rwx access for group readwrite and user
> toto
> bash# hdfs dfs -setfacl -m default:group:readwrite:rwx /tmp/ACLS
> bash# hdfs dfs -setfacl -m default:user:toto:rwx /tmp/ACLS
> 3/ check ACLs /tmp/ACLS/
> bash# hdfs dfs -getfacl /tmp/ACLS/
> # file: /tmp/ACLS
> # owner: hdfs
> # group: hadoop
> user::rwx
> group::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> user::rwx | group::r-x | other::--- matches with the umaskmode defined in
> hdfs-site.xml, everything ok !
> default:group:readwrite:rwx allow readwrite group with rwx access for
> inhéritance.
> default:user:toto:rwx allow toto user with rwx access for inhéritance.
> default:mask::rwx inhéritance mask is rwx, so no mask
> 4/ Create a subdir to test inheritance of ACL
> bash# hdfs dfs -mkdir /tmp/ACLS/hdfs
> 5/ check ACLs /tmp/ACLS/hdfs
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs
> # file: /tmp/ACLS/hdfs
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx #effective:r-x
> group::r-x
> group:readwrite:rwx #effective:r-x
> mask::r-x
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> Here we can see that the readwrite group has rwx ACL bu only r-x is effective
> because the mask is r-x (mask::r-x) in spite of default mask for inheritance
> is set to default:mask::rwx on /tmp/ACLS/
> 6/ Modifiy hdfs-site.xml et restart namenode
> <property>
> <name>dfs.umaskmode</name>
> <value>010</value>
> </property>
> 7/ Create a subdir to test inheritance of ACL with new parameter umaskmode
> bash# hdfs dfs -mkdir /tmp/ACLS/hdfs2
> 8/ Check ACL on /tmp/ACLS/hdfs2
> bash# hdfs dfs -getfacl /tmp/ACLS/hdfs2
> # file: /tmp/ACLS/hdfs2
> # owner: hdfs
> # group: hadoop
> user::rwx
> user:toto:rwx #effective:rw-
> group::r-x #effective:r--
> group:readwrite:rwx #effective:rw-
> mask::rw-
> other::---
> default:user::rwx
> default:user:toto:rwx
> default:group::r-x
> default:group:readwrite:rwx
> default:mask::rwx
> default:other::---
> So HDFS masks the ACL value (user, group and other -- exepted the POSIX
> owner -- ) with the group mask of dfs.umaskmode properties when creating
> directory with inherited ACL.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
