[
https://issues.apache.org/jira/browse/HDFS-6326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-6326:
--------------------------------
Attachment: HDFS-6326.1.patch
Here is a recap of how we got here.
In the HDFS-4685 design document, we considered the possibility of changing
{{FileStatus}}, but we rejected it for compatibility reasons. {{FileStatus}}
is a {{Writable}}, and we didn't see a way to change it without impacting
forward and backward compatibility of serialization.
Instead, we chose to reserve a previously unused bit of {{FsPermission}} as an
ACL indicator bit. This was implemented in HADOOP-10220.
Later, there was additional debate about whether or not the ACL bit was a good
idea, and eventually it was backed out in HDFS-5923. The rationale was that
the ACL bit created a risk of inconsistency bugs inside the NameNode metadata.
(A code bug could cause us to set the {{AclFeature}} on the inode, but not
activate the ACL bit.) Removing the ACL bit eliminated that entire class of
possible bugs.
Related to this, HDFS-5932 added the alternate way for ls to identify if the
file had an ACL by trying {{getAclStatus}}, with error handling to detect the
missing method on the server side for compatibility or method simply not
implemented in the {{FileSystem}} subclass. Unfortunately, we didn't
anticipate that different file system implementations would have other failure
behaviors, such as what is reported here for WebHDFS.
It seems we need a compromise that allows for some kind of ACL indicator and
avoids brittle error response parsing. We're seeing the issue with WebHDFS
right now, but other custom file systems could have unexpected trouble too.
How about if we go ahead and add the ACL bit back in, but we don't use it as a
piece of persistent data inside the NameNode? We'll just toggle the flag on as
needed in outbound {{FileStatus}} responses. That still ought to avoid the
potential consistency problems. [~wheat9] and [~daryn], what do you think?
I'm attaching a patch that implements that approach. This was mostly just a
matter of resurrecting code that I previously wrote on the HDFS-4685 branch and
then discarded. IOW, this is mostly code that already passed review once. The
logic in {{FSDirectory}} is different this time though.
> WebHdfs ACL compatibility is broken
> -----------------------------------
>
> Key: HDFS-6326
> URL: https://issues.apache.org/jira/browse/HDFS-6326
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 3.0.0, 2.4.0
> Reporter: Daryn Sharp
> Priority: Blocker
> Attachments: HDFS-6326.1.patch
>
>
> 2.4 ACL support is completely incompatible with <2.4 webhdfs servers. The NN
> throws an {{IllegalArgumentException}} exception.
> {code}
> hadoop fs -ls webhdfs://nn/
> Found 21 items
> ls: Invalid value for webhdfs parameter "op": No enum constant
> org.apache.hadoop.hdfs.web.resources.GetOpParam.Op.GETACLSTATUS
> [... 20 more times...]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
