[
https://issues.apache.org/jira/browse/HDFS-6165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13982001#comment-13982001
]
Yongjun Zhang commented on HDFS-6165:
-------------------------------------
I need to amend the rmr section in my last update.
Command: "hdfs dfs -rm -r <targetDir>"
Expected result: recursively delete all components under <targetDir> and
<targetDir> itself.
>From POSIX quote from our earlier discussion, for any component to be deleted,
if the component is a file or empty-dir, permissions required are:
- Search permission of all components of the path prefix,
- WRITE permission of the parent directory of the directory to be removed.
if the component to delete is a non-empty-dir, recursively apply the above
permissions to its child components.
The PROBLEM we are solving here is, that currently FsActions.ALL is passed to
subAccess parameter:
{code}
void checkPermission(String path, INodeDirectory root, boolean doCheckOwner,
FsAction ancestorAccess, FsAction parentAccess, FsAction access,
FsAction subAccess, boolean resolveLink)
{code}
which disallows removal of file and empty-dir component under the <targetDir>
(got exception asking for ALL permission), if the component doesn't have ALL
permissions..
Given an example how to solve this, "hdfs dfs -rm -r <targetDir>", assume the
<targetDir> contains:
{code}
a
a/emptyDir
a/nonEmptyDir
a/nonEmptyDir/filex
{code}
to delete <targetDir>, we need the following permissions (ALL means RWX):
{code}
Prefix components of <targetDir>: search permission (READ and EXECUTE)
Parent componet of <targetDir>: ALL
<targetDir>: ALL
a: ALL
a/emptyDir: NONE
a/nonEmptyDir: ALL
a/nonEmptyDir/filex: NONE
{code}
So what we can do is to modify checkPermission. Since checkPermission used by
multiple operations, the safe change for delete operation is to
1. add a parameter leafAccess:
{code}
void checkPermission(String path, INodeDirectory root, boolean doCheckOwner,
FsAction ancestorAccess, FsAction parentAccess, FsAction access,
FsAction subAccess, FsAction leafAccess, boolean resolveLink)
{code}
Notice that I changed "emptyDirSubAccess" of my earlier version to "leafAccess"
to cover both file and dir which are leaf (To be a leaf, the component has to
be either a file or an empty dir). And we pass null to "leafAccess" parameter
in deleteInternal function.
2. modify checkPermission to apply subAccess on non-leaf component, and
leafAccess to leaf component.
This change is only applicable to "delete" operation (by deleteInternal
method), since checkPermission function is shared across the board, need to
examine how to make this change not to impact other operations.
Thanks.
> "hdfs dfs -rm -r" and "hdfs -rmdir" commands can't remove empty directory
> --------------------------------------------------------------------------
>
> Key: HDFS-6165
> URL: https://issues.apache.org/jira/browse/HDFS-6165
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 2.3.0
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Priority: Minor
> Attachments: HDFS-6165.001.patch, HDFS-6165.002.patch,
> HDFS-6165.003.patch, HDFS-6165.004.patch, HDFS-6165.004.patch
>
>
> Given a directory owned by user A with WRITE permission containing an empty
> directory owned by user B, it is not possible to delete user B's empty
> directory with either "hdfs dfs -rm -r" or "hdfs dfs -rmdir". Because the
> current implementation requires FULL permission of the empty directory, and
> throws exception.
> On the other hand, on linux, "rm -r" and "rmdir" command can remove empty
> directory as long as the parent directory has WRITE permission (and prefix
> component of the path have EXECUTE permission), For the tested OSes, some
> prompt user asking for confirmation, some don't.
> Here's a reproduction:
> {code}
> [root@vm01 ~]# hdfs dfs -ls /user/
> Found 4 items
> drwxr-xr-x - userabc users 0 2013-05-03 01:55 /user/userabc
> drwxr-xr-x - hdfs supergroup 0 2013-05-03 00:28 /user/hdfs
> drwxrwxrwx - mapred hadoop 0 2013-05-03 00:13 /user/history
> drwxr-xr-x - hdfs supergroup 0 2013-04-14 16:46 /user/hive
> [root@vm01 ~]# hdfs dfs -ls /user/userabc
> Found 8 items
> drwx------ - userabc users 0 2013-05-02 17:00 /user/userabc/.Trash
> drwxr-xr-x - userabc users 0 2013-05-03 01:34 /user/userabc/.cm
> drwx------ - userabc users 0 2013-05-03 01:06
> /user/userabc/.staging
> drwxr-xr-x - userabc users 0 2013-04-14 18:31 /user/userabc/apps
> drwxr-xr-x - userabc users 0 2013-04-30 18:05 /user/userabc/ds
> drwxr-xr-x - hdfs users 0 2013-05-03 01:54 /user/userabc/foo
> drwxr-xr-x - userabc users 0 2013-04-30 16:18
> /user/userabc/maven_source
> drwxr-xr-x - hdfs users 0 2013-05-03 01:40
> /user/userabc/test-restore
> [root@vm01 ~]# hdfs dfs -ls /user/userabc/foo/
> [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -r -skipTrash /user/userabc/foo
> rm: Permission denied: user=userabc, access=ALL,
> inode="/user/userabc/foo":hdfs:users:drwxr-xr-x
> {code}
> The super user can delete the directory.
> {code}
> [root@vm01 ~]# sudo -u hdfs hdfs dfs -rm -r -skipTrash /user/userabc/foo
> Deleted /user/userabc/foo
> {code}
> The same is not true for files, however. They have the correct behavior.
> {code}
> [root@vm01 ~]# sudo -u hdfs hdfs dfs -touchz /user/userabc/foo-file
> [root@vm01 ~]# hdfs dfs -ls /user/userabc/
> Found 8 items
> drwx------ - userabc users 0 2013-05-02 17:00 /user/userabc/.Trash
> drwxr-xr-x - userabc users 0 2013-05-03 01:34 /user/userabc/.cm
> drwx------ - userabc users 0 2013-05-03 01:06
> /user/userabc/.staging
> drwxr-xr-x - userabc users 0 2013-04-14 18:31 /user/userabc/apps
> drwxr-xr-x - userabc users 0 2013-04-30 18:05 /user/userabc/ds
> -rw-r--r-- 1 hdfs users 0 2013-05-03 02:11
> /user/userabc/foo-file
> drwxr-xr-x - userabc users 0 2013-04-30 16:18
> /user/userabc/maven_source
> drwxr-xr-x - hdfs users 0 2013-05-03 01:40
> /user/userabc/test-restore
> [root@vm01 ~]# sudo -u userabc hdfs dfs -rm -skipTrash /user/userabc/foo-file
> Deleted /user/userabc/foo-file
> {code}
> Using "hdfs dfs -rmdir" command:
> {code}
> bash-4.1$ hadoop fs -lsr /
> lsr: DEPRECATED: Please use 'ls -R' instead.
> drwxr-xr-x - hdfs supergroup 0 2014-03-25 16:29 /user
> drwxr-xr-x - hdfs supergroup 0 2014-03-25 16:28 /user/hdfs
> drwxr-xr-x - usrabc users 0 2014-03-28 23:39 /user/usrabc
> drwxr-xr-x - abc abc 0 2014-03-28 23:39
> /user/usrabc/foo-empty1
> [root@vm01 usrabc]# su usrabc
> [usrabc@vm01 ~]$ hdfs dfs -rmdir /user/usrabc/foo-empty1
> rmdir: Permission denied: user=usrabc, access=ALL,
> inode="/user/usrabc/foo-empty1":abc:abc:drwxr-xr-x
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
