[
https://issues.apache.org/jira/browse/HDFS-15607?focusedWorklogId=493107&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-493107
]
ASF GitHub Bot logged work on HDFS-15607:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Sep/20 18:36
Start Date: 30/Sep/20 18:36
Worklog Time Spent: 10m
Work Description: smengcl commented on a change in pull request #2352:
URL: https://github.com/apache/hadoop/pull/2352#discussion_r497720649
##########
File path:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
##########
@@ -2094,6 +2103,41 @@ public Void next(final FileSystem fs, final Path p)
}.resolve(this, absF);
}
+ /**
+ * Helper function to check if a trash root exists in the given directory,
+ * remove the trash root if it is empty, or throw IOException if not empty
+ * @param p Path to a directory.
+ */
+ private void checkTrashRootAndRemoveIfEmpty(final Path p) throws IOException
{
+ Path trashRoot = new Path(p, FileSystem.TRASH_PREFIX);
+ try {
+ // listStatus has 4 possible outcomes here:
+ // 1) throws FileNotFoundException: the trash root doesn't exist.
+ // 2) returns empty array: the trash path is an empty directory.
+ // 3) returns non-empty array, len >= 2: the trash root is not empty.
+ // 4) returns non-empty array, len == 1:
+ // i) if the element's path is exactly p, the trash path is not a dir.
+ // e.g. a file named .Trash. Ignore.
+ // ii) if the element's path isn't p, the trash root is not empty.
+ FileStatus[] fileStatuses = listStatus(trashRoot);
+ if (fileStatuses.length == 0) {
+ DFSClient.LOG.debug("Removing empty trash root {}", trashRoot);
+ delete(trashRoot, false);
+ } else {
+ if (fileStatuses.length == 1
+ && !fileStatuses[0].isDirectory()
+ && !fileStatuses[0].getPath().equals(p)) {
+ // Ignore the trash path because it is not a directory.
+ DFSClient.LOG.warn("{} is not a directory.", trashRoot);
Review comment:
I get your point. But I don't think it is worth it to prevent the user
from doing so. At best, we can throw some client-side warnings when the user is
doing so.
There are so many ways to circumvent this that I can think of so far if the
user really wants to: the user could create the `.Trash` file before allowing
snapshot, rename `.Trash` file from another place.
Even if we have placed restrictions on a newer version of HDFS NameNode,
they might have already created the `.Trash` before the NN upgrade.
Also, regular user trash also faces the same issue.
```
$ hdfs dfs -touch hdfs://127.0.0.1:9999/user/smeng/.Trash
$ hdfs dfs -touch hdfs://127.0.0.1:9999/file3
$ hdfs dfs -rm hdfs://127.0.0.1:9999/file3
2020-09-30 11:27:43,062 WARN fs.TrashPolicyDefault: Can't create trash
directory: hdfs://127.0.0.1:9999/user/smeng/.Trash/Current
org.apache.hadoop.fs.ParentNotDirectoryException: /user/smeng/.Trash (is not
a directory)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkIsDirectory(FSPermissionChecker.java:743)
...
rm: Failed to move to trash: hdfs://127.0.0.1:9999/file3: /user/smeng/.Trash
(is not a directory)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 493107)
Time Spent: 40m (was: 0.5h)
> Create trash dir when allowing snapshottable dir
> ------------------------------------------------
>
> Key: HDFS-15607
> URL: https://issues.apache.org/jira/browse/HDFS-15607
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Affects Versions: 3.4.0
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In {{TrashPolicyDefault}}, the {{.Trash}} directory will be created with
> permission 700 (and without sticky bit) by the first user that moves a file
> to the trash. This is an issue when other users try to move files to that
> trash because they may not have the permission to move to that trash if the
> trash root is shared. -- in this case, snapshottable directories.
> This only affects users when trash is enabled inside snapshottable
> directories ({{dfs.namenode.snapshot.trashroot.enabled}} set to true), and
> when a user performing move to trash operations doesn't have admin
> permissions.
> Solution: Create a {{.Trash}} directory with 777 permission and sticky bits
> enabled (similar solution as HDFS-10324).
> Also need to deal with some corner cases:
> 1. even when the snapshottable directory trash root config is not enabled
> ({{dfs.namenode.snapshot.trashroot.enabled}} set to false), create the
> {{.Trash}} directory anyway? Or should we ask the admin to provision trash
> manually after enabling {{dfs.namenode.snapshot.trashroot.enabled}} on an
> existing cluster?
> - If the cluster is just upgraded, we need to provision trash manually anyway.
> 2. When immediately disallowing trash, it shouldn't fail. just remove the
> .Trash directory when disallowing snapshot on a dir if it is empty?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
