[
https://issues.apache.org/jira/browse/HDFS-15982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17339268#comment-17339268
]
Viraj Jasani commented on HDFS-15982:
-------------------------------------
Perhaps there was some misunderstanding w.r.t FileSystem behaviour that was
mentioned above, my bad.
If you are talking about FileSystem#delete API, there is no change done in
FileSystem as part of this Jira (i.e WebHdfsFileSystem is not updated). As part
of this Jira, we have introduced change in WebHdfs NameNode implementation
(DELETE API endpoint in NamenodeWebHdfsMethods) by introducing an additional
query param: skiptrash (similar to skipTrash param in DELETE shell command).
This Jira is marked incompatible because of the default value of skiptrash
query param, which is false as of now. If we change the default value to true,
this Jira would not need to be marked incompatible. However, if you take a
closer look, DELETE shell command (rm) has default value of skipTrash param as
false, and hence by replicating the same behaviour in WebHdfs NameNode
implementation of DELETE API, we are trying to be compatible with shell command
behaviour (but at the same time incompatible with previous behaviour of
NamenodeWebHdfsMethods DELETE API, which never had moving to trash feature).
Please take a look at Wei-Chiu's comments above:
{quote}This is a big incompatible change. If we think this should be part of
3.4.0, risking our compatibility guarantee (which I think makes sense, given
how many times I was involved in accidental data deletion), I think it can be
part of 3.3.1. We traditionally regard 3.3.0 as non-production ready, so making
an incompat change in 3.3.1 probably is justifiable.
{quote}
I am of the similar opinion and hence I tried to keep default value of
skiptrash query param as false, this was the consensus so far. However, if the
consensus changes to make this Jira a compatible change (by changing default
value of skiptrash as true), I will be happy to provide addendum patch for the
same.
{quote}It is never like for improvements we just mark incompatible and then
side-line the compatibility because someone feels it would be good. For Bugs,
yes we do so if it is the last option, for which he have a flag telling sorry
we had no choice.
{quote}
I agree. However, if we look at this feature from a different viewpoint, we
might feel this was a bug in NamenodeWebHdfsMethods implementation, which did
not allow moving files to trash similar to how Delete shell command (rm) always
did. But as I mentioned, if consensus changes to make this Jira compatible, I
can provide addendum to change default value of skiptrash as true.
{quote}Regarding the router part, Is the call going back to the router, since
the path needs to be resolved with respect to the mount table, or is it
resolving with respect to the Namenode itself?
{quote}
NamenodeWebHdfsMethods DELETE implementation, before using FileSystem#delete,
uses Trash#moveToAppropriateTrash utility which internally takes care of
resolving symlinks, mount points etc using FileSystem#rename.
As per the doc:
{code:java}
/**
* In case of the symlinks or mount points, one has to move the appropriate
* trashbin in the actual volume of the path p being deleted.
*
* Hence we get the file system of the fully-qualified resolved-path and
* then move the path p to the trashbin in that volume,
* @param fs - the filesystem of path p
* @param p - the path being deleted - to be moved to trasg
* @param conf - configuration
* @return false if the item is already in the trash or trash is disabled
* @throws IOException on error
*/
public static boolean moveToAppropriateTrash(FileSystem fs, Path p,
Configuration conf) throws IOException {
Path fullyResolvedPath = fs.resolvePath(p);
.........{code}
Unless I am mistaken, fs.resolvePath(p) should resolve path through
DistributedFileSystem -> DFSClient -> NameNode.
> Deleted data using HTTP API should be saved to the trash
> --------------------------------------------------------
>
> Key: HDFS-15982
> URL: https://issues.apache.org/jira/browse/HDFS-15982
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs, hdfs-client, httpfs, webhdfs
> Reporter: Bhavik Patel
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Attachments: Screenshot 2021-04-23 at 4.19.42 PM.png, Screenshot
> 2021-04-23 at 4.36.57 PM.png
>
> Time Spent: 12h 10m
> Remaining Estimate: 0h
>
> If we delete the data from the Web UI then it should be first moved to
> configured/default Trash directory and after the trash interval time, it
> should be removed. currently, data directly removed from the system[This
> behavior should be the same as CLI cmd]
> This can be helpful when the user accidentally deletes data from the Web UI.
> Similarly we should provide "Skip Trash" option in HTTP API as well which
> should be accessible through Web UI.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
