[
https://issues.apache.org/jira/browse/HADOOP-18444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603823#comment-17603823
]
ASF GitHub Bot commented on HADOOP-18444:
-----------------------------------------
xinglin commented on code in PR #4869:
URL: https://github.com/apache/hadoop/pull/4869#discussion_r970195382
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Trash.java:
##########
@@ -94,6 +96,34 @@ public static boolean moveToAppropriateTrash(FileSystem fs,
Path p,
LOG.warn("Failed to get server trash configuration", e);
throw new IOException("Failed to get server trash configuration", e);
}
+
+ /*
+ * In HADOOP-18144, we fixed the logical path vs. target path bug of
getTrashRoot() in ViewFileSystem.
+ * moveToTrash works for ViewFileSystem now. ViewFileSystem will do path
resolution internally by itself.
+ *
+ * When localized trash flag is enabled:
+ * 1). if fs is a ViewFileSystem, we can initialize Trash() with this
ViewFileSystem object;
+ * 2). When fs is not a ViewFileSystem, the only place we would need to
resolve a path is for symbolic links.
+ * However, symlink is not enabled in Hadoop due to the complexity
to support it (HADOOP-10019).
+ */
+ if (conf.getBoolean(CONFIG_VIEWFS_TRASH_FORCE_INSIDE_MOUNT_POINT,
+ CONFIG_VIEWFS_TRASH_FORCE_INSIDE_MOUNT_POINT_DEFAULT)) {
+ // Save the original config in savedValue for localized trash config.
+ String savedValue =
fs.getConf().get(CONFIG_VIEWFS_TRASH_FORCE_INSIDE_MOUNT_POINT);
Review Comment:
Hi Owen,
This is mainly to address a case we encountered during our hive-cli testing.
Here is what we observed.
1. we started a hive-cli. it will initialize a viewfs object, without
localized trash flag set to true.
2. then, we use the "set fs.viewfs.enforce-inside-trash" to set this flag.
3. we do some operations which leads to moveToTrash(fs, path, conf) to be
called.
When that happens, the fs object we gets in moveToTrash does not have the
localized trash flag set. Even when we use FileSystem.get(conf) with the new
conf object inside moveToTrash(), the fs we get doesn't have the localized
trash flag set when _fs.cache_ is enabled: we get the existing fs object from
the cache, which is using the old conf object.
We just want to make sure when the flag is set in Conf, we need to turn it
on for fs.getConf() as well.
> Add Support for localized trash for ViewFileSystem in
> Trash.moveToAppropriateTrash
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-18444
> URL: https://issues.apache.org/jira/browse/HADOOP-18444
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Xing Lin
> Assignee: Xing Lin
> Priority: Major
> Labels: pull-request-available
>
> Trash.moveToAppropriateTrash is used by _hadoop cli -rm_ and hive, to move
> files to trash. However, its current implementation does not support
> localized trash policy we added to ViewFileSystem in HADOOP-18144.
> The reason is in moveToAppropriateTrash, it first resolves a path and then
> uses the resolvedFs, to initialize the trash. As a result, it uses
> getTrashRoot() implementation from targetFs, not ViewFileSystem. The new
> localized trash policy we implemented in ViewFileSystem is not invoked.
> With the new localized trash policy for ViewFileSystem, the trash root would
> be local to a mount point, thus, for ViewFileSystem with this flag turned on,
> there is no need to resolve the path in moveToAppropriateTrash. Rename in
> ViewFileSystem can resolve the logical paths correctly and be able to move a
> file to trash within a mount point.
> Code section of current moveToAppropriateTrash implementation.
> {code:java}
> public static boolean moveToAppropriateTrash(FileSystem fs, Path p,
> Configuration conf) throws IOException {
> Path fullyResolvedPath = fs.resolvePath(p);
> FileSystem fullyResolvedFs =
> FileSystem.get(fullyResolvedPath.toUri(), conf);
> ...
> Trash trash = new Trash(fullyResolvedFs, conf);
> return trash.moveToTrash(fullyResolvedPath);
> }{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
