[
https://issues.apache.org/jira/browse/HDFS-14117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818190#comment-16818190
]
Íñigo Goiri commented on HDFS-14117:
------------------------------------
The failed unit tests is caused by not doing the check for the file locations.
The issue is that now we don't know where the file really is (we just now it's
in the federation).
Then we know that the file might be in ns0 or ns1 and we know the destination
is in ns1.
When we trigger the rename, we only trigger it against ns1.
Then we get a FileNotFoundException from ns1 when it should be a "not eligible
location".
The way to handle this would be to infer that if we found the file first and
now we get a FileNotFoundException, it must be because the src/dst is not
allowed.
With the previous approach we knew this failure case in just the
{{getFileInfo}} while now we have to do a {{getFileInfo}} and a {{rename2}} to
realize.
Here there is a trade off between the amount of calls in the success and
failure code paths and readability.
To summarize:
* First approach ([^HDFS-14117-HDFS-13891.017.patch]): read everywhere and
rename in the right place.
** Positive case: getFileInfo in all subclusters, rename2 in the right
subcluster.
** Negative case: getFileInfo in all subclusters. No rename2 needed.
* Second approach ([^HDFS-14117-HDFS-13891.018.patch]); read in order and
rename everywhere.
** Positive case: getFileInfo in the right subcluster, rename2 in the right
subcluster.
** Negative case: getFileInfo in the right subcluster, rename2 in all
subclusters (+ a non-intuitive code path that catches a FileNotFoundException
and infers a NotEligibleRenameException).
Keep in mind rename2 uses the writeLock while getFileInfo only readLock.
I'm fine with either one.
[~ayushtkn] which one you prefer?
> RBF: We can only delete the files or dirs of one subcluster in a cluster with
> multiple subclusters when trash is enabled
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-14117
> URL: https://issues.apache.org/jira/browse/HDFS-14117
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: venkata ramkumar
> Assignee: venkata ramkumar
> Priority: Major
> Labels: RBF
> Attachments: HDFS-14117-HDFS-13891.001.patch,
> HDFS-14117-HDFS-13891.002.patch, HDFS-14117-HDFS-13891.003.patch,
> HDFS-14117-HDFS-13891.004.patch, HDFS-14117-HDFS-13891.005.patch,
> HDFS-14117-HDFS-13891.006.patch, HDFS-14117-HDFS-13891.007.patch,
> HDFS-14117-HDFS-13891.008.patch, HDFS-14117-HDFS-13891.009.patch,
> HDFS-14117-HDFS-13891.010.patch, HDFS-14117-HDFS-13891.011.patch,
> HDFS-14117-HDFS-13891.012.patch, HDFS-14117-HDFS-13891.013.patch,
> HDFS-14117-HDFS-13891.014.patch, HDFS-14117-HDFS-13891.015.patch,
> HDFS-14117-HDFS-13891.016.patch, HDFS-14117-HDFS-13891.017.patch,
> HDFS-14117-HDFS-13891.018.patch, HDFS-14117.001.patch, HDFS-14117.002.patch,
> HDFS-14117.003.patch, HDFS-14117.004.patch, HDFS-14117.005.patch
>
>
> When we delete files or dirs in hdfs, it will move the deleted files or dirs
> to trash by default.
> But in the global path we can only mount one trash dir /user. So we mount
> trash dir /user of the subcluster ns1 to the global path /user. Then we can
> delete files or dirs of ns1, but when we delete the files or dirs of another
> subcluser, such as hacluster, it will be failed.
> h1. Mount Table
> ||Global path||Target nameservice||Target path||Order||Read
> only||Owner||Group||Permission||Quota/Usage||Date Modified||Date Created||
> |/test|hacluster2|/test| | |securedn|users|rwxr-xr-x|[NsQuota: -/-, SsQuota:
> -/-]|2018/11/29 14:37:42|2018/11/29 14:37:42|
> |/tmp|hacluster1|/tmp| | |securedn|users|rwxr-xr-x|[NsQuota: -/-, SsQuota:
> -/-]|2018/11/29 14:37:05|2018/11/29 14:37:05|
> |/user|hacluster2,hacluster1|/user|HASH| |securedn|users|rwxr-xr-x|[NsQuota:
> -/-, SsQuota: -/-]|2018/11/29 14:42:37|2018/11/29 14:38:20|
> commands:
> {noformat}
> 1./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -ls /test/.
> 18/11/30 11:00:47 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> -rw-r--r-- 3 securedn supergroup 8081 2018-11-30 10:56 /test/hdfs.cmd
> 2./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -ls /tmp/.
> 18/11/30 11:00:40 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Found 1 items
> -rw-r--r-- 3 securedn supergroup 6311 2018-11-30 10:57 /tmp/mapred.cmd
> 3../opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -rm
> /tmp/mapred.cmd
> 18/11/30 11:01:02 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> rm: Failed to move to trash: hdfs://router/tmp/mapred.cmd: rename destination
> parent /user/securedn/.Trash/Current/tmp/mapred.cmd not found.
> 4./opt/HAcluater_ram1/install/hadoop/router/bin> ./hdfs dfs -rm /test/hdfs.cmd
> 18/11/30 11:01:20 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 18/11/30 11:01:22 INFO fs.TrashPolicyDefault: Moved:
> 'hdfs://router/test/hdfs.cmd' to trash at:
> hdfs://router/user/securedn/.Trash/Current/test/hdfs.cmd
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
