[
https://issues.apache.org/jira/browse/HDFS-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253343#comment-17253343
]
zhengchenyu edited comment on HDFS-14343 at 12/22/20, 8:52 AM:
---------------------------------------------------------------
[~inigoiri][~ayushtkn].
Hi, I have some question, can you give me some suggestion. In this patch, use
isMultiDestDirectory to check whether rename all location or not. I think the
design of MultipleDestinationMountTableResolver may consider there is no
repeated file among multi nameservice.
If /user/userA is mountable which mounts two nameservice: ns1, ns2. But if both
hdfs://ns1/user/userA/a.log and hdfs://ns2/user/userA/a.log exists. I want to
remove hdfs://ns-fed/user/userA/a.log (Note: a.log is file) to trash, then only
one nameservice take effect. I think it means inconsistence.
(Note: In fact, when migration data from one ns1 to ns2 (a mountable mouts ns1
and ns2) , and rewrite some hive table's old partition, this problem would
occure!)
In other way, if we hdfs://ns-fed/user/userA/dirA (Note: dirA is directroy.) If
hdfs://ns1/user/userA/dirA's permission is not same with
hdfs://ns2/user/userA/dirA's permission. Or one of nameservice is down. We
maybe can't rename all location. I think it also means inconsistence. I think
we need stricter check. If one operation error, we should throw exception.
was (Author: zhengchenyu):
[~inigoiri][~ayushtkn].
Hi, I have some question, can you give me some suggestion. In this patch, use
isMultiDestDirectory to check whether rename all location or not. I think the
design of MultipleDestinationMountTableResolver may consider there is no
repeated file among multi nameservice.
If /user/userA is mountable which mounts two nameservice: ns1, ns2. But if both
hdfs://ns1/user/userA/a.log and hdfs://ns2/user/userA/a.log exists. I want to
remove hdfs://ns-fed/user/userA/a.log (Note: a.log is file) to trash, then only
one nameservice take effect. I think it means inconsistence.
(Note: In fact, when migration data from one nameservice to nameservice, and
rewrite some hive table's old partition, this problem would occure!)
In other way, if we hdfs://ns-fed/user/userA/dirA (Note: dirA is directroy.) If
hdfs://ns1/user/userA/dirA's permission is not same with
hdfs://ns2/user/userA/dirA's permission. Or one of nameservice is down. We
maybe can't rename all location. I think it also means inconsistence. I think
we need stricter check. If one operation error, we should throw exception.
> RBF: Fix renaming folders spread across multiple subclusters
> ------------------------------------------------------------
>
> Key: HDFS-14343
> URL: https://issues.apache.org/jira/browse/HDFS-14343
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Assignee: Ayush Saxena
> Priority: Major
> Fix For: 3.3.0, HDFS-13891
>
> Attachments: HDFS-14343-HDFS-13891-01.patch,
> HDFS-14343-HDFS-13891-02.patch, HDFS-14343-HDFS-13891-03.patch,
> HDFS-14343-HDFS-13891-04.patch, HDFS-14343-HDFS-13891-05.patch
>
>
> The {{RouterClientProtocol#rename()}} function assumes that we are renaming
> files and only renames one of them (i.e., {{invokeSequential()}}). In the
> case of folders which are in all subclusters (e.g., HASH_ALL) we should
> rename all locations (i.e., {{invokeAll()}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
