[
https://issues.apache.org/jira/browse/HDFS-15750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17288350#comment-17288350
]
zhengchenyu commented on HDFS-15750:
------------------------------------
In our test cluster, I modify our code, we make RequireResponse in location
level. I set location level's RequireResponse by DestinationOrder.
I think for some strict DestinationOrder which requireResponse is set to false.
If only one namservice operation failed, the operation will throw exception.
Then we cant make sure consistent between mulit cluster.
I submit first version path. Please give me some sugesstion.
> RBF: Make sure the multi destination are consistent after write operation
> --------------------------------------------------------------------------
>
> Key: HDFS-15750
> URL: https://issues.apache.org/jira/browse/HDFS-15750
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: zhengchenyu
> Assignee: zhengchenyu
> Priority: Major
> Attachments: HDFS-15750.001.patch
>
>
> Nowdays, RBF can't make sure the multi destination are consistent.
> Case 1: RBF can't remove multi destination's file.
> If /user/userA is mountable which mounts two nameservice: ns1, ns2. But if
> both hdfs://ns1/user/userA/a.log and hdfs://ns2/user/userA/a.log exists. I
> want to remove hdfs://ns-fed/user/userA/a.log (Note: a.log is file) to trash,
> then only one nameservice take effect. I think it means inconsistence.
> Through HDFS-14343 already solve the problem in some level, but not
> completed.
> Case 2: RBF regard the operation success, through only one of multi
> destination operations success.
> In other way, if we wanna delete hdfs://ns-fed/user/userA/dirA (Note: dirA is
> directroy.) If hdfs://ns1/user/userA/dirA's permission is not same with
> hdfs://ns2/user/userA/dirA's permission, or one of namenode is down.If one
> destination's result be success, the rbf regard the operation success
> (invokeConcurrent and invokeAll's logical). We maybe can't rename all
> location. I think it also means inconsistence.
> I think we need stricter check. If one operation (which shoud success) failed
> , we should throw exception.
> Note: In fact,If we only use hdfs://ns-fed, I think there is no problem.But
> when migration data from one ns1 to ns2 (a mountable mouts ns1 and ns2) , and
> rewrite some hive table's old partition (which mount multi destination), this
> problem would occure!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
