[
https://issues.apache.org/jira/browse/HADOOP-12634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15083652#comment-15083652
]
Gaurav Kanade commented on HADOOP-12634:
----------------------------------------
[~cnauroth] - Thanks for the review.
With regards to testing there are two aspects to consider here given the change
we made to the behavior of lazy rename pending operation.
1. Post the change the lazy rename pending will behave in the case of src and
dest both existing the same way it will behave in the case of src only
existing. The new test tries to somewhat cover this case - i.e. if we call a
rename on a file already existing does rename happen correctly? Thus this part
of the change effect is covered.
2. The only other thing left to test is does the changed code path get executed
in lazy rename pending - this as you see and as described is hard to capture in
a single test given the various process crash and client side copy it requires
to simulate. However the code change here is fairly straightforward; i.e. a
simple change in condition; the fact that this condition change behaves
appropriately is tested above.
I am not clear yet as to have a manner of testing 2 cleanly; let me know your
thoughts.
I will also fix the braces issue with the next version of the patch
> Change Lazy Rename Pending Operation Completion of WASB to address case of
> potential data loss due to partial copy
> ------------------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12634
> URL: https://issues.apache.org/jira/browse/HADOOP-12634
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Gaurav Kanade
> Assignee: Gaurav Kanade
> Priority: Critical
> Attachments: HADOOP-12634.01.patch
>
>
> HADOOP-12334 changed mode of Copy Operation of HBase WAL Archiving to bypass
> Azure Storage Throttling after retries. This was via client side copy.
> However a process crash when the copy is partially done would result in a
> scenario where the source and destination blobs will have different contents
> and lazy rename pending operation will not handle this thus causing data
> loss. We need to fix the lazy rename pending operation to address this issue
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
