[
https://issues.apache.org/jira/browse/HADOOP-12334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803320#comment-14803320
]
Gaurav Kanade commented on HADOOP-12334:
----------------------------------------
[~linchan]: Thanks!
[~cnauroth]: Thanks for the review! Re your comments:
1. The reason we are doing this for a page blob and not a block blob: a) We do
not expect the throttling to occur on block blobs b) This is supposed to be a
very rare case code path c) There might be some technical difficulties in doing
the copy process in a simple manner like we did for page blobs - hence leaving
it at page blobs for now
2. I addressed the concerns re throwing of exceptions - see latest patch
3. We have separately tested the individual functionality of the new method of
copy; it would likely not be in the scope of a unit test to actually test with
the injection of SERVER-BUSY error
[~dchickabasapa] Could you please review now?
> Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage
> Throttling after retries
> -----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12334
> URL: https://issues.apache.org/jira/browse/HADOOP-12334
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools
> Reporter: Gaurav Kanade
> Assignee: Gaurav Kanade
> Attachments: HADOOP-12334.01.patch, HADOOP-12334.02.patch,
> HADOOP-12334.03.patch, HADOOP-12334.04.patch, HADOOP-12334.05.patch,
> HADOOP-12334.06.patch
>
>
> HADOOP-11693 mitigated the problem of HMaster aborting regionserver due to
> Azure Storage Throttling event during HBase WAL archival. The way this was
> achieved was by applying an intensive exponential retry when throttling
> occurred.
> As a second level of mitigation we will change the mode of copy operation if
> the operation fails even after all retries -i.e. we will do a client side
> copy of the blob and then copy it back to destination. This operation will
> not be subject to throttling and hence should provide a stronger mitigation.
> However it is more expensive, hence we do it only in the case we fail after
> all retries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
