Lakshmi Manasa Gaduputi created SAMZA-2699:
----------------------------------------------
Summary: AzureBlob SystemProducer: remove retries when uploading
block to azure-blob-storage to prevent exceptions being swallowed till samza
retries are over
Key: SAMZA-2699
URL: https://issues.apache.org/jira/browse/SAMZA-2699
Project: Samza
Issue Type: Bug
Reporter: Lakshmi Manasa Gaduputi
Assignee: Lakshmi Manasa Gaduputi
Problem: Current samza code to retry catches exceptions (other than
InterruptedException) and keeps uploading till retry attempts expire. Problem
is when InterruptedException is wrapped by another exception, then
InterruptedException is ignored. see
https://issues.apache.org/jira/browse/SAMZA-2665 which was fixed by bubbling up
InterruptedException immediately but InterruptedException wrapped in another
exception.
Solution: remove retries when uploading block to azure-blob-storage as azure
sdk internally does retry for 4 times as a default.
Samza uses
[BlobServiceClientBuilder|https://github.com/apache/samza/blob/master/samza-azure/src/main/java/org/apache/samza/system/azureblob/AzureBlobClientBuilder.java#L89]
which uses
[RequestRetryOptions|https://github.com/Azure/azure-sdk-for-java/blob/5d602a2d97d36c1b20fc56863313222897c032ed/sdk/storage/azure-storage-blob/src/main/java/com/azure/storage/blob/BlobServiceClientBuilder.java#L72]
as the default retry policy and the [RequestRetryOptions sets max retries to 4
with exponential
backoff|https://github.com/Azure/azure-sdk-for-java/blob/3f31d68eed6fbe11516ca3afe3955c8840a6e974/sdk/storage/azure-storage-common/src/main/java/com/azure/storage/common/policy/RequestRetryOptions.java#L97].
hence it is safe to remove retries with samza.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
