[
https://issues.apache.org/jira/browse/HADOOP-18759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anuj Modi resolved HADOOP-18759.
--------------------------------
Fix Version/s: 3.4.1
(was: 3.5.0)
Release Note: https://github.com/apache/hadoop/pull/5881
Target Version/s: 3.4.0 (was: 3.3.4)
Resolution: Fixed
[Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for
connection timeout. by anujmodi2021 · Pull Request #5881 · apache/hadoop
(github.com)|https://github.com/apache/hadoop/pull/5881]
> [ABFS][Backoff-Optimization] Have a Static retry policy for connection
> timeout failures
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-18759
> URL: https://issues.apache.org/jira/browse/HADOOP-18759
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.4
> Reporter: Anuj Modi
> Assignee: Anuj Modi
> Priority: Major
> Fix For: 3.4.1
>
>
> Today when a request fails with connection timeout, it falls back into the
> loop for exponential retry. Unlike Azure Storage, there are no guarantees of
> success on exponentially retried request or recommendations for ideal retry
> policies for Azure network or any other general failures. Faster failure and
> retry might be more beneficial for such generic connection timeout failures.
> This PR introduces a new Static Retry Policy which will currently be used
> only for Connection Timeout failures. It means all the requests failing with
> Connection Timeout errors will be retried after a constant retry(sleep)
> interval independent of how many times that request has failed. Max Retry
> Count check will still be in place.
> Following Configurations will be introduced in the change:
> # "fs.azure.static.retry.for.connection.timeout.enabled" - default: true,
> true: static retry will be used for CT, false: Exponential retry will be used.
> # "fs.azure.static.retry.interval" - default: 1000ms.
> This also introduces a new field in x-ms-client-request-id only for the
> requests that are being retried after connection timeout failure. New filed
> will tell what retry policy was used to get the sleep interval before making
> this request.
> Header "x-ms-client-request-id " right now has only the retryCount and
> retryReason this particular API call is. For ex:
> :eb06d8f6-5693-461b-b63c-5858fa7655e6:29cb0d19-2b68-4409-bc35-cb7160b90dd8:::CF:1_CT.
> Moving ahead for retryReason "CT" it will have retry policy abbreviation as
> well.
> For ex:
> :eb06d8f6-5693-461b-b63c-5858fa7655e6:29cb0d19-2b68-4409-bc35-cb7160b90dd8:::CF:1_CT_E.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
