Steve Loughran created HADOOP-19804:
---------------------------------------
Summary: Switch to adaptive AWS SDK retry policy
Key: HADOOP-19804
URL: https://issues.apache.org/jira/browse/HADOOP-19804
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.5.0, 3.4.3
Reporter: Steve Loughran
The AWS SDK includes an adaptive retry policy which throttles back on all HTTP
requests when 503 responses come in, to stop issuing so many rejected requests
when a store is being overloaded.
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/retry-strategy.html
The s3a connector meets the single-service-endpoint requirement of this poiicy.
"The adaptive retry strategy assumes that the client works against a single
resource (for example, one DynamoDB table or one Amazon S3 bucket)."
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
