[
https://issues.apache.org/jira/browse/HADOOP-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18023590#comment-18023590
]
ASF GitHub Bot commented on HADOOP-19027:
-----------------------------------------
steveloughran opened a new pull request, #8003:
URL: https://github.com/apache/hadoop/pull/8003
This is a fraction of #6425 backported to the V1 SDK, which does a lot of
V2-specific translation/unwinding.
416 responses are mapped to RangeNotSatisfiableEOFException, whose retry
policy is: fail.
Classic EOFException is now retried as connection failure. calls to read()
and lazyseek all retry on this with full retry policy, including handling of
socket errors.
Everything related to new sdk, its complex http channel stuff: omitted.
This means that the following will be handled
* EOF on socket read
* connection reset on socket read
Out of range GET is mappped to RangeNotSatisfiableEOFException so it is not
retried,
this is critical. Without it ITestS3AOpenCost.testReadPastEOF() will time
out as it is continually retrying.
### How was this patch tested?
Cherrypick of new ITestS3AOpenCost.
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [X] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> S3A: S3AInputStream doesn't recover from HTTP/channel exceptions
> ----------------------------------------------------------------
>
> Key: HADOOP-19027
> URL: https://issues.apache.org/jira/browse/HADOOP-19027
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.5.0, 3.4.1
>
>
> S3AInputStream doesn't seem to recover from Http exceptions raised through
> HttpClient or through OpenSSL.
> * review the recovery code to make sure it is retrying enough, it looks
> suspiciously like it doesn't
> * detect the relevant openssl, shaded httpclient and unshaded httpclient
> exceptions, map to a standard one and treat as comms error in our retry policy
> This is not the same as the load balancer/proxy returning 443/444 which we
> map to AWSNoResponseException. We can't reuse that as it expects to be
> created from an
> {{software.amazon.awssdk.awscore.exception.AwsServiceException}} exception
> with the relevant fields...changing it could potentially be incompatible.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
