steveloughran opened a new pull request, #7044:
URL: https://github.com/apache/hadoop/pull/7044
This is a major change which handles 400 error responses when uploading
large files from memory heap/buffer (or staging committer) and the remote S3
store returns a 500 response from a upload of a block in a multipart upload.
The SDK's own streaming code seems unable to fully replay the upload; at
attempts to but then blocks and the S3 store returns a 400 response
"Your socket connection to the server was not read from or written to
within the timeout period. Idle connections will be closed.
(Service: S3, Status Code: 400...)"
There is an option to control whether or not the S3A client itself attempts
to retry on a 50x error other than 503 throttling events (which are
independently processed as before)
Option: fs.s3a.retry.http.5xx.errors
Default: true
500 errors are very rare from standard AWS S3, which has a five nines SLA.
It may be more common against S3 Express which has lower guarantees.
Third party stores have unknown guarantees, and the exception may indicate a
bad server configuration. Consider setting fs.s3a.retry.http.5xx.errors to
false when working with such stores.
Signification Code changes:
There is now a custom set of implementations of
software.amazon.awssdk.http.ContentStreamProvidercontent in the class
org.apache.hadoop.fs.s3a.impl.UploadContentProviders.
These:
* Restart on failures
* Do not copy buffers/byte buffers into new private byte arrays, so avoid
exacerbating memory problems..
There new IOStatistics for specific http error codes -these are collected
even when all recovery is performed within the SDK.
S3ABlockOutputStream has major changes, including handling of
Thread.interrupt() on the main thread, which now triggers and briefly awaits
cancellation of any ongoing uploads.
If the writing thread is interrupted in close(), it is mapped to an
InterruptedIOException. Applications like Hive and Spark must catch these after
cancelling a worker thread.
Contributed by Steve Loughran
### How was this patch tested?
in progress
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
