[
https://issues.apache.org/jira/browse/HADOOP-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18031517#comment-18031517
]
ASF GitHub Bot commented on HADOOP-19654:
-----------------------------------------
steveloughran commented on PR #7882:
URL: https://github.com/apache/hadoop/pull/7882#issuecomment-3428848006
latest iteration works with third party stores without MPU (so no magic or
use of memory for upload buffering), or bulk delete.
tested google gcs, only underful buffers which can be ignored.
```
[ERROR] Failures:
[ERROR]
ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testMultipleUnbuffers:108->AbstractContractUnbufferTest.validateFullFileContents:141->AbstractContractUnbufferTest.validateFileContents:148
failed to read expected number of bytes from stream. This may be transient ==>
expected: <1024> but was: <533>
[ERROR]
ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testUnbufferAfterRead:61->AbstractContractUnbufferTest.validateFullFileContents:141->AbstractContractUnbufferTest.validateFileContents:148
failed to read expected number of bytes from stream. This may be transient ==>
expected: <1024> but was: <533>
[ERROR]
ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testUnbufferBeforeRead:71->AbstractContractUnbufferTest.validateFullFileContents:141->AbstractContractUnbufferTest.validateFileContents:148
failed to read expected number of bytes from stream. This may be transient ==>
expected: <1024> but was: <539>
[ERROR]
ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testUnbufferOnClosedFile:91->AbstractContractUnbufferTest.validateFullFileContents:141->AbstractContractUnbufferTest.validateFileContents:148
failed to read expected number of bytes from stream. This may be transient ==>
expected: <1024> but was: <539>
[INFO]
[ERROR] Tests run: 1253, Failures: 4, Errors: 0, Skipped: 450
[INFO]
```
> Upgrade AWS SDK to 2.33.x
> -------------------------
>
> Key: HADOOP-19654
> URL: https://issues.apache.org/jira/browse/HADOOP-19654
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build, fs/s3
> Affects Versions: 3.5.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> Upgrade to a recent version of 2.33.x or later while off the critical path of
> things.
> HADOOP-19485 froze the sdk at a version which worked with third party stores.
> Apparently the new version works; early tests show that Bulk Delete calls
> with third party stores complain about lack of md5 headers, so some tuning is
> clearly going to be needed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
