[
https://issues.apache.org/jira/browse/HADOOP-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-19485:
------------------------------------
Release Note:
AWS SDK 2.30.0 and later are (currently) incompatible with third party stores.
Accordingly, this release is kept at a 2.29 version. See HADOOP-19490. There
may now be some problems using the AWS4SignerType and S3.
Description:
This SDK update doesn't need to come with some changes to disable some new
features (default integrity protections),
and to apply critical changes related to the SDK
Default integrity protection came with 2.30, and is on unless disabled.
https://github.com/aws/aws-sdk-java-v2/issues/5801
As well as being incompatible with third party stores, it has also affected S3
multiregion Access Points: https://github.com/aws/aws-sdk-java-v2/issues/5878
This has broken most interaction with third party stores, hence fixes in
Iceberg https://github.com/apache/iceberg/pull/12264 and Trinio
https://github.com/trinodb/trino/pull/24954
There's also [AWS v2.30 SDK InputStream behavior changes #5859](AWS v2.30 SDK
InputStream behavior changes).
It looks like our code is safer from that, but it did require code review.
SDK 2.30.19 seems good with this
was:
This SDK update doesn't need to come with some changes to disable some new
features (default integrity protections),
and to apply critical changes related to the SDK
Default integrity protection came with 2.30, and is on unless disabled.
https://github.com/aws/aws-sdk-java-v2/issues/5801
As well as being incompatible with third party stores, it has also affected S3
multiregion Access Points: https://github.com/aws/aws-sdk-java-v2/issues/5878
This has broken most interaction with third party stores, hence fixes in
Iceberg https://github.com/apache/iceberg/pull/12264 and Trinio
https://github.com/trinodb/trino/pull/24954
There's also [AWS v2.30 SDK InputStream behavior changes #5859](AWS v2.30 SDK
InputStream behavior changes).
It looks like our code is safer from that, but it did require code review.
SDK 2.30.19 seems good with this
> S3A: Upgrade AWS V2 SDK
> -----------------------
>
> Key: HADOOP-19485
> URL: https://issues.apache.org/jira/browse/HADOOP-19485
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: build, fs/s3
> Affects Versions: 3.4.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> This SDK update doesn't need to come with some changes to disable some new
> features (default integrity protections),
> and to apply critical changes related to the SDK
> Default integrity protection came with 2.30, and is on unless disabled.
> https://github.com/aws/aws-sdk-java-v2/issues/5801
> As well as being incompatible with third party stores, it has also affected
> S3 multiregion Access Points:
> https://github.com/aws/aws-sdk-java-v2/issues/5878
> This has broken most interaction with third party stores, hence fixes in
> Iceberg https://github.com/apache/iceberg/pull/12264 and Trinio
> https://github.com/trinodb/trino/pull/24954
> There's also [AWS v2.30 SDK InputStream behavior changes #5859](AWS v2.30 SDK
> InputStream behavior changes).
> It looks like our code is safer from that, but it did require code review.
> SDK 2.30.19 seems good with this
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
