[
https://issues.apache.org/jira/browse/HADOOP-18073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17881122#comment-17881122
]
Ahnaf Khan edited comment on HADOOP-18073 at 9/11/24 8:51 PM:
--------------------------------------------------------------
[[email protected]] which version specifically did it change for? For context,
we are trying to upgrade from hadoop 3.3.3 to hadoop 3.3.6 and it looks like
the defaults3clientfactory code still seems to be pulling from aws java sdk v1
instead of v2
([https://javadoc.io/doc/org.apache.hadoop/hadoop-aws/3.3.6/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.html]).
I guess in other words, is Hadoop 3.3.6 compatible with v1 only or v2 only?
was (Author: JIRAUSER306988):
[[email protected]] which version specifically did it change for? For context,
we are trying to upgrade from hadoop 3.3.3 to hadoop 3.3.6 and it looks like
the defaults3clientfactory code still seems to be pulling from aws java sdk v1
instead of v2
(https://javadoc.io/doc/org.apache.hadoop/hadoop-aws/3.3.6/org/apache/hadoop/fs/s3a/DefaultS3ClientFactory.html).
> S3A: Upgrade AWS SDK to V2
> --------------------------
>
> Key: HADOOP-18073
> URL: https://issues.apache.org/jira/browse/HADOOP-18073
> Project: Hadoop Common
> Issue Type: Task
> Components: auth, fs/s3
> Affects Versions: 3.3.1
> Reporter: xiaowei sun
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7-aws
>
> Attachments: Upgrading S3A to SDKV2.pdf
>
>
> This task tracks upgrading Hadoop's AWS connector S3A from AWS SDK for Java
> V1 to AWS SDK for Java V2.
> Original use case:
> {quote}We would like to access s3 with AWS SSO, which is supported in
> software.amazon.awssdk:sdk-core:2.*.
> In particular, from
> [https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html],
> when to set 'fs.s3a.aws.credentials.provider', it must be
> "com.amazonaws.auth.AWSCredentialsProvider". We would like to support
> "software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider" which
> supports AWS SSO, so users only need to authenticate once.
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
