[
https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576941#comment-17576941
]
ASF GitHub Bot commented on HADOOP-18382:
-----------------------------------------
steveloughran commented on code in PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#discussion_r940562613
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/aws_sdk_upgrade.md:
##########
@@ -0,0 +1,76 @@
+<!---
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# Upgrading S3A to AWS SDK V2
+
+This document explains the upcoming work for upgrading S3A to AWS SDK V2.
+This work is tracked in
[HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073).
+
+## Why do we want to upgrade?
Review Comment:
prefer "why the upgrade"
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:
##########
@@ -849,6 +849,10 @@ private void bindAWSClient(URI name, boolean dtEnabled)
throws IOException {
// with it if so.
LOG.debug("Using delegation tokens");
+ LOG.warn(
Review Comment:
you need to use a LogOnce; we should consider having a special log just for
v2 warnings, so people can turn it off entirely (i would for example)
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -637,6 +637,11 @@ public static AWSCredentialProviderList
buildAWSProviderList(
AWSCredentialProviderList providers = new AWSCredentialProviderList();
for (Class<?> aClass : awsClasses) {
+ if (aClass.getName().contains(AWS_AUTH_CLASS_PREFIX)) {
Review Comment:
again, log through the proposed LogOnce log
> Upgrade AWS SDK to V2 - Prerequisites
> --------------------------------------
>
> Key: HADOOP-18382
> URL: https://issues.apache.org/jira/browse/HADOOP-18382
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Ahmar Suhail
> Assignee: Ahmar Suhail
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> We want to update the AWS SDK to V2, before we do this we should warn on
> things that will no longer supported. The following changes should be made:
>
> *
> [getAmazonS3Client()|https://github.com/apache/hadoop/blob/221eb2d68d5b52e4394fd36cb30d5ee9ffeea7f0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L1174]
> - Warn that this method will be removed
> *
> [initCustomSigners()|https://github.com/apache/hadoop/blob/03cfc852791c14fad39db4e5b14104a276c08e59/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/SignerManager.java#L65]
> - Warn that the interface is changing, any custom signers will need to be
> updated
> *
> [bindAWSClient|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L840]
> - If DT is enabled, warn that credential providers interface is changing,
> any custom cred providers used in binding classes will need to be updated
> *
> [buildAWSProviderList|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java#L618]
> - if any SDK V1 cred providers are in this list, warn that these will be
> removed
> *
> [S3ClientFactory|https://github.com/apache/hadoop/blob/221eb2d68d5b52e4394fd36cb30d5ee9ffeea7f0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java]
> - Update javadocs to say this interface will be replaced by a V2 client
> factory, mark interface deprecated?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
