[
https://issues.apache.org/jira/browse/HADOOP-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17615211#comment-17615211
]
ASF GitHub Bot commented on HADOOP-18481:
-----------------------------------------
steveloughran commented on code in PR #4973:
URL: https://github.com/apache/hadoop/pull/4973#discussion_r991486622
##########
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java:
##########
@@ -640,7 +640,8 @@ public static AWSCredentialProviderList
buildAWSProviderList(
AWSCredentialProviderList providers = new AWSCredentialProviderList();
for (Class<?> aClass : awsClasses) {
- if (aClass.getName().contains(AWS_AUTH_CLASS_PREFIX)) {
+ if
(!aClass.getSimpleName().equals("EnvironmentVariableCredentialsProvider")
Review Comment:
how about having an array of simple names to skip on, with that,
EC2ContainerCredentialsProviderWrapper, and InstanceProfileCredentialsProvider
in. we only added our own IAMInstanceCredentialsProvider with delegation
tokens, HADOOP-15449, and until then just included the aws sdk one directly.
any refs to either can be mapped to IAMInstanceCredentialsProvider before
trying to instantiate
> AWS v2 SDK warning to skip warning of EnvironmentVariableCredentialsProvider
> ----------------------------------------------------------------------------
>
> Key: HADOOP-18481
> URL: https://issues.apache.org/jira/browse/HADOOP-18481
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> looking at test output with the sdk warnings enabled, it is now always
> warning of a v1 provider reference, even if the user hasn't set any
> fs.s3a.credential.provider option
> {code}
> 2022-10-05 14:09:09,733 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
> 2022-10-05 14:09:09,733 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
> 2022-10-05 14:09:09,734 [setup] WARN s3a.SDKV2Upgrade
> (LogExactlyOnce.java:warn(39)) - Directly referencing AWS SDK V1 credential
> provider com.amazonaws.auth.EnvironmentVariableCredentialsProvider. AWS SDK
> V1 credential providers will be removed once S3A is upgraded to SDK V2
> 2022-10-05 14:09:09,734 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is com.amazonaws.auth.EnvironmentVariableCredentialsProvider
> 2022-10-05 14:09:09,734 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
> {code}
> This is because the EnvironmentVariableCredentialsProvider provider is on the
> default list of providers.
> Everybody who is using the S3 a connector and who has not explicitly declared
> a new set of providers excluding this one will be seeing the error message.
> Proposed:
> Don't warn on this provider. Instead with the v2 move the classname can be
> patched to switch to a modified one.
> The alternative would be to provide an s3a specific env var provider subclass
> of this; and while that is potentially good in future it is a bit more effort
> for the forthcoming 3.3.5 release.
> And especially because and it will not be in previous versions people cannot
> explicitly switch to it in their configs and be confident it will always be
> there,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
