[
https://issues.apache.org/jira/browse/HADOOP-12723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288907#comment-15288907
]
Steve Loughran commented on HADOOP-12723:
-----------------------------------------
# The patch reverts a couple of things in the toString() call. I'll go with
line 1395, but not sure about the L1383 change; the existing code omits the
line completely.
# {{getAWSCredentialsProvider}} : need to have consensus there on what
classloader to use
# Once HADOOP-13130 gets in then it'll break {{TestS3AAWSCredentialsProvider}};
all Amazon exceptions will be caught and wrapped by IOEs; your test will need
to change. It'll be
a race to see whose gets in first...
# {{TestS3AAWSCredentialsProvider}}: codepaths that expected to raise an
exception should have a {{fail(message)}} on the path which is expected to
fail. Otherwise test failures could be missed
# you'll need to cover the new properties in the AWS {{site/index.md}} file.
One question I have here is how would this relate to HADOOP-12537; STS support?
I believe That mandates the STS JAR on the CP, and adds extra config options
for the binding. If it's the latter, the binding info, this means that your
patch isn't going to be sufficient on its own. What would it take to support
it?
Presumably, it'd need a new provider which looked up the session key from the
properties, yes? If so, that'd be the first one to add after this patch,
marking the STS JAR as a "provided" dependency of hadoop-aws.
What other providers do you have in mind?
> S3A: Add ability to plug in any AWSCredentialsProvider
> ------------------------------------------------------
>
> Key: HADOOP-12723
> URL: https://issues.apache.org/jira/browse/HADOOP-12723
> Project: Hadoop Common
> Issue Type: New Feature
> Components: fs/s3
> Affects Versions: 2.7.1
> Reporter: Steven Wong
> Assignee: Steven Wong
> Attachments: HADOOP-12723.0.patch, HADOOP-12723.1.patch,
> HADOOP-12723.2.patch, HADOOP-12723.3.patch, HADOOP-12723.4.patch
>
>
> Although S3A currently has built-in support for
> {{org.apache.hadoop.fs.s3a.BasicAWSCredentialsProvider}},
> {{com.amazonaws.auth.InstanceProfileCredentialsProvider}}, and
> {{org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider}}, it does not
> support any other credentials provider that implements the
> {{com.amazonaws.auth.AWSCredentialsProvider}} interface. Supporting the
> ability to plug in any {{com.amazonaws.auth.AWSCredentialsProvider}} instance
> will expand the options for S3 credentials, such as:
> * temporary credentials from STS, e.g. via
> {{com.amazonaws.auth.STSSessionCredentialsProvider}}
> * IAM role-based credentials, e.g. via
> {{com.amazonaws.auth.STSAssumeRoleSessionCredentialsProvider}}
> * a custom credentials provider that satisfies one's own needs, e.g.
> bucket-specific credentials, user-specific credentials, etc.
> To support this, we can add a configuration for the fully qualified class
> name of a credentials provider, to be loaded by
> {{S3AFileSystem.initialize(URI, Configuration)}}.
> The configured credentials provider should implement
> {{com.amazonaws.auth.AWSCredentialsProvider}} and have a constructor that
> accepts {{(URI uri, Configuration conf)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
