[
https://issues.apache.org/jira/browse/HADOOP-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065035#comment-17065035
]
Steve Loughran commented on HADOOP-16930:
-----------------------------------------
Happy to take a PR against trunk with the docs.
With hadoop 3.3 we support delegation tokens so we can either marshall those
secrets into a DT or generate session/role credentials from it, again, which
are marshalled over. So you can use your own secrets to run work in a hadoop
cluster without any credentials.
We're only doing changes against branch-3.x right now; I don't really want to
add a new config provider to the default list if I can avoid it. More testing,
docs, support. Note, we now use the container aware
{{com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper}} for VMs
> Add com.amazonaws.auth.profile.ProfileCredentialsProvider to hadoop-aws docs
> ----------------------------------------------------------------------------
>
> Key: HADOOP-16930
> URL: https://issues.apache.org/jira/browse/HADOOP-16930
> Project: Hadoop Common
> Issue Type: Improvement
> Components: documentation, fs/s3
> Reporter: Nicholas Chammas
> Priority: Minor
>
> There is a very, very useful S3A authentication method that is not currently
> documented: {{com.amazonaws.auth.profile.ProfileCredentialsProvider}}
> This provider lets you source your AWS credentials from a shared credentials
> file, typically stored under {{~/.aws/credentials}}, using a [named
> profile|https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html].
> All you need is to set the {{AWS_PROFILE}} environment variable, and the
> provider will get the appropriate credentials for you.
> I discovered this from my coworkers, but cannot find it in the docs for
> hadoop-aws. I'd expect to see it at least mentioned in [this
> section|https://hadoop.apache.org/docs/r2.9.2/hadoop-aws/tools/hadoop-aws/index.html#S3A_Authentication_methods].
> It should probably be added to the docs for every minor release that
> supports it, which I'd guess includes 2.8 on up.
> (This provider should probably also be added to the default list of
> credential provider classes, but we can address that in another ticket. I can
> say that at least in 2.9.2, it's not in the default list.)
> (This is not to be confused with
> {{com.amazonaws.auth.InstanceProfileCredentialsProvider}}, which serves a
> completely different purpose.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
