[
https://issues.apache.org/jira/browse/HADOOP-14556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16665595#comment-16665595
]
Steve Loughran commented on HADOOP-14556:
-----------------------------------------
Patch 016
# improve subclassing, scope of methods
# lifecycle change: serviceStart triggers bonded/unbonded immediately. This has
forced me to add a package-scoped way to reset/rebind a delegation token for
ease of testing binding without going near UGI; that's a special codepath
# the DT Binding always provides a list of credential providers, even when
deployed without a DT.
# move to Optional<> over nullables for fields, embrace java8
#* document how to subclass
# Use of origin diagnostics string through identifiers is consistent
# empty AWS credentials can be marshalled
The fact that credential setup is always controlled when you turn DTs on is
signficant but needed once you start doing really complex stuff with the
bindings: your DT provider needs to be able to bootstrap your login directly.
While things like wasb/abfs force you declare a consistent pair of (issue,
auth) entries, having the dt binding do everything lets it instantiate
instances all glued together
Tests, s3 ireland. {{ITestS3ATemporaryCredentials.}} tests are failing because
somehow a dt binding is being enabled (shared fs instance?) and this dynamic
binding isnt' switching to the session creds, only key & secret are being
handed in, which is rejected "UnrecognizedClientException"
I'm going to fix that, and in the process tune the default auth chain to be
that which the DTs will also use by default, to, in order
* Temp/session credentials (fs.s3a.{access, secret, session)
* long-lived credentials (fs.s3a.{access, secret)
* env vars
* IAM ref
I'm also considering adding an IAMInstance cred provider which does async
refresh, but for that it needs to be ref counted, so I'm pushing it out of this
JIRA
> S3A to support Delegation Tokens
> --------------------------------
>
> Key: HADOOP-14556
> URL: https://issues.apache.org/jira/browse/HADOOP-14556
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Attachments: HADOOP-14556-001.patch, HADOOP-14556-002.patch,
> HADOOP-14556-003.patch, HADOOP-14556-004.patch, HADOOP-14556-005.patch,
> HADOOP-14556-007.patch, HADOOP-14556-008.patch, HADOOP-14556-009.patch,
> HADOOP-14556-010.patch, HADOOP-14556-010.patch, HADOOP-14556-011.patch,
> HADOOP-14556-012.patch, HADOOP-14556-013.patch, HADOOP-14556-014.patch,
> HADOOP-14556-015.patch, HADOOP-14556-016.patch, HADOOP-14556.oath-002.patch,
> HADOOP-14556.oath.patch
>
>
> S3A to support delegation tokens where
> * an authenticated client can request a token via
> {{FileSystem.getDelegationToken()}}
> * Amazon's token service is used to request short-lived session secret & id;
> these will be saved in the token and marshalled with jobs
> * A new authentication provider will look for a token for the current user
> and authenticate the user if found
> This will not support renewals; the lifespan of a token will be limited to
> the initial duration. Also, as you can't request an STS token from a
> temporary session, IAM instances won't be able to issue tokens.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
