[
https://issues.apache.org/jira/browse/HADOOP-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17753118#comment-17753118
]
Daniel Carl Jones commented on HADOOP-18095:
--------------------------------------------
I'd like to see things like the ARNResource retired (or simplified) following
the SDK V2 upgrade. With SDK V2 (and maybe V1 I think), you can pass the ARN
into the bucket field and it'll know which endpoint (and optionally region) to
use. If we can encourage users to configure the bucket name with ARN and not
set the endpoint, I'm optimistic it'll work and also handle other partitions
without us needing to worry.
> S3A connector to improve support for all AWS partitions
> -------------------------------------------------------
>
> Key: HADOOP-18095
> URL: https://issues.apache.org/jira/browse/HADOOP-18095
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Priority: Minor
>
> There are some minor issues in using the S3A connector's more advanced
> features in china
> see https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
> Specifically, that "arn:aws:" prefix we use for all arns needs to be
> configurable so that aws-cn can be used instead.
> This means finding where we create and use these in production code
> (dynamically creating IAM role policies) and in tests, and making it
> configurable.
> proposed
> * add an option {{fs.s3a.aws.partition}}, default aws.
> * new StoreContext methods to query this, and create the arn for the current
> bucket (string concat or from the bucket's ARN if created with an AP ARN)
> * docs
> I remember ABFS had a problem with oauth endpoints, that was a lot more
> serious.
> Can't think of real tests for this, other than verifying that if you create
> an invalid partition "aws-mars" some things break.
> someone needs to run all our existing tests in china, including those with
> IAM roles and SSE-KMS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
