[
https://issues.apache.org/jira/browse/HADOOP-18095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Carl Jones updated HADOOP-18095:
---------------------------------------
Summary: S3A connector to improve support for all AWS partitions (was: s3a
connector to fully support AWS partitions,)
> S3A connector to improve support for all AWS partitions
> -------------------------------------------------------
>
> Key: HADOOP-18095
> URL: https://issues.apache.org/jira/browse/HADOOP-18095
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Priority: Minor
>
> There are some minor issues in using the S3A connector's more advanced
> features in china
> see https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
> Specifically, that "arn:aws:" prefix we use for all arns needs to be
> configurable so that aws-cn can be used instead.
> This means finding where we create and use these in production code
> (dynamically creating IAM role policies) and in tests, and making it
> configurable.
> proposed
> * add an option {{fs.s3a.aws.partition}}, default aws.
> * new StoreContext methods to query this, and create the arn for the current
> bucket (string concat or from the bucket's ARN if created with an AP ARN)
> * docs
> I remember ABFS had a problem with oauth endpoints, that was a lot more
> serious.
> Can't think of real tests for this, other than verifying that if you create
> an invalid partition "aws-mars" some things break.
> someone needs to run all our existing tests in china, including those with
> IAM roles and SSE-KMS.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
