[
https://issues.apache.org/jira/browse/HADOOP-13336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13336:
------------------------------------
Resolution: Fixed
Fix Version/s: 2.8.0
Target Version/s: (was: 2.9.0)
Status: Resolved (was: Patch Available)
This is now in Hadoop 2.8+, meaning everyone will get the ability to declare
different endpoints, auth and fadvice options for different buckets.
thank you to everyone who reviewed, from core design to the security auditing.
For reference, this was the merge strategy. I don't know if it was ideal, and
it was slow/careful going, but it worked (AFAIK)
# apply trunk branch
# cherry pick to branch-2; test/verify
# cherry pick to branch 2.8; test verify (and when problem found, fix and build
new patch.
# cherry pick to branch 2.8.0
# fork off Hadoop-13345 branch, apply trunk 012 patch, _and generate diff
between that and the branch with the full HADOOP-13336-HADOOP-13445-013 patch_
# merge trunk into HADOOP-13345 branch
# apply the diff previously generated
# commit and push
Yes, this did take >1 attempt
> S3A to support per-bucket configuration
> ---------------------------------------
>
> Key: HADOOP-13336
> URL: https://issues.apache.org/jira/browse/HADOOP-13336
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Fix For: 2.8.0
>
> Attachments: HADOOP-13336-006.patch, HADOOP-13336-007.patch,
> HADOOP-13336-010.patch, HADOOP-13336-011.patch,
> HADOOP-13336-HADOOP-13345-001.patch, HADOOP-13336-HADOOP-13345-002.patch,
> HADOOP-13336-HADOOP-13345-003.patch, HADOOP-13336-HADOOP-13345-004.patch,
> HADOOP-13336-HADOOP-13345-005.patch, HADOOP-13336-HADOOP-13345-006.patch,
> HADOOP-13336-HADOOP-13345-008.patch, HADOOP-13336-HADOOP-13345-009.patch,
> HADOOP-13336-HADOOP-13345-010.patch
>
>
> S3a now supports different regions, by way of declaring the endpoint —but you
> can't do things like read in one region, write back in another (e.g. a distcp
> backup), because only one region can be specified in a configuration.
> If s3a supported region declaration in the URL, e.g. s3a://b1.frankfurt
> s3a://b2.seol , then this would be possible.
> Swift does this with a full filesystem binding/config: endpoints, username,
> etc, in the XML file. Would we need to do that much? It'd be simpler
> initially to use a domain suffix of a URL to set the region of a bucket from
> the domain and have the aws library sort the details out itself, maybe with
> some config options for working with non-AWS infra
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
