[
https://issues.apache.org/jira/browse/HADOOP-14661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17235490#comment-17235490
]
Steve Loughran commented on HADOOP-14661:
-----------------------------------------
Oh, that an *Awesome* way to fix this!
I'd assumed that every s3 request we built would need the header and I was
going to have to get the factored out request factory of HADOOP-16848 in before
worrying about it -I'd be patching every request, the way we do with encryption
settings on read/write/copy.
Setting it once and for all, well, that's easier isn't it?
I'd propose we formalise this with the default factory reading in an fs.s3a
option to enable requester pays from the config, setting it and then creating
the AWS console. Pretty much what you've done, just: wrapping in a config and
then enabling if set.
Testing would be fun. CommonCrawl provide some requester pays buckets, we'd
need to do the LIST/HEAD/GET sequence to validate the read path., and assume if
they work, so will writing (which is much more niche, isn't it?)
Do you fancy working on this?
> S3A to support Requester Pays Buckets
> -------------------------------------
>
> Key: HADOOP-14661
> URL: https://issues.apache.org/jira/browse/HADOOP-14661
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: common, util
> Affects Versions: 3.0.0-alpha3
> Reporter: Mandus Momberg
> Assignee: Mandus Momberg
> Priority: Minor
> Attachments: HADOOP-14661.patch
>
> Original Estimate: 2h
> Remaining Estimate: 2h
>
> Amazon S3 has the ability to charge the requester for the cost of accessing
> S3. This is called Requester Pays Buckets.
> In order to access these buckets, each request needs to be signed with a
> specific header.
> http://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
