[
https://issues.apache.org/jira/browse/KNOX-1204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399550#comment-16399550
]
Steve Loughran commented on KNOX-1204:
--------------------------------------
What starts off as "simple" soon gets complex fast. First its the "lets
translate this exception" and before long you are in deep trouble
We offer some structured access to the bits behind the curtain in the S3 client
(See WriteOperationsHelper).
going this way gives you
* someone else to handle the problem of keeping the aws jars in sync with the
rest of the hadoop codebase. Look at any of the "update AWS SDK" JIRAs for a
hint of that
* metrics wrapping all use of the calls
* retry policy round the calls
* translation of AWS exceptions in to various IOEs
(S3AUtils.translateException()
* fault injection for testing
> KIP-11 - S3 Access through Knox API
> -----------------------------------
>
> Key: KNOX-1204
> URL: https://issues.apache.org/jira/browse/KNOX-1204
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Reporter: Larry McCay
> Assignee: Larry McCay
> Priority: Major
> Fix For: 1.1.0
>
>
> h1. UC-5: S3 Integration
> While the Knox WebHDFS integration may still work for many cloud deployments,
> it does seem like a gap that there is no way to move files in and out of S3
> or other cloud storage mechanisms through Knox.
> We can actually combine UC-2 above to acquire temporary credentials on behalf
> of the authenticated users. We would request the IAM role and permissions
> that are appropriate for the user and their group memberships in order to
> access buckets protected with IAM roles. We could also combine with UC-4
> above to have encrypted files put into S3 that will only be able to be
> decrypted on-prem.
> It would require Knox to be granted permission in a given cloud deployment to
> make STS calls and may require AWS credentials for the Knox user to be an IAM
> role. We may also be able to assumeRole to the needed role for STS access.
> It will also require a Jersey service hosted in Knox to put files into S3
> (KnoxS3?) or we can create a pluggable backend and make it a more generic
> object store API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
