[
https://issues.apache.org/jira/browse/HADOOP-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624188#comment-15624188
]
Aaron Fabbri commented on HADOOP-13651:
---------------------------------------
Following up your security comments, [[email protected]]. To make sure I'm
understanding, is it correct to say that:
- S3A FileSystem authorization is delegated to the AWS S3 SDK client.
- S3A code does not check hadoop user permissions, nor map hadoop users to AWS
credentials.
- So authorization is not "per user" in the hadoop sense, but "per
configuration" as that is where S3A credentials / instance roles / etc. are
defined.
- If a user tries to open a s3a:// FileSystem and they do not supply/config
proper AWS credentials, S3AFileSystem.initialize() will throw an exception in
verifyBucketExists() -> s3.doesBucketExist()
- It should be sufficient to only allow MetadataStore read/write operations
after success of S3 read/write operation (respectively).
Questions:
- If a user has valid AWS credentials, but no read permissions for given
bucket, what happens? Does initialize() succeed? (I can test this if needed)
- What needs to be done before we can commit this patch (besides the
LOG.isDebugEnabled thing)? I'd like to get this basic support in the feature
branch so [~eddyxu] and [~liuml07] can integrate with it. I agree we need to
address security and add tests to demonstrate its correctness. I'd be happy to
take a followup JIRA on that as well, or we can hold this patch up.
> S3Guard: S3AFileSystem Integration with MetadataStore
> -----------------------------------------------------
>
> Key: HADOOP-13651
> URL: https://issues.apache.org/jira/browse/HADOOP-13651
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Attachments: HADOOP-13651-HADOOP-13345.001.patch,
> HADOOP-13651-HADOOP-13345.002.patch, HADOOP-13651-HADOOP-13345.003.patch
>
>
> Modify S3AFileSystem et al. to optionally use a MetadataStore for metadata
> consistency and caching.
> Implementation should have minimal overhead when no MetadataStore is
> configured.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
