[
https://issues.apache.org/jira/browse/HADOOP-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506982#comment-16506982
]
Steve Loughran commented on HADOOP-15525:
-----------------------------------------
As you note, *most of this is done* in HADOOP-15176. See [Assumed
Roles|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/assumed_roles.md];
while {{org.apache.hadoop.fs.s3a.auth.RoleModel}} gives you a declarative
model to set up a role for testing; makes it relatively straightforward to
generate S3A filesystem instances with reduced rights and see that things fall
back.
# the post-create actions are happy with perm failures, they just end up
leaving markers around. So I'm not worried about
# the pre-create actions look for the parent entry & make sure it isn't a file;
maybe handle a permission failure there?
# big issue is rename, especially with S3Guard involved
HADOOP-15183 covers that. If you try to rename a directory and you don't have
read or delete perms to a path underneath, rename() fails *without updating
s3guard as to its current state*. It needs to sync up & add tombstones for all
deleted entries, new entries for created stuff.
* Delete will also need to add the tombstones for all successful deletes,
* createFakeDirectory to handle failure on the GET or the PUT
* maybe also: add specific metric for permissions failures
rename() is what worries me the most with s3guard, as its the one which will
end up most inconsistent.
Everything is setup to add tests for these; harder part is fixing. Thank you
for volunteering :)
> s3a: clarify / improve support for mixed ACL buckets
> ----------------------------------------------------
>
> Key: HADOOP-15525
> URL: https://issues.apache.org/jira/browse/HADOOP-15525
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.0.0
> Reporter: Aaron Fabbri
> Assignee: Aaron Fabbri
> Priority: Major
>
> Scenario: customer wants to only give a Hadoop cluster access to a subtree of
> an S3 bucket.
> For example, assume Hadoop uses some IAM identity "hadoop", which they wish
> to grant full permission to everything under the following path:
> s3a://bucket/a/b/c/hadoop-dir
> they don't want hadoop user to be able to read/list/delete anything outside
> of the hadoop-dir "subdir"
> Problems:
> To implement the "directory structure on flat key space" emulation logic we
> use to present a Hadoop FS on top of a blob store, we need to create / delete
> / list ancestors of {{hadoop-dir}}. (to maintain the invariants (1) zero-byte
> object with key ending in '/' exists iff empty directory is there and (2)
> files cannot live beneath files, only directories.)
> I'd like us to (1) document a an example with IAM ACLs policies that gets
> this basic functionality, and consider (2) making improvements to make this
> easier.
> We've discussed some of these issues before but I didn't see a dedicated JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
