[
https://issues.apache.org/jira/browse/HADOOP-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16809234#comment-16809234
]
Steve Loughran commented on HADOOP-16233:
-----------------------------------------
Given the +1 by larry
bq. Seems to me that this means of determining whether something is public or
private is brittle since it is derived by assumptions about certain
filesystems. I'd like to see a follow up JIRA for addressing this in a more
explicit way to consider something public.
yeah Maybe the PathCapabilities patch could declare if a path supported POSIX
permissions, which on, say, ABFS could change on a file-by-file basis, so you
really would need to check the path, not just the FS.
Or you let apps hint which files can be considered for public sharing and they
must be expected to be world readable. Hey, maybe we could fallback: if the NM
can't D/L a public file, it just says "lets try as the user" and convert to
private? That'd handle things like ranger policies, object store permission
policies etc.
bq. Here is my +1 for the change to align S3A filesystem with those assumptions.
thanks
> S3AFileStatus to declare that isEncrypted() is always true
> ----------------------------------------------------------
>
> Key: HADOOP-16233
> URL: https://issues.apache.org/jira/browse/HADOOP-16233
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.0, 3.1.1, 3.0.3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
>
> Some bits of yarn treat encrypted files differently than unencrypted ones,
> and get confused by S3A (and other stores) where the permissions say "world
> readable" but they aren't really.
> Proposed: always declare that the file/dir is encrypted.
> Need to fix {{ITestS3AContractOpen}} to skip the encryption test, or just
> push it down to HDFS/webhdfs, as the rule "empty files are not encrypted"
> doesn't have to hold elsewhere
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
