[
https://issues.apache.org/jira/browse/YARN-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702059#comment-16702059
]
Suma Shivaprasad edited comment on YARN-9039 at 11/28/18 3:57 PM:
------------------------------------------------------------------
Thanks [~bibinchundatt] for your suggestions. Unfortunately the Cloud Storage
ACLs are a bit more complicated than that.
User specific folder access control could work if only a single user needs
access to the folder. But gets cumbersome when implementing application
specific acls on the logs objects stored under that folder i.e granting access
to other users to read the logs
- There are limits on the size of bucket/IAM policies -
https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-my-controlling-access-to-s3-resources/.
- Having a bucket per user also has limitations due to the 100 buckets per
account limits in S3.
In the long term, there are couple of options to address this
- Logs CLI talks to Log Webservice instead of talking to storage directly and
only yarn user has access to write/read from the log aggregation storage
bucket. Can track this effort in a separate jira to fix the issue of YARN CLI
imposing ACLs
- All FileSystem calls go through a FS proxy which can authorize storage ACLs
via an Authorization framework like Apache Ranger
In the current jira, we could address the issue of ATSv2 LogWebservice not
checking ACLs while serving logs through REST/UI and I could upload a patch for
the same
Makes sense?
Did not get what is the fix needed for LogAggregationHtmlBlock#checkAcls ? It
doesnt seem to be calling LogAggregationFileController.readAggregatedLogs
was (Author: suma.shivaprasad):
Thanks [~bibinchundatt] for your suggestions. Unfortunately the Cloud Storage
ACLs are a bit more complicated than that.
User specific folder access control could work if only a single user needs
access to the folder. But gets cumbersome when implementing application
specific acls on the logs objects stored under that folder i.e granting access
to other users to read the logs
- There are limits on the size of bucket/IAM policies -
https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-my-controlling-access-to-s3-resources/.
- Having a bucket per user also has limitations due to the 100 buckets per
account limits in S3.
In the long term, there are couple of options to address this
- Logs CLI talks to Log Webservice instead of talking to storage directly and
only yarn user has access to write/read from the log aggregation storage
bucket. Can track this effort in a separate jira to fix the issue of YARN CLI
imposing ACLs
- All FileSystem calls go through a FS proxy which can authorize storage ACLs
via an Authorization framework like Apache Ranger
In the current jira, we could address the issue of ATSv2 LogWebservice not
checking ACLs while serving logs through REST/UI and I could upload a patch for
the same
Makes sense?
> App ACLs are not validated when serving logs from LogWebService
> ---------------------------------------------------------------
>
> Key: YARN-9039
> URL: https://issues.apache.org/jira/browse/YARN-9039
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Critical
> Attachments: YARN-9039.1.patch, YARN-9039.2.patch
>
>
> App Acls are not being validated while serving logs through REST and UI2 via
> Log Webservice
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
