[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008910#comment-14008910 ] Hudson commented on YARN-1937: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1782 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1782/]) YARN-1937. Added owner-only ACLs support for Timeline Client and server. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1597186) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelinePutResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/MemoryTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security/TimelineACLsManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security/TestTimelineACLsManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestTimelineWebServices.java Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: 2.5.0 Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch, YARN-1937.4.patch, YARN-1937.5.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14008409#comment-14008409 ] Hudson commented on YARN-1937: -- FAILURE: Integrated in Hadoop-trunk-Commit #5609 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5609/]) YARN-1937. Added owner-only ACLs support for Timeline Client and server. Contributed by Zhijie Shen. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1597186) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timeline/TimelinePutResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/ApplicationHistoryServer.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/MemoryTimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/TimelineStore.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security/TimelineACLsManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/AHSWebApp.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/timeline/security/TestTimelineACLsManager.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TestTimelineWebServices.java Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: 2.5.0 Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch, YARN-1937.4.patch, YARN-1937.5.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14006938#comment-14006938 ] Zhijie Shen commented on YARN-1937: --- bq. A meta comment - may be this isn't a RESTy way of rejecting requests? The situation here is that we may not deny the whole request, but part of the entities may not be put. Otherwise, we can simply return a HTTP 403. However, in this case we have to do the customized response, don't we? bq. We should also make this a public enum so that users know what system-filters exist bq. Do we really need TimelinePutError.SYSTEM_FILTER_CONFLICT? Similarly injectOwnerInfo. Or is it better to simply ignore the overriding filters? Not sure, thinking aloud. I intentionally don't allow user to set or modify the system filter, preventing them from affecting the system logic. For example, if user1 post the entity by setting ENTITY_OWNER = user2, the posted entity will never be accessible by user1.Therefore the enums don't need to be visible by users. However, in the documententation, we can explicitly tell users what are the reserved filter names by the timeline service. Users shouldn't use it. bq. Agree with Varun about admins. You should simply start respecting YarnConfiguration.YARN_ADMIN_ACL. See ApplicationACLsManager for e.g and reuse AdminACLsManager here itself. Sure. As I already filed a ticket about adding admin acls. How about working on this issue separately? Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007414#comment-14007414 ] Varun Vasudev commented on YARN-1937: - +1 on the latest patch. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch, YARN-1937.4.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[
https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007555#comment-14007555
]
Hadoop QA commented on YARN-1937:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646469/YARN-1937.4.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3797//console
This message is automatically generated.
Add entity-level access control of the timeline data for owners only
Key: YARN-1937
URL: https://issues.apache.org/jira/browse/YARN-1937
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch,
YARN-1937.4.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[
https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007686#comment-14007686
]
Hadoop QA commented on YARN-1937:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646580/YARN-1937.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3803//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3803//console
This message is automatically generated.
Add entity-level access control of the timeline data for owners only
Key: YARN-1937
URL: https://issues.apache.org/jira/browse/YARN-1937
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch,
YARN-1937.4.patch, YARN-1937.5.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007715#comment-14007715 ] Vinod Kumar Vavilapalli commented on YARN-1937: --- +1, looks good. Checking this in. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch, YARN-1937.4.patch, YARN-1937.5.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[
https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007759#comment-14007759
]
Hadoop QA commented on YARN-1937:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12646580/YARN-1937.5.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3810//console
This message is automatically generated.
Add entity-level access control of the timeline data for owners only
Key: YARN-1937
URL: https://issues.apache.org/jira/browse/YARN-1937
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Fix For: 2.5.0
Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch,
YARN-1937.4.patch, YARN-1937.5.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007800#comment-14007800 ] Zhijie Shen commented on YARN-1937: --- Tested the timeline security stack so far end-to-end on my local cluster. It seems to work fine, authentication works as expected, and only owner can view his posted timeline data. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: 2.5.0 Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch, YARN-1937.4.patch, YARN-1937.5.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001572#comment-14001572 ] Varun Vasudev commented on YARN-1937: - My feedback - 1. admins should be allowed to view all entities - the current patch only allows the owner 2. There should be a way to prevent un-authenticated users from posting entities. In the current patch, the owner is set to null but the entity is saved. Admins should be allowed to insist that users be authenticated before posting entities. Otherwise it looks fine to me. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002145#comment-14002145 ] Zhijie Shen commented on YARN-1937: --- Hi Varun, thanks for review! W.R.T to you concern, see my comments bellow: bq. 1. admins should be allowed to view all entities - the current patch only allows the owner Yeah, we definitely need to allow admin as well as users/groups on the allowed access list. However, for now, since we still haven't admin module, I prefer to defer the admin check until we support admin role (see YARN-2059, YARN-2060). bq. 2. There should be a way to prevent un-authenticated users from posting entities. In the current patch, the owner is set to null but the entity is saved. Admins should be allowed to insist that users be authenticated before posting entities. IMHO, we should allow un-authenticated to post entities. Otherwise, the unsecured cluster cannot leverage the timeline service. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[
https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002160#comment-14002160
]
Varun Vasudev commented on YARN-1937:
-
{quote}
IMHO, we should allow un-authenticated to post entities. Otherwise, the
unsecured cluster cannot leverage the timeline service.
{quote}
Sorry, I should have explained myself better. You are entirely correct that
unsecured clusters should be able to leverage the timeline service. My point
was that in a secure cluster, the admin should be allowed to insist that all
posts to the timeline server be authenticated.
Add entity-level access control of the timeline data for owners only
Key: YARN-1937
URL: https://issues.apache.org/jira/browse/YARN-1937
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1937) Add entity-level access control of the timeline data for owners only
[ https://issues.apache.org/jira/browse/YARN-1937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14002178#comment-14002178 ] Zhijie Shen commented on YARN-1937: --- bq. My point was that in a secure cluster, the admin should be allowed to insist that all posts to the timeline server be authenticated. When authentication is enabled, putEntities API is only accessible by the authenticated users. YARN-1936 is to make the client be able to put the timeline data in secure mode. Therefore, we don't need to worry about that un-authenticated users will post the timeline data. Add entity-level access control of the timeline data for owners only Key: YARN-1937 URL: https://issues.apache.org/jira/browse/YARN-1937 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-1937.1.patch, YARN-1937.2.patch, YARN-1937.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
