[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172276#comment-14172276 ] Hudson commented on YARN-2656: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #712 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/712/]) YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. (zjshen: rev 1220bb72d452521c6f09cebe1dd77341054ee9dd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilterInitializer.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesDelegationTokenAuthentication.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilter.java * hadoop-yarn-project/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Zhijie Shen Fix For: 2.6.0 Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172376#comment-14172376 ] Hudson commented on YARN-2656: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1902 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1902/]) YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. (zjshen: rev 1220bb72d452521c6f09cebe1dd77341054ee9dd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesDelegationTokenAuthentication.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilterInitializer.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java * hadoop-yarn-project/CHANGES.txt RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Zhijie Shen Fix For: 2.6.0 Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14172454#comment-14172454 ] Hudson commented on YARN-2656: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1927 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1927/]) YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. (zjshen: rev 1220bb72d452521c6f09cebe1dd77341054ee9dd) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilterInitializer.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesDelegationTokenAuthentication.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Zhijie Shen Fix For: 2.6.0 Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171354#comment-14171354 ] Zhijie Shen commented on YARN-2656: --- Kick the jenkins again, as HADOOP-11181 is already committed. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, YARN-2656.4.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171456#comment-14171456
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674303/YARN-2656.4.patch
against trunk revision cdce883.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.ha.TestZKFailoverControllerStress
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5388//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5388//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5388//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, YARN-2656.4.patch,
apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171756#comment-14171756
]
Xuan Gong commented on YARN-2656:
-
overall Looks good. One comment:
{code}
+ public void doFilter(ServletRequest request, ServletResponse response,
+ FilterChain filterChain) throws IOException, ServletException {
+HttpServletRequest req = (HttpServletRequest) request;
+// For backward compatibility, allow use of the old header field
+final String oldHeader = req.getHeader(OLD_HEADER);
+if (oldHeader != null !oldHeader.isEmpty()) {
+ String newHeader =
+ req.getHeader(DelegationTokenAuthenticator.DELEGATION_TOKEN_HEADER);
+ if (newHeader == null || newHeader.isEmpty()) {
+HttpServletRequestWrapper wrapper = new HttpServletRequestWrapper(req)
{
+ @Override
+ public String getHeader(String name) {
+if (name
+ .equals(DelegationTokenAuthenticator.DELEGATION_TOKEN_HEADER)) {
+ return oldHeader;
+}
+return super.getHeader(name);
+ }
+};
+super.doFilter(wrapper, response, filterChain);
}
+} else {
+ super.doFilter(request, response, filterChain);
}
{code}
Here, we handled case:
1)when oldHeader is null/empty
2)When oldHeader is not null/empty and newHeader is null/empty.
Do we need to handle the case when oldHeader is not null/empty and newHeader is
not null/empty here as well ?
So, maybe we could check newHeader first. As I understand, if newHeader is not
null/empty, it will be used no matter whether oldHeader is set or not.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, YARN-2656.4.patch,
apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171803#comment-14171803 ] Varun Vasudev commented on YARN-2656: - If both headers are specified, we should use the new one. That code is only there for backwards compatibility. If the new header is specified, there's no need for us to do anything. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, YARN-2656.4.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171813#comment-14171813 ] Xuan Gong commented on YARN-2656: - No even need to call super.doFilter() ??? RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, YARN-2656.4.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171816#comment-14171816 ] Varun Vasudev commented on YARN-2656: - bq. No even need to call super.doFilter() ??? I completely missed that. Great catch! [~zjshen], we need to handle the case Xuan pointed out. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, YARN-2656.4.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171918#comment-14171918 ] Xuan Gong commented on YARN-2656: - +1 LGTM RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Zhijie Shen Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171932#comment-14171932
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674896/YARN-2656.5.patch
against trunk revision 0260231.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5394//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5394//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5394//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Zhijie Shen
Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch,
apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14171993#comment-14171993 ] Hudson commented on YARN-2656: -- FAILURE: Integrated in Hadoop-trunk-Commit #6262 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6262/]) YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. (zjshen: rev 1220bb72d452521c6f09cebe1dd77341054ee9dd) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilterInitializer.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/security/http/RMAuthenticationFilter.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/TestRMWebServicesDelegationTokenAuthentication.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAdminService.java RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Zhijie Shen Fix For: 2.6.0 Attachments: YARN-2656.3.patch, YARN-2656.4.patch, YARN-2656.5.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14169857#comment-14169857 ] Varun Vasudev commented on YARN-2656: - The latest patch looks good to me. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, YARN-2656.4.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166479#comment-14166479
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674111/YARN-2656.3.patch
against trunk revision 684170d.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5358//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5358//artifact/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5358//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5358//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, apache-yarn-2656.0.patch,
apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166943#comment-14166943
]
Varun Vasudev commented on YARN-2656:
-
Thanks for the patch [~zjshen].
{quote}
2. Change proxyuser prefix to yarn.resourcemanager. The other
DelegationTokenAuthenticationFilter use cases take the specific prefix as well
instead of the common one hadoop. The benefit should be different use cases
can have their individual proxyuser setting.
{quote}
I don't think we need to introduce a new variable to configure proxyusers. I
think it's reasonable for now to assume that proxyusers are setup for RPC and
REST. I think we should use hadoop.proxyusers instead of introducing
yarn.resourcemanager.proxyusers.
Apart from that the patch looks good to me. Thanks again for all the help!
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, apache-yarn-2656.0.patch,
apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167116#comment-14167116
]
Varun Vasudev commented on YARN-2656:
-
{quote}
One more thing: RMAuthenticationHandler seems to be useless, we may want to
discard it. Let's do it separately?
{quote}
Forgot to address this - lets do it as part of a separate ticket.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, apache-yarn-2656.0.patch,
apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167130#comment-14167130 ] Zhijie Shen commented on YARN-2656: --- bq. I think we should use hadoop.proxyusers instead of introducing yarn.resourcemanager.proxyusers. KMS and HTTPFS are using this feature as well, but using their individual prefix before proxyusers. I think it makes sense because each component can edit its own proxyusers. In particular, it's desired in single-node cluster. Thinking about the property name again, should we use yarn.resourcemanager.webapp to be consistent with the existing YARN config names? RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167137#comment-14167137 ] Varun Vasudev commented on YARN-2656: - Thanks for pointing out the KMS and HTTPFS examples. I think the yarn.resourcemanager.webapp.proxyuser prefix probably makes most sense. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: YARN-2656.3.patch, apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14167922#comment-14167922
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12674303/YARN-2656.4.patch
against trunk revision f4b7e99.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5369//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5369//artifact/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/5369//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5369//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: YARN-2656.3.patch, YARN-2656.4.patch,
apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164866#comment-14164866 ] Zhijie Shen commented on YARN-2656: --- I did some investigation on the test failure. It seems not to be the problem the test case, but the issue of DelegationTokenManager, which allows user to set external tokenManager, but make the assumption that all the token is o.a.h.security.token.delegation.DelegationTokenIdentifier. RM send RMDelegationTokenIdentifier and serialize the token in one way, and now DelegationTokenManager receives the token, and deserialize it in another way, resulting the bug. Will file ticket for this issue. RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[ https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14166414#comment-14166414 ] Zhijie Shen commented on YARN-2656: --- [~vvasudev], thanks for the patch. It looks good to me overall. Based on this patch I made code polishing as well as the following changes: 1. Keep the filter name being RMAuthenticationFilter as we don't change the class name. 2. Change proxyuser prefix to yarn.resourcemanager. The other DelegationTokenAuthenticationFilter use cases take the specific prefix as well instead of the common one hadoop. The benefit should be different use cases can have their individual proxyuser setting. Upload a new patch accordingly. One more thing: RMAuthenticationHandler seems to be useless, we may want to discard it. Let's do it separately? RM web services authentication filter should add support for proxy user --- Key: YARN-2656 URL: https://issues.apache.org/jira/browse/YARN-2656 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2656.0.patch, apache-yarn-2656.1.patch, apache-yarn-2656.2.patch The DelegationTokenAuthenticationFilter adds support for doAs functionality. The RMAuthenticationFilter should expose this as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163260#comment-14163260
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673554/apache-yarn-2656.0.patch
against trunk revision 1efd9c9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
org.apache.hadoop.yarn.server.resourcemanager.TestRMAdminService
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5322//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5322//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2656.0.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14163849#comment-14163849
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673611/apache-yarn-2656.1.patch
against trunk revision 1efd9c9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
org.apache.hadoop.yarn.server.resourcemanager.TestRMAdminService
The following test timeouts occurred in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.http.TestHttpServerLifecycle
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5326//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5326//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2656.0.patch, apache-yarn-2656.1.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2656) RM web services authentication filter should add support for proxy user
[
https://issues.apache.org/jira/browse/YARN-2656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14164038#comment-14164038
]
Hadoop QA commented on YARN-2656:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12673640/apache-yarn-2656.2.patch
against trunk revision e16e25a.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5328//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5328//console
This message is automatically generated.
RM web services authentication filter should add support for proxy user
---
Key: YARN-2656
URL: https://issues.apache.org/jira/browse/YARN-2656
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Attachments: apache-yarn-2656.0.patch, apache-yarn-2656.1.patch,
apache-yarn-2656.2.patch
The DelegationTokenAuthenticationFilter adds support for doAs functionality.
The RMAuthenticationFilter should expose this as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
