[jira] [Updated] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sevada Abraamyan updated YARN-2892: --- Attachment: YARN-2892.patch Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228579#comment-14228579
]
Sevada Abraamyan commented on YARN-2892:
After researching Hadoop security a bit further, I believe the shared cluster
scenario I was thinking about earlier shouldn't be a problem. Administrators
define the mapping from principle to short username via the
{{hadoop.security.auth_to_local}} configuration and if two principles from
different kerberos realms mapped to the same user then it would have to be done
on purpose by the administrator. In which case the assumed behavior is that
both principles are describing the same subject and therefore the users will
have the authorization privileges. Please correct me if I'm wrong and of course
I would love to get a code review of the latest patch.
_In regards to the other mismatches between short/full username in
ClientRMService, I will be opening a different JIRA ticket so as to not dilute
the main purpose of this ticket which was to fix the missing AMRM tokens._
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-2911) Issues with GetApplications request in secure cluster
Sevada Abraamyan created YARN-2911:
--
Summary: Issues with GetApplications request in secure cluster
Key: YARN-2911
URL: https://issues.apache.org/jira/browse/YARN-2911
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Both problems arise from the fact that the RM stores the short username of the
app submitter.
1) When the {{GetApplicationsRequest}} contains a
{{ApplicationsRequestScope.OWN}} filter, i.e. it wants to filter out all apps
not owned by the user. The RM attempts to match the full username of the
GetApplications requester against the stored short username to determine if the
requester is the owner of the app. In a secure cluster this can fail as the two
are not always equivalent.
2) The {{GetApplicationsRequest}} can be used to filter the the set of app
returned to be only those which were submitted/owned by a set of users. Once
again there is a mismatch here between short/full usernames. Since the client
specifies the set of users, theoretically they can pass in a set of short
usernames which would makes this feature work in a secure cluster. However, it
is not expected that a client will have the correct
{{hadoop.security.auth_to_local}} configuration and therefore they can not
always be expected to get the correct short usernames.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228601#comment-14228601
]
Hadoop QA commented on YARN-2892:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12684218/YARN-2892.patch
against trunk revision 1556f86.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
org.apache.hadoop.yarn.server.resourcemanager.TestRM
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5950//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5950//console
This message is automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228639#comment-14228639 ] Sevada Abraamyan commented on YARN-2892: I do not believe these test failures are related to the submitted patch. Also, I'm unable to reproduce the errors on my local machine. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
