[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954747#comment-15954747 ] Hoang-Mai Dinh commented on YARN-2892: -- Please fix this ticket! > Unable to get AMRMToken in unmanaged AM when using a secure cluster > --- > > Key: YARN-2892 > URL: https://issues.apache.org/jira/browse/YARN-2892 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/unmanaged-AM-launcher, resourcemanager >Reporter: Sevada Abraamyan >Assignee: Sevada Abraamyan > Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch > > > An AMRMToken is retrieved from the ApplicationReport by the YarnClient. > When the RM creates the ApplicationReport and sends it back to the client it > makes a simple security check whether it should include the AMRMToken in the > report (See createAndGetApplicationReport in RMAppImpl).This security check > verifies that the user who submitted the original application is the same > user who is requesting the ApplicationReport. If they are indeed the same > user then it includes the AMRMToken, otherwise it does not include it. > The problem arises from the fact that when an application is submitted, the > RM saves the short username of the user who created the application (See > submitApplication in ClientRmService). Afterwards when the ApplicationReport > is requested, the system tries to match the full username of the requester > against the previously stored short username. > In a secure cluster using Kerberos this check fails because the principle is > stripped from the username when we request a short username. So for example > the short username might be "Foo" whereas the full username is > "f...@company.com" > Note: A very similar problem has been previously reported > ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867718#comment-15867718 ] Varun Saxena commented on YARN-2892: What happened to this fix? Maybe someone can rebase and we can get this in. > Unable to get AMRMToken in unmanaged AM when using a secure cluster > --- > > Key: YARN-2892 > URL: https://issues.apache.org/jira/browse/YARN-2892 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/unmanaged-AM-launcher, resourcemanager >Reporter: Sevada Abraamyan >Assignee: Sevada Abraamyan > Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch > > > An AMRMToken is retrieved from the ApplicationReport by the YarnClient. > When the RM creates the ApplicationReport and sends it back to the client it > makes a simple security check whether it should include the AMRMToken in the > report (See createAndGetApplicationReport in RMAppImpl).This security check > verifies that the user who submitted the original application is the same > user who is requesting the ApplicationReport. If they are indeed the same > user then it includes the AMRMToken, otherwise it does not include it. > The problem arises from the fact that when an application is submitted, the > RM saves the short username of the user who created the application (See > submitApplication in ClientRmService). Afterwards when the ApplicationReport > is requested, the system tries to match the full username of the requester > against the previously stored short username. > In a secure cluster using Kerberos this check fails because the principle is > stripped from the username when we request a short username. So for example > the short username might be "Foo" whereas the full username is > "f...@company.com" > Note: A very similar problem has been previously reported > ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14527497#comment-14527497 ] Wangda Tan commented on YARN-2892: -- [~vinodkv], Agree, this is also what [~jianhe] suggested. [~sevada], could you rebase the patch? Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525968#comment-14525968 ] Jian He commented on YARN-2892: --- I think this does change the behavior that clientToAMTokenIdentifer was earlier constructed with userName, but now changed to shortUserName instead. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14525374#comment-14525374 ] Junping Du commented on YARN-2892: -- Sure. [~leftnoteasy], I agree. please go ahead. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524357#comment-14524357
]
Hadoop QA commented on YARN-2892:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 11s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 45s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 51s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 45s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | yarn tests | 52m 6s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 26s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesFairScheduler |
| | hadoop.yarn.server.resourcemanager.TestAppManager |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestAllocationFileLoaderService
|
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684732/YARN-2892.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / d3d019c |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7584/console |
This message was automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524174#comment-14524174
]
Hadoop QA commented on YARN-2892:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 15m 6s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 42s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 45s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 50s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 16s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | yarn tests | 52m 19s | Tests passed in
hadoop-yarn-server-resourcemanager. |
| | | 89m 30s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12684732/YARN-2892.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 3393461 |
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/7577/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/7577/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/7577/console |
This message was automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14523938#comment-14523938 ] Wangda Tan commented on YARN-2892: -- [~djp], I think this fix only changes how to get token instead of incompatible behavior change. Thoughts? Retriggerred Jenkins to see if patch still good. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238000#comment-14238000 ] Junping Du commented on YARN-2892: -- Wait... I looked at the patch again, looks like it will bring serious incompatible issue: the applicationReport return to client include short name now rather than full name before. We should be super carefully now as we are supporting YARN rolling upgrade since 2.6. Any other ways? Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238137#comment-14238137 ] Rohith commented on YARN-2892: -- bq. the applicationReport return to client include short name now rather than full name before. I did not get where exactly breaking compatibility.. Before patch also, application report sends short name instead of full name. Am I missing anything? Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238307#comment-14238307
]
Sevada Abraamyan commented on YARN-2892:
I don't see this either. However, one thing I did notice is that with the patch
we are now changing how ClientToAMToken is constructed as we are using the
short name instead of full name.
{code}
@Override
public ApplicationReport createAndGetApplicationReport(String clientUserName,
boolean allowAccess) {
if (UserGroupInformation.isSecurityEnabled()) {
// get a token so the client can communicate with the app attempt
// NOTE: token may be unavailable if the attempt is not running
TokenClientToAMTokenIdentifier attemptClientToAMToken =
this.currentAttempt.createClientToken(clientUserName);
if (attemptClientToAMToken != null) {
clientToAMToken = BuilderUtils.newClientToAMToken(
attemptClientToAMToken.getIdentifier(),
attemptClientToAMToken.getKind().toString(),
attemptClientToAMToken.getPassword(),
attemptClientToAMToken.getService().toString());
}
...
{code}
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238520#comment-14238520
]
Sevada Abraamyan commented on YARN-2892:
On second thought I think [~djp] was referring directly to the code I
referenced above. Since we'd rather not modify the public interface of RMApp,
maybe we should continue passing in the full username to
_createAndGetApplicationReport_ and prior to AMRM security check we can use
this full username to construct a short username. It seems a bit hacky but I'm
not sure how else we can avoid not breaking the public interface.
The easiest way I can see doing this is by using something like the following:
{code}
UserGroupInformation remoteUser =
UserGroupInformation.getRemoteUser(clientUserName);
String shortUsername = remoteUser.getShortUsername();
{code}
Another solution could be to do the following:
{code}
//if security is set to kerberos...
HadoopKerberosName kbName = new HadoopKerberosName(clientUserName);
String shortUsername = kbName.getShortUsername()
{code}
The first solution is a bit strange but looks more attractive to me as it
allows _RMAppImpl_ to stay agnostic to the underlying security framework. Any
suggestions?
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232753#comment-14232753 ] Rohith commented on YARN-2892: -- +1(non-binding) lgtm Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14233199#comment-14233199 ] Junping Du commented on YARN-2892: -- +1. Patch looks good. Will commit it shortly. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14231311#comment-14231311 ] Rohith commented on YARN-2892: -- Thanks for the patch!! Patch looks good to me overall. One minor comment in test, user Logger instead of sysout. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232179#comment-14232179 ] Sevada Abraamyan commented on YARN-2892: Thanks for catching that mistake [~ rohithsharma]. I've submitted a new patch which no longer uses sysout. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14232273#comment-14232273
]
Hadoop QA commented on YARN-2892:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12684732/YARN-2892.patch
against trunk revision 52bcefc.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5984//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5984//console
This message is automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228579#comment-14228579
]
Sevada Abraamyan commented on YARN-2892:
After researching Hadoop security a bit further, I believe the shared cluster
scenario I was thinking about earlier shouldn't be a problem. Administrators
define the mapping from principle to short username via the
{{hadoop.security.auth_to_local}} configuration and if two principles from
different kerberos realms mapped to the same user then it would have to be done
on purpose by the administrator. In which case the assumed behavior is that
both principles are describing the same subject and therefore the users will
have the authorization privileges. Please correct me if I'm wrong and of course
I would love to get a code review of the latest patch.
_In regards to the other mismatches between short/full username in
ClientRMService, I will be opening a different JIRA ticket so as to not dilute
the main purpose of this ticket which was to fix the missing AMRM tokens._
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228601#comment-14228601
]
Hadoop QA commented on YARN-2892:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12684218/YARN-2892.patch
against trunk revision 1556f86.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
org.apache.hadoop.yarn.server.resourcemanager.TestRM
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5950//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5950//console
This message is automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch, YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14228639#comment-14228639 ] Sevada Abraamyan commented on YARN-2892: I do not believe these test failures are related to the submitted patch. Also, I'm unable to reproduce the errors on my local machine. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Sevada Abraamyan Attachments: YARN-2892.patch, YARN-2892.patch An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14226939#comment-14226939
]
Sevada Abraamyan commented on YARN-2892:
In an offline discussion [~ahadr] brought up a good point by mentioning that
validating against the short name might pose a potential security hole. Does
anyone know what the original reasoning was to have the app stored using the
short name instead of the fully qualified name? If I'm understanding the
potential impact then it means in a shared cluster scenario where there are two
kerberos principles it is possible for two users to have the same username
prefix and therefore access each others tokens? In addition to
{{ClientRMService}} we are also using getShortUsername in
{{ApplicationACLsManager}} and {{QueueACLsManager}} to name a few.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225685#comment-14225685
]
Rohith commented on YARN-2892:
--
Thanks [~sevada] for providing patch.
Looking deeply, {{createAndGetApplicationReport}} is called from
{{getApplicationReport}} {{getApplications}} and {{getQueueInfo}}. These
methods also should use getShortUserName instead getUserName. {{getQueueInfo}}
not required since user name passed is null.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225752#comment-14225752
]
Hadoop QA commented on YARN-2892:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683754/YARN-2892.patch
against trunk revision a655973.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/5941//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5941//console
This message is automatically generated.
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[
https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225840#comment-14225840
]
Sevada Abraamyan commented on YARN-2892:
[~rohithsharma] thanks for pointing this out. It also looks like there are
other instances in ClientRMService unrelated to createAndGetApplicationReport.
For example the following code in getApplications is incorrect as well.
{code}
public GetApplicationsResponse getApplications(
...
if (scope == ApplicationsRequestScope.OWN
!callerUGI.getUserName().equals(application.getUser())) {
continue;
}
...
{code}
I'll do a more thorough sweep of ClientRMService and update the patch. Also,
while I'm working on this, maybe it might be good to revist why getQueueInfo
passes a null username to createAndGetApplicationReport. Is there any reason we
wouldn't want the owner of the application to get the AMRMTokens in the
getQueueInfo request?
Unable to get AMRMToken in unmanaged AM when using a secure cluster
---
Key: YARN-2892
URL: https://issues.apache.org/jira/browse/YARN-2892
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Sevada Abraamyan
Assignee: Sevada Abraamyan
Attachments: YARN-2892.patch
An AMRMToken is retrieved from the ApplicationReport by the YarnClient.
When the RM creates the ApplicationReport and sends it back to the client it
makes a simple security check whether it should include the AMRMToken in the
report (See createAndGetApplicationReport in RMAppImpl).This security check
verifies that the user who submitted the original application is the same
user who is requesting the ApplicationReport. If they are indeed the same
user then it includes the AMRMToken, otherwise it does not include it.
The problem arises from the fact that when an application is submitted, the
RM saves the short username of the user who created the application (See
submitApplication in ClientRmService). Afterwards when the ApplicationReport
is requested, the system tries to match the full username of the requester
against the previously stored short username.
In a secure cluster using Kerberos this check fails because the principle is
stripped from the username when we request a short username. So for example
the short username might be Foo whereas the full username is
f...@company.com
Note: A very similar problem has been previously reported
([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232])
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222363#comment-14222363 ] Rohith commented on YARN-2892: -- [~jianhe] Could you please assign this issue to [~sevada]. Even I am not able to assign issue to him, may be he is not yet contributor!! Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Rohith An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221879#comment-14221879 ] Rohith commented on YARN-2892: -- Thanks [~sevada] for reporting the issue. Would you like to contribute patch or shall I take over this? Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14221883#comment-14221883 ] Rohith commented on YARN-2892: -- [~sevada] I have assigned the issue to me as of now. Feel free to assign your self if you would like to contribute. Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Rohith An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2892) Unable to get AMRMToken in unmanaged AM when using a secure cluster
[ https://issues.apache.org/jira/browse/YARN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14222086#comment-14222086 ] Sevada Abraamyan commented on YARN-2892: [~rohithsharma], I did intend to submit a patch. However, I couldn't find out how to assign the issue to myself. Maybe I don't have the correct privaleges? In any case, if you have not yet started on a patch, can you please assign this to me? Thanks! Unable to get AMRMToken in unmanaged AM when using a secure cluster --- Key: YARN-2892 URL: https://issues.apache.org/jira/browse/YARN-2892 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Reporter: Sevada Abraamyan Assignee: Rohith An AMRMToken is retrieved from the ApplicationReport by the YarnClient. When the RM creates the ApplicationReport and sends it back to the client it makes a simple security check whether it should include the AMRMToken in the report (See createAndGetApplicationReport in RMAppImpl).This security check verifies that the user who submitted the original application is the same user who is requesting the ApplicationReport. If they are indeed the same user then it includes the AMRMToken, otherwise it does not include it. The problem arises from the fact that when an application is submitted, the RM saves the short username of the user who created the application (See submitApplication in ClientRmService). Afterwards when the ApplicationReport is requested, the system tries to match the full username of the requester against the previously stored short username. In a secure cluster using Kerberos this check fails because the principle is stripped from the username when we request a short username. So for example the short username might be Foo whereas the full username is f...@company.com Note: A very similar problem has been previously reported ([Yarn-2232|https://issues.apache.org/jira/browse/YARN-2232]) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
