[
https://issues.apache.org/jira/browse/YARN-9886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970813#comment-16970813
]
Szilard Nemeth edited comment on YARN-9886 at 11/9/19 2:16 PM:
---------------------------------------------------------------
Hi [~kmarton]!
1. I was wondering what happens if
{{RMAppManager#getUserNameFromApplicationTag}} receives a list of application
tags, like: {"u=someuser", "u="}, so the second user is missing. So I written a
testcase:
{code:java}
@Test
public void testWronglyQualifiedUserNameInTag()
throws YarnException {
String user = "user1";
String expectedQueue = "user1Queue";
String userIdTag = "u=user2";
String wrongUserIdTag = "u=";
setApplicationTags("tag1", wrongUserIdTag, userIdTag, "tag2");
enableApplicationTagPlacement(true, user);
verifyPlacementUsername(expectedQueue, user, user);
}
{code}
Just put into TestAppManager and run it. An ArrayIndexOutOfBoundsException can
come so easily.
Please fix this one!
For reference, here's the stacktrace of the test run:
{code:java}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.getUserNameFromApplicationTag(RMAppManager.java:985)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.getUserNameForPlacement(RMAppManager.java:943)
at
org.apache.hadoop.yarn.server.resourcemanager.AppManagerTestBase$TestRMAppManager.getUserNameForPlacement(AppManagerTestBase.java:111)
at
org.apache.hadoop.yarn.server.resourcemanager.TestAppManager.verifyPlacementUsername(TestAppManager.java:1322)
at
org.apache.hadoop.yarn.server.resourcemanager.TestAppManager.testWronglyQualifiedUserNameInTag(TestAppManager.java:1290)
{code}
2. Nit: In {{RMAppManager#getUserNameForPlacement}}:
The following declaration + assignment could be one statement:
{code:java}
UserGroupInformation callerUGI;
callerUGI = UserGroupInformation.createRemoteUser(userNameFromAppTag);
{code}
3. {{TestAppManager#testGetUserNameForPlacementTagBasedPlacementWrongUserId}}:
I think this method name is a little bit misleading. The user id is not wrong,
it's just not whitelisted.
Btw, what's the difference between this testcase and
{{testGetUserNameForPlacementNotWhitelistedUser}}?
Thanks!
was (Author: snemeth):
Hi [~kmarton]!
1. I was wondering what happens if
{{RMAppManager#getUserNameFromApplicationTag}} receives a list of application
tags, like: {"u=someuser", "u="}, so the second user is missing. So I written a
testcase:
{code:java}
@Test
public void testWronglyQualifiedUserNameInTag()
throws YarnException {
String user = "user1";
String expectedQueue = "user1Queue";
String userIdTag = "u=user2";
String wrongUserIdTag = "u=";
setApplicationTags("tag1", wrongUserIdTag, userIdTag, "tag2");
enableApplicationTagPlacement(true, user);
verifyPlacementUsername(expectedQueue, user, user);
}
{code}
Just put into TestAppManager and run it. An ArrayIndexOutOfBoundsException can
come so easily.
Please fix this one!
For reference, here's the stacktrace of the test run:
{code:java}
java.lang.ArrayIndexOutOfBoundsException: 1
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.getUserNameFromApplicationTag(RMAppManager.java:985)
at
org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.getUserNameForPlacement(RMAppManager.java:943)
at
org.apache.hadoop.yarn.server.resourcemanager.AppManagerTestBase$TestRMAppManager.getUserNameForPlacement(AppManagerTestBase.java:111)
at
org.apache.hadoop.yarn.server.resourcemanager.TestAppManager.verifyPlacementUsername(TestAppManager.java:1322)
at
org.apache.hadoop.yarn.server.resourcemanager.TestAppManager.testWronglyQualifiedUserNameInTag(TestAppManager.java:1290)
{code}
2. Nit: In {{RMAppManager#getUserNameForPlacement}}:
The following declaration + assignment could be one statement:
{code:java}
UserGroupInformation callerUGI;
callerUGI = UserGroupInformation.createRemoteUser(userNameFromAppTag);
{code}
3. {{TestAppManager#testGetUserNameForPlacementTagBasedPlacementWrongUserId}}:
I think this method name is a little bit misleading. The user id is not wrong,
it's just not whitelisted.
Btw, what's the difference between this testcase and
testGetUserNameForPlacementNotWhitelistedUser?
Thanks!
> Queue mapping based on userid passed through application tag
> ------------------------------------------------------------
>
> Key: YARN-9886
> URL: https://issues.apache.org/jira/browse/YARN-9886
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Reporter: Kinga Marton
> Assignee: Kinga Marton
> Priority: Major
> Attachments: YARN-9886-WIP.patch, YARN-9886.001.patch,
> YARN-9886.002.patch, YARN-9886.003.patch
>
>
> There are situations when the real submitting user differs from the user what
> arrives to YARN. For example in case of a Hive application when Hive
> impersonation is turned off, the hive queries will run as Hive user and the
> mapping is done based on this username. Unfortunately in this case YARN
> doesn't have any information about the real user and there are cases when the
> customer may want to map these applications to the real submitting user's
> queue instead of the Hive queue.
> For these cases, if they would pass the username in the application tag we
> may read it and use it during the queue mapping, if that user has rights to
> run on the real user's queue.
> [~sunilg] please correct me if I missed something.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
