[ https://issues.apache.org/jira/browse/YARN-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291612#comment-17291612 ]
Peter Bacsko edited comment on YARN-10652 at 2/26/21, 12:26 PM: ---------------------------------------------------------------- [~sahuja] you're right in saying that it has no direct relation to the placement. In the first part of my comment, I was just thinking out loud that MAYBE using "_" instead of "." in the property is also a solution, but it comes with its own problems. The placement stuff is different, it's something that we haven't considered so far. Currently, the new placement engine simply replaces placeholders like "root.users.%user" to "root.users.firstname.lastname", which is likely not what we want. It will not work in percentage mode, because "firstname" is a parent and you can't create parents under a ManagedParentQueue. In the new weight mode, it can work, but again, the intention is to have something like "root.users.firstname_lastname", just a single leaf. was (Author: pbacsko): [~sahuja] you're right in saying that it has no direct relation in the placement. In the first part of my comment, I was just thinking out loud that MAYBE using "_" instead of "." in the property is also a solution, but it comes with its own problems. The placement stuff is different, it's something that we haven't considered so far. Currently, the new placement engine simply replaces placeholders like "root.users.%user" to "root.users.firstname.lastname", which is likely not what we want. It will not work in percentage mode, because "firstname" is a parent and you can't create parents under a ManagedParentQueue. In the new weight mode, it can work, but again, the intention is to have something like "root.users.firstname_lastname", just a single leaf. > Capacity Scheduler fails to handle user weights for a user that has a "." > (dot) in it > ------------------------------------------------------------------------------------- > > Key: YARN-10652 > URL: https://issues.apache.org/jira/browse/YARN-10652 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Affects Versions: 3.3.0 > Reporter: Siddharth Ahuja > Assignee: Siddharth Ahuja > Priority: Major > Attachments: Correct user weight of 0.76 picked up for the user with > a dot after the patch.png, Incorrect default user weight of 1.0 being picked > for the user with a dot before the patch.png, YARN-10652.001.patch > > > AD usernames can have a "." (dot) in them i.e. they can be of the format -> > {{firstname.lastname}}. However, if you specify a username with this format > against the Capacity Scheduler setting -> > {{yarn.scheduler.capacity.root.default.user-settings.firstname.lastname.weight}}, > it fails to be applied and is instead assigned the default of 1.0f weight. > This renders the user weight feature (being used as a means of setting user > priorities for a queue) unusable for such users. > This limitation comes from [1]. From [1], only word characters (A word > character: [a-zA-Z_0-9]) (see [2]) are permissible at the moment which is no > good for AD names that contain a "." (dot). > Similar discussion has been had in a few HADOOP jiras e.g. HADOOP-7050 and > HADOOP-15395 and the outcome was to use non-whitespace characters i.e. > instead of {{\w+}}, use {{\S+}}. > We could go down similar path and unblock this feature for the AD usernames > with a "." (dot) in them. > [1] > https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java#L1953 > [2] > https://docs.oracle.com/javase/tutorial/essential/regex/pre_char_classes.html -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org