[
https://issues.apache.org/jira/browse/YARN-11393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17651337#comment-17651337
]
ASF GitHub Bot commented on YARN-11393:
---------------------------------------
brumi1024 commented on code in PR #5201:
URL: https://github.com/apache/hadoop/pull/5201#discussion_r1055653806
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/FSConfigToCSConfigConverter.java:
##########
@@ -412,6 +418,20 @@ private void emitDefaultMaxAMShare() {
queueMaxAMShareDefault);
}
}
+
+ private void emitDefaultUserLimitFactor(Collection<FSLeafQueue> fsLeafQueue)
{
+ fsLeafQueue
+ .forEach((leafQueue) -> {
+ if (!capacitySchedulerConfig.
+ isAutoQueueCreationV2Enabled(leafQueue.getName())) {
+ capacitySchedulerConfig.setFloat(
+ CapacitySchedulerConfiguration.
+ PREFIX + leafQueue.getName() + DOT +
USER_LIMIT_FACTOR,
+ -1.0f);
+ }
+ });
+ }
+
Review Comment:
On the second look this class is more like for general config changes, while
there is a class called FSQueueConverter, which has the queue specific
converter logic implemented. Can you please move this whole logic there?
##########
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/converter/TestFSConfigToCSConfigConverter.java:
##########
@@ -182,7 +183,24 @@ public void testDefaultMaxAMShare() throws Exception {
conf.get(PREFIX + "root.admins.alice.maximum-am-resource-percent"));
assertNull("root.users.joe maximum-am-resource-percent should be null",
- conf.get(PREFIX + "root.users.joe maximum-am-resource-percent"));
+ conf.get(PREFIX + "root.users.joe.maximum-am-resource-percent"));
+ }
+
+ @Test
+ public void testDefaultUserLimitFactor() throws Exception {
+ converter.convert(config);
+
+ Configuration conf = converter.getCapacitySchedulerConfig();
+ String userLimitFactor =
+ conf.get(PREFIX + "root.default." + USER_LIMIT_FACTOR);
+
+ assertEquals("Default user limit factor", "-1.0", userLimitFactor);
+
+ assertEquals("root.users.joe user-limit-factor", "-1.0",
+ conf.get(PREFIX + "root.users.joe.user-limit-factor"));
+
+ assertEquals("root.admins.bob user-limit-factor", "-1.0",
+ conf.get(PREFIX + "root.admins.bob.user-limit-factor"));
Review Comment:
Can you please add an assertion where a parent class doesn't get the
user-limit-factor?
> Fs2cs could be extended to set ULF to -1 upon conversion
> --------------------------------------------------------
>
> Key: YARN-11393
> URL: https://issues.apache.org/jira/browse/YARN-11393
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Reporter: Susheel Gupta
> Assignee: Susheel Gupta
> Priority: Major
> Labels: pull-request-available
>
> A global configuration to set the default User Limit Factor to -1 on newly
> created queues.
> To solve this is to make fs2cs (Fair Scheduler to Capacity Scheduler tool)
> add the user-limit-factor value -1 to the conversion as default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
