brumi1024 commented on a change in pull request #3361:
URL: https://github.com/apache/hadoop/pull/3361#discussion_r703001701
##########
File path:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractManagedParentQueue.java
##########
@@ -173,50 +168,31 @@ public AutoCreatedLeafQueueConfig getLeafQueueTemplate() {
return queueManagementPolicy;
}
- protected SortedMap<String, String> getConfigurationsWithPrefix
- (SortedMap<String, String> sortedConfigs, String prefix) {
- return sortedConfigs.subMap( prefix, prefix + Character.MAX_VALUE );
- }
-
- protected SortedMap<String, String> sortCSConfigurations() {
- SortedMap<String, String> sortedConfigs = new TreeMap(
- new Comparator<String>() {
- public int compare(String s1, String s2) {
- return s1.compareToIgnoreCase(s2);
- }
+ protected Map<String, String> getCSConfigurationsWithPrefix(String prefix) {
+ Map<String, String> configsWithPrefix = new HashMap<>();
- });
-
- for (final Iterator<Map.Entry<String, String>> iterator =
- csContext.getConfiguration().iterator(); iterator.hasNext(); ) {
- final Map.Entry<String, String> confKeyValuePair = iterator.next();
- sortedConfigs.put(confKeyValuePair.getKey(),
confKeyValuePair.getValue());
+ for (Map.Entry<String, String> confKeyValuePair :
+ csContext.getConfiguration().getPropsWithPrefix(prefix).entrySet()) {
+ configsWithPrefix.put(prefix + confKeyValuePair.getKey(),
confKeyValuePair.getValue());
}
- return sortedConfigs;
+
+ return configsWithPrefix;
}
- protected CapacitySchedulerConfiguration initializeLeafQueueConfigs(String
- configPrefix) {
+ protected CapacitySchedulerConfiguration initializeLeafQueueConfigs(String
configPrefix) {
CapacitySchedulerConfiguration leafQueueConfigs = new
- CapacitySchedulerConfiguration(new Configuration(false), false);
+ CapacitySchedulerConfiguration(csContext.getConf(), false);
Review comment:
Hi @JackWangCS,
Sorry I was not entirely clear. By now I mean the currently merged code
creates a new Configuration object in
ManagedParentQueue.getLeafQueueConfigs(CapacitySchedulerConfiguration
templateConfig, String leafQueueName) (which is called every time an
AutoCreatedLeafQueue is instantiated), but your patch solves that issue.
However I'm thinking about a bigger change by completely removing the config
object duplication (because it is error prone, see the bugfix
YarnConfiguration.RESOURCE_TYPES property cloning) by doing something similar
which the AQCv2 does (see the AutoCreatedQueueTemplate variable and its uses in
ParentQueue and LeafQueue). This way the new template configs are added as if
they would be normal user-defined configs, but it's better than creating a new
half-filled config for every AutoCreatedLeafQueue or after your patch
ManagedParent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
