[
https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144472#comment-16144472
]
ASF GitHub Bot commented on YARN-2162:
--------------------------------------
Github user kambatla commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/261#discussion_r135650575
--- Diff:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
---
@@ -287,27 +289,69 @@ public float getReservableNodes() {
*
* @throws AllocationConfigurationException
*/
- public static Resource parseResourceConfigValue(String val)
+ public static ConfigurableResource parseResourceConfigValue(String val)
throws AllocationConfigurationException {
+ ConfigurableResource configurableResource;
try {
val = StringUtils.toLowerCase(val);
- int memory = findResource(val, "mb");
- int vcores = findResource(val, "vcores");
- return BuilderUtils.newResource(memory, vcores);
+ if (val.contains("%")) {
+ configurableResource = new ConfigurableResource(
+ getResourcePercentage(val));
+ } else {
+ int memory = findResource(val, "mb");
+ int vcores = findResource(val, "vcores");
+ configurableResource = new ConfigurableResource(
+ BuilderUtils.newResource(memory, vcores));
+ }
} catch (AllocationConfigurationException ex) {
throw ex;
} catch (Exception ex) {
throw new AllocationConfigurationException(
"Error reading resource config", ex);
}
+ return configurableResource;
+ }
+
+ private static double[] getResourcePercentage(
+ String val) throws AllocationConfigurationException {
+ double[] resourcePercentage = new double[ResourceType.values().length];
+ String[] strings = val.split(",");
+ if (strings.length == 1) {
+ double percentage = findPercentage(strings[0], "");
+ for (int i = 0 ; i < ResourceType.values().length ; i++) {
+ resourcePercentage[i] = percentage/100;
+ }
+ } else {
+ double memPercentage = findPercentage(val, "memory");
+ double vcorePercentage = findPercentage(val, "cpu");
+ resourcePercentage[0] = memPercentage/100;
--- End diff --
With ResourceTypes coming in, I feel this has to be handled better. Loop
over and use that index instead?
> add ability in Fair Scheduler to optionally configure maxResources in terms
> of percentage
> -----------------------------------------------------------------------------------------
>
> Key: YARN-2162
> URL: https://issues.apache.org/jira/browse/YARN-2162
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler, scheduler
> Reporter: Ashwin Shankar
> Assignee: Yufei Gu
> Labels: scheduler
> Attachments: YARN-2162.001.patch, YARN-2162.002.patch,
> YARN-2162.003.patch
>
>
> minResources and maxResources in fair scheduler configs are expressed in
> terms of absolute numbers X mb, Y vcores.
> As a result, when we expand or shrink our hadoop cluster, we need to
> recalculate and change minResources/maxResources accordingly, which is pretty
> inconvenient.
> We can circumvent this problem if we can optionally configure these
> properties in terms of percentage of cluster capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
