[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Attachment: YARN-2162.002.patch Uploaded patch v2 to address style issues and the unit test failure. > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > Attachments: YARN-2162.001.patch, YARN-2162.002.patch > > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Attachment: (was: YARN-2162.001.patch) > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > Attachments: YARN-2162.001.patch > > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[
https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yufei Gu updated YARN-2162:
---
Attachment: YARN-2162.001.patch
Patch v1 add ability to configure maxShare as a percentage of cluster resources.
To do that, we need to also changes {{maxChildResources}},
{{queueMaxResourcesDefault}}. The calculation of max share is done in
FSQueue#getMaxShare(), which is decoupled from allocation file loading since it
takes cluster resources as one input. Some logic also needs to move from
allocation file loading phase.
> Fair Scheduler :ability to optionally configure minResources and maxResources
> in terms of percentage
>
>
> Key: YARN-2162
> URL: https://issues.apache.org/jira/browse/YARN-2162
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler, scheduler
>Reporter: Ashwin Shankar
>Assignee: Yufei Gu
> Labels: scheduler
> Attachments: YARN-2162.001.patch
>
>
> minResources and maxResources in fair scheduler configs are expressed in
> terms of absolute numbers X mb, Y vcores.
> As a result, when we expand or shrink our hadoop cluster, we need to
> recalculate and change minResources/maxResources accordingly, which is pretty
> inconvenient.
> We can circumvent this problem if we can optionally configure these
> properties in terms of percentage of cluster capacity.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Attachment: YARN-2162.001.patch > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > Attachments: YARN-2162.001.patch > > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Attachment: (was: YARN-2162.001.patch) > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Attachment: YARN-2162.001.patch > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > Attachments: YARN-2162.001.patch > > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-2162: --- Component/s: fairscheduler > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: fairscheduler, scheduler >Reporter: Ashwin Shankar >Assignee: Yufei Gu > Labels: scheduler > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-2162: - Target Version/s: (was: 2.8.0) > Fair Scheduler :ability to optionally configure minResources and maxResources > in terms of percentage > > > Key: YARN-2162 > URL: https://issues.apache.org/jira/browse/YARN-2162 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Reporter: Ashwin Shankar > Labels: scheduler > > minResources and maxResources in fair scheduler configs are expressed in > terms of absolute numbers X mb, Y vcores. > As a result, when we expand or shrink our hadoop cluster, we need to > recalculate and change minResources/maxResources accordingly, which is pretty > inconvenient. > We can circumvent this problem if we can optionally configure these > properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated YARN-2162: -- Target Version/s: 2.6.1, 2.7.2 (was: 2.6.0) Moving all tickets targeted for the already closed release 2.6.0 into 2.6.1/2.7.2. Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage Key: YARN-2162 URL: https://issues.apache.org/jira/browse/YARN-2162 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Ashwin Shankar Labels: scheduler minResources and maxResources in fair scheduler configs are expressed in terms of absolute numbers X mb, Y vcores. As a result, when we expand or shrink our hadoop cluster, we need to recalculate and change minResources/maxResources accordingly, which is pretty inconvenient. We can circumvent this problem if we can optionally configure these properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-2162: --- Target Version/s: 2.6.0 (was: 2.5.0) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage Key: YARN-2162 URL: https://issues.apache.org/jira/browse/YARN-2162 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Ashwin Shankar Labels: scheduler minResources and maxResources in fair scheduler configs are expressed in terms of absolute numbers X mb, Y vcores. As a result, when we expand or shrink our hadoop cluster, we need to recalculate and change minResources/maxResources accordingly, which is pretty inconvenient. We can circumvent this problem if we can optionally configure these properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2162) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage
[ https://issues.apache.org/jira/browse/YARN-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Shankar updated YARN-2162: - Summary: Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage (was: Fair Scheduler :ability to configure minResources and maxResources in terms of percentage) Fair Scheduler :ability to optionally configure minResources and maxResources in terms of percentage Key: YARN-2162 URL: https://issues.apache.org/jira/browse/YARN-2162 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Ashwin Shankar Labels: scheduler minResources and maxResources in fair scheduler configs are expressed in terms of absolute numbers X mb, Y vcores. As a result, when we expand or shrink our hadoop cluster, we need to recalculate and change minResources/maxResources accordingly, which is pretty inconvenient. We can circumvent this problem if we can optionally configure these properties in terms of percentage of cluster capacity. -- This message was sent by Atlassian JIRA (v6.2#6252)
