[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-4399: --- Labels: oct16-hard (was: oct16-easy) > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-hard > Attachments: YARN-4399.001.patch, YARN-4399.002.patch, > YARN-4399.003.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-4399: - Labels: oct16-easy (was: ) > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Labels: oct16-easy > Attachments: YARN-4399.001.patch, YARN-4399.002.patch, > YARN-4399.003.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Billie Rinaldi updated YARN-4399: - Component/s: (was: scheduler) fairscheduler > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: fairscheduler >Affects Versions: 2.7.1 >Reporter: Yiqun Lin >Assignee: Yiqun Lin > Attachments: YARN-4399.001.patch, YARN-4399.002.patch, > YARN-4399.003.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4399: Attachment: YARN-4399.003.patch Resolve the remained compile error > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4399.001.patch, YARN-4399.002.patch, > YARN-4399.003.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4399: Attachment: YARN-4399.002.patch Resolve the comile error. > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4399.001.patch, YARN-4399.002.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-4399) FairScheduler allocated container should resetSchedulingOpportunities count of its priority
[ https://issues.apache.org/jira/browse/YARN-4399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated YARN-4399: Attachment: YARN-4399.001.patch > FairScheduler allocated container should resetSchedulingOpportunities count > of its priority > --- > > Key: YARN-4399 > URL: https://issues.apache.org/jira/browse/YARN-4399 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Affects Versions: 2.7.1 >Reporter: Lin Yiqun >Assignee: Lin Yiqun > Attachments: YARN-4399.001.patch > > > There is a bug on fairScheduler allocating containers when you configurate > the locality configs.When you attempt to assigned a container,it will invoke > {{FSAppAttempt#addSchedulingOpportunity}} whenever it can be assigned > successfully or not. And if you configurate the > yarn.scheduler.fair.locality.threshold.node and > yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value > will influence the locality of containers.Because if one container is > assigned successfully and its priority schedulingOpportunity count will be > increased, and second container will be increased again.This will may be let > their priority of allowedLocality degrade. And this will let this container > dealt by rackRequest. So I think in fairScheduler allocating container, if > the previous container was dealt, its priority of schedulerCount should be > reset to 0, and don't let its value influence container's allocating in next > iteration and this will increased the locality of containers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)