[jira] [Commented] (YARN-111) Application level priority in Resource Manager Schedulers
[ https://issues.apache.org/jira/browse/YARN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573142#comment-13573142 ] nemon lou commented on YARN-111: Finally i use two Queues in Capacity Scheduler to basically meet our needs. Both queue has a Absolute Max Capacity of 100% .The queue with higher priority has more Absolute Capacity configured(85%). Job which need high priority will be submitted to the queue which has more Absolute Capacity configured. Application level priority in Resource Manager Schedulers - Key: YARN-111 URL: https://issues.apache.org/jira/browse/YARN-111 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.0.1-alpha Reporter: nemon lou We need application level priority for Hadoop 2.0,both in FIFO scheduler and Capacity Scheduler. In Hadoop 1.0.x,job priority is supported. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-111) Application level priority in Resource Manager Schedulers
[ https://issues.apache.org/jira/browse/YARN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13573157#comment-13573157 ] Vinod Kumar Vavilapalli commented on YARN-111: -- So, can we close this as won't fix? Though it is a useful feature, it has many dangerous pitfalls as noted and clearly also has alternative means of achieving it. Application level priority in Resource Manager Schedulers - Key: YARN-111 URL: https://issues.apache.org/jira/browse/YARN-111 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.0.1-alpha Reporter: nemon lou We need application level priority for Hadoop 2.0,both in FIFO scheduler and Capacity Scheduler. In Hadoop 1.0.x,job priority is supported. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-111) Application level priority in Resource Manager Schedulers
[ https://issues.apache.org/jira/browse/YARN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459420#comment-13459420 ] nemon lou commented on YARN-111: to Harsh J I have looked into the code. CS's LeafQueue keeps active applications and pending applacations in TreeSet. TreeSet's comparator comes from CapacityScheduler's applicationComparator . ApplicationComparator's compare method is like this: return a1.getApplicationId().getId() - a2.getApplicationId().getId(); (org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.java line 106) So application's priority doesn't take effect.Am i right? to Robert I'm not sure whether job priority is removed or not in recent MR1 code.But it will be very nice of you to add application level priority in YARN. :) Application level priority in Resource Manager Schedulers - Key: YARN-111 URL: https://issues.apache.org/jira/browse/YARN-111 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.0.1-alpha Reporter: nemon lou We need application level priority for Hadoop 2.0,both in FIFO scheduler and Capacity Scheduler. In Hadoop 1.0.x,job priority is supported. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-111) Application level priority in Resource Manager Schedulers
[ https://issues.apache.org/jira/browse/YARN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13458680#comment-13458680 ] Robert Joseph Evans commented on YARN-111: -- I thought that job priority was added into the 1.0 line a while ago, and then it was more or less removed. You could still specify a priority, but the priority itself was ignored. I am in favor of adding in priority again, but I just want to be sure that we address the reasons why it was removed/disabled in 1.0. Assuming that I am remembering things correctly. Application level priority in Resource Manager Schedulers - Key: YARN-111 URL: https://issues.apache.org/jira/browse/YARN-111 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.0.1-alpha Reporter: nemon lou We need application level priority for Hadoop 2.0,both in FIFO scheduler and Capacity Scheduler. In Hadoop 1.0.x,job priority is supported. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-111) Application level priority in Resource Manager Schedulers
[ https://issues.apache.org/jira/browse/YARN-111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459016#comment-13459016 ] Harsh J commented on YARN-111: -- Robert, I still see Job priority exist in MR1 (1.x). Which JIRA removed this, per your comment above? Or is this something CapacityScheduler specific we're discussing? In YARN I see Priority coming in for generally all resource requests (which I assume does apply to the AM too) and hence http://hadoop.apache.org/docs/current/api/org/apache/hadoop/yarn/api/records/ApplicationSubmissionContext.html#setPriority(org.apache.hadoop.yarn.api.records.Priority) ought to work, as the CS's LeafQueue does look at it? Application level priority in Resource Manager Schedulers - Key: YARN-111 URL: https://issues.apache.org/jira/browse/YARN-111 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.0.1-alpha Reporter: nemon lou We need application level priority for Hadoop 2.0,both in FIFO scheduler and Capacity Scheduler. In Hadoop 1.0.x,job priority is supported. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
