[jira] [Commented] (YARN-2393) Fair Scheduler : Implement steady fair share
[ https://issues.apache.org/jira/browse/YARN-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105903#comment-14105903 ] Wei Yan commented on YARN-2393: --- Thanks, karthik. The javadoc looks good to me. Fair Scheduler : Implement steady fair share Key: YARN-2393 URL: https://issues.apache.org/jira/browse/YARN-2393 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: Ashwin Shankar Assignee: Wei Yan Attachments: YARN-2393-1.patch, YARN-2393-2.patch, YARN-2393-3.patch, yarn-2393-4.patch Static fair share is a fair share allocation considering all(active/inactive) queues.It would be shown on the UI for better predictability of finish time of applications. We would compute static fair share only when needed, like on queue creation, node added/removed. Please see YARN-2026 for discussions on this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2393) Fair Scheduler : Implement steady fair share
[ https://issues.apache.org/jira/browse/YARN-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105910#comment-14105910 ] Karthik Kambatla commented on YARN-2393: Cool. +1, pending Jenkins. Fair Scheduler : Implement steady fair share Key: YARN-2393 URL: https://issues.apache.org/jira/browse/YARN-2393 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: Ashwin Shankar Assignee: Wei Yan Attachments: YARN-2393-1.patch, YARN-2393-2.patch, YARN-2393-3.patch, yarn-2393-4.patch Static fair share is a fair share allocation considering all(active/inactive) queues.It would be shown on the UI for better predictability of finish time of applications. We would compute static fair share only when needed, like on queue creation, node added/removed. Please see YARN-2026 for discussions on this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2393) Fair Scheduler : Implement steady fair share
[ https://issues.apache.org/jira/browse/YARN-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14106398#comment-14106398 ] Wei Yan commented on YARN-2393: --- For the maxAMResource, the patch still uses the dynamic one. Karthik and I discussed offline, and even we change to steady fair share, one queues still can be blocked by AM containers if more queues are created during runtime. Fair Scheduler : Implement steady fair share Key: YARN-2393 URL: https://issues.apache.org/jira/browse/YARN-2393 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: Ashwin Shankar Assignee: Wei Yan Attachments: YARN-2393-1.patch, YARN-2393-2.patch, YARN-2393-3.patch, yarn-2393-4.patch Static fair share is a fair share allocation considering all(active/inactive) queues.It would be shown on the UI for better predictability of finish time of applications. We would compute static fair share only when needed, like on queue creation, node added/removed. Please see YARN-2026 for discussions on this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2393) Fair Scheduler : Implement steady fair share
[ https://issues.apache.org/jira/browse/YARN-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099392#comment-14099392 ] Karthik Kambatla commented on YARN-2393: bq. Should we bound maxAMResource by steady fair share instead, which doesn't fluctuate? If we use steady fairshare, creating many queues can lead to not having enough resources to run an AM. Using instantaneous fairshare doesn't let this happen. However, it is possible that the amShare becomes greater than what it should be for a potentially lower instantaneous fairshare at a later time. We could preempt AM containers as well, but need to tread this very carefully. Fair Scheduler : Implement steady fair share Key: YARN-2393 URL: https://issues.apache.org/jira/browse/YARN-2393 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: Ashwin Shankar Assignee: Wei Yan Attachments: YARN-2393-1.patch, YARN-2393-2.patch, YARN-2393-3.patch Static fair share is a fair share allocation considering all(active/inactive) queues.It would be shown on the UI for better predictability of finish time of applications. We would compute static fair share only when needed, like on queue creation, node added/removed. Please see YARN-2026 for discussions on this. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2393) Fair Scheduler : Implement steady fair share
[ https://issues.apache.org/jira/browse/YARN-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099499#comment-14099499 ] Wei Yan commented on YARN-2393: --- bq. However, it is possible that the amShare becomes greater than what it should be for a potentially lower instantaneous fairshare at a later time. We could preempt AM containers as well, but need to tread this very carefully. For the preemption, one concern is that do we need to turn on the preemption.enabled. Users may be confused if AM containers killed, but the preemption is disabled. Fair Scheduler : Implement steady fair share Key: YARN-2393 URL: https://issues.apache.org/jira/browse/YARN-2393 Project: Hadoop YARN Issue Type: Improvement Components: fairscheduler Reporter: Ashwin Shankar Assignee: Wei Yan Attachments: YARN-2393-1.patch, YARN-2393-2.patch, YARN-2393-3.patch Static fair share is a fair share allocation considering all(active/inactive) queues.It would be shown on the UI for better predictability of finish time of applications. We would compute static fair share only when needed, like on queue creation, node added/removed. Please see YARN-2026 for discussions on this. -- This message was sent by Atlassian JIRA (v6.2#6252)
