[
https://issues.apache.org/jira/browse/YARN-6151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854730#comment-15854730
]
Yufei Gu commented on YARN-6151:
--------------------------------
Uploaded one simple solution is that before we choose which queue to preempt,
we check if this queue can be preempted. For leaf queue, we just check if the
queue uses more resources than its fair share. For parent queue, we need to
recursively go down to each level to check. There might be a performance
concern, but considering the performance is still acceptable before YARN-5969
which did recursively get resource usage anyway, it should be fine.
> FS Preemption doesn't filter out queues which cannot be preempted
> -----------------------------------------------------------------
>
> Key: YARN-6151
> URL: https://issues.apache.org/jira/browse/YARN-6151
> Project: Hadoop YARN
> Issue Type: Bug
> Components: fairscheduler
> Affects Versions: 2.8.0
> Reporter: Yufei Gu
> Assignee: Yufei Gu
> Attachments: YARN-6151.001.patch
>
>
> This is preemption bug happens before 2.8.0, which also described in
> YARN-3405.
> Queue hierarchy described as below:
> {noformat}
> root
> / \
> queue-1 queue-2
> / \
> queue-1-1 queue-1-2
> {noformat}
> Assume cluster resource is 100 and all queues have same weights.
> # queue-1-1 and queue-2 has apps. Each get 50 usage and 50 fairshare.
> # When queue-1-2 is active, supposedly it will preempt 25 from queue-1-1, but
> this doesn't happen because preemption happens top-down, queue-2 could be the
> preemption candidate as long as queue-2 is less needy than queue-1, and
> queue-2 doesn't exceed the fair share which means preemption won't happen.
> We need to filter out queue-2 since it isn't a valid candidate.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
