[
https://issues.apache.org/jira/browse/YARN-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13979295#comment-13979295
]
Carlo Curino commented on YARN-1957:
------------------------------------
Attached a patch fixing the issues above. Conceptually we compute the fix-point
allocation for non-zero queues, and then spread the remaining capacity
uniformly among zero-size queues. Also each "offer" to a queue now handles
maxCapacity.
The old behavior for zero-size queues (i.e., preemption of all containers
regardless to external pressure) should be possible to obtain by setting
maxCapacity = capacity = 0;
Test have been updated to verify maxCapacity behavior as well. We also tested
on a medium size cluster running gridmix with lots of preemptions and it seems
to behave as expected,
however a careful code review from someone else is more than welcome.
> ProportionalCapacitPreemptionPolicy handling of corner cases...
> ---------------------------------------------------------------
>
> Key: YARN-1957
> URL: https://issues.apache.org/jira/browse/YARN-1957
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-1957.patch, YARN-1957_test.patch
>
>
> The current version of ProportionalCapacityPreemptionPolicy should be
> improved to deal with the following two scenarios:
> 1) when rebalancing over-capacity allocations, it potentially preempts
> without considering the maxCapacity constraints of a queue (i.e., preempting
> possibly more than strictly necessary)
> 2) a zero capacity queue is preempted even if there is no demand (coherent
> with old use of zero-capacity to disabled queues)
> The proposed patch fixes both issues, and introduce few new test cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
