[
https://issues.apache.org/jira/browse/YARN-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305014#comment-15305014
]
Carlo Curino edited comment on YARN-5164 at 5/28/16 12:12 AM:
--------------------------------------------------------------
Given some convo with [~chris.douglas] there might be a workaround if we avoid
to report the exact point of violation. The intuition is to compute separately
the entire set of integral "additions" and "removals" separately, and then use
our usual {{RLESparseResourceAllocation.merge()}} to compute the correct
integral and compare it to the avg constraint. This could work (passes all the
tests), but we should dig deeper in corner-cases, as it is tricky. Posting an
initial patch (fresh our of hacking it) to see if folks can help spot issues. I
will circle back on this.
There is however a risk of false-positive/false-negatives during the
transitions (as we are not capturing the "slope" of the integral, per our
previous comment).
was (Author: curino):
Given some convo with [~chris.douglas] there might be a workaround if we avoid
to report the exact point of violation. The intuition is to compute separately
the entire set of integral "additions" and "removals" separately, and then use
our usual {{RLESparseResourceAllocation.merge()}} to compute the correct
integral and compare it to the avg constraint. This could work (passes all the
tests), but we should dig deeper in corner-cases, as it is tricky. Posting an
initial patch (fresh our of hacking it) to see if folks can help spot issues. I
will circle back on this.
> CapacityOvertimePolicy does not take advantaged of plan RLE
> -----------------------------------------------------------
>
> Key: YARN-5164
> URL: https://issues.apache.org/jira/browse/YARN-5164
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler, fairscheduler, resourcemanager
> Reporter: Carlo Curino
> Assignee: Carlo Curino
> Attachments: YARN-5164-example.pdf, YARN-5164.1.patch
>
>
> As a consequence small time granularities (e.g., 1 sec) and long time horizon
> for a reservation (e.g., months) run rather slow (10 sec).
> Proposed resolution is to switch to interval math in checking, similar to how
> YARN-4359 does for agents.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
