[
https://issues.apache.org/jira/browse/YARN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546467#comment-14546467
]
Carlo Curino commented on YARN-3656:
------------------------------------
I worked with Ishai and Jonathan closely on this, and the integration with
YARN-1051 is done rather carefully.
After a month running experiments they confirmed consistently better
performance on all the key metrics and
reasonable runtimes.
I would argue that after a careful code review and some more testing, the
LowCost agent he propose
should become our "default" for use of reservations, as dominates the greedy
agent we have today.
> LowCost: A Cost-Based Placement Agent for YARN Reservations
> -----------------------------------------------------------
>
> Key: YARN-3656
> URL: https://issues.apache.org/jira/browse/YARN-3656
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, resourcemanager
> Reporter: Ishai Menache
> Attachments: LowCostRayonExternal.pdf
>
>
> YARN-1051 enables SLA support by allowing users to reserve cluster capacity
> ahead of time. YARN-1710 introduced a greedy agent for placing user
> reservations. The greedy agent makes fast placement decisions but at the cost
> of ignoring the cluster committed resources, which might result in blocking
> the cluster resources for certain periods of time, and in turn rejecting some
> arriving jobs.
> We propose LowCost – a new cost-based planning algorithm. LowCost “spreads”
> the demand of the job throughout the allowed time-window according to a
> global, load-based cost function.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
