[
https://issues.apache.org/jira/browse/YARN-3656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627082#comment-14627082
]
Ishai Menache commented on YARN-3656:
-------------------------------------
Thank you [~asuresh] for your comments!
We will remove the redundant “if” conditions from TryManyReservationAgents.java.
Regarding your suggestion of incorporating a fallback strategy for
ReservationAgent – it is an interesting idea, but we believe it might lead to a
couple of issues.
The behavior of TryManyReservationAgents is not straightforward. For example,
if you have an agent defined by RA = TryManyReservationAgents{ RA1, RA2, RA3 },
the overall performance of RA depends on how the algorithms RA1, RA2 and RA3
affect one another, and this dependency can be complex. In general, you might
want to implement several TryMany policies with different fallback policies,
depending on the available capacity, job characteristics etc. This is why we
preferred to implement the collection of the algorithms as a LinkedList, which
gives full flexibility to the user. On a related note, the fallback setting
might accidentally end up in a cyclic dependency between the different
algorithms. Please let us know if you agree.
> LowCost: A Cost-Based Placement Agent for YARN Reservations
> -----------------------------------------------------------
>
> Key: YARN-3656
> URL: https://issues.apache.org/jira/browse/YARN-3656
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler, resourcemanager
> Affects Versions: 2.6.0
> Reporter: Ishai Menache
> Assignee: Jonathan Yaniv
> Labels: capacity-scheduler, resourcemanager
> Attachments: LowCostRayonExternal.pdf, YARN-3656-v1.1.patch,
> YARN-3656-v1.2.patch, YARN-3656-v1.patch, lowcostrayonexternal_v2.pdf
>
>
> YARN-1051 enables SLA support by allowing users to reserve cluster capacity
> ahead of time. YARN-1710 introduced a greedy agent for placing user
> reservations. The greedy agent makes fast placement decisions but at the cost
> of ignoring the cluster committed resources, which might result in blocking
> the cluster resources for certain periods of time, and in turn rejecting some
> arriving jobs.
> We propose LowCost – a new cost-based planning algorithm. LowCost “spreads”
> the demand of the job throughout the allowed time-window according to a
> global, load-based cost function.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
