Tao Yang created YARN-9598:
------------------------------
Summary: Make reservation work well when multi-node enabled
Key: YARN-9598
URL: https://issues.apache.org/jira/browse/YARN-9598
Project: Hadoop YARN
Issue Type: Bug
Components: capacityscheduler
Reporter: Tao Yang
Assignee: Tao Yang
This issue is to solve problems about reservation when multi-node enabled:
# As discussed in YARN-9576, re-reservation proposal may be always generated
on the same node and break the scheduling for this app and later apps. I think
re-reservation in unnecessary and we can replace it with LOCALITY_SKIPPED to
let scheduler have a chance to look up follow candidates for this app when
multi-node enabled.
# Scheduler iterates all nodes and try to allocate for reserved container in
LeafQueue#allocateFromReservedContainer. Here there are two problems:
** The node of reserved container should be taken as candidates instead of all
nodes when calling FiCaSchedulerApp#assignContainers, otherwise later scheduler
may generate a reservation-fulfilled proposal on another node, which will
always be rejected in FiCaScheduler#commonCheckContainerAllocation.
** Assignment returned by FiCaSchedulerApp#assignContainers could never be
null even if it's just skipped, it will break the normal scheduling process for
this leaf queue because of the if clause in LeafQueue#assignContainers: "if
(null != assignment) \{ return assignment;}"
# Nodes which have been reserved should be skipped when iterating candidates
in RegularContainerAllocator#allocate, otherwise scheduler may generate
allocation or reservation proposal on these node which will always be rejected
in FiCaScheduler#commonCheckContainerAllocation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
