Xie YiFan created YARN-11643:
--------------------------------
Summary: Skip unnecessary pre-check in Multi Node Placement
Key: YARN-11643
URL: https://issues.apache.org/jira/browse/YARN-11643
Project: Hadoop YARN
Issue Type: Improvement
Components: capacityscheduler
Reporter: Xie YiFan
Assignee: Xie YiFan
When Multi Node Placement enabled, RegularContainerAllocator do a while loop to
find one node from candidate set to allocate for a given scheduler key. Before
do allocate, pre-check be called to check if current node satisfies check. If
this node does not pass all checks, just continue to next node.
{code:java}
if (reservedContainer == null) {
result = preCheckForNodeCandidateSet(node,
schedulingMode, resourceLimits, schedulerKey);
if (null != result) {
continue;
}
} {code}
But some checks are related to scheduler Key or Application which return
PRIORITY_SKIPPED or APP_SKIPPED. It means that if first node does not pass
check, the following nodes also do not pass.
If cluster have 5000 nodes in default partition, Scheduler will waste 5000
times loop for just one scheduler key.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
