[
https://issues.apache.org/jira/browse/YARN-10589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277031#comment-17277031
]
Qi Zhu commented on YARN-10589:
-------------------------------
[~tanu.ajmera]
I agree with [~ztang] that we need to split the code to partition related
logic, then to return:
{code:java}
public boolean precheckNode(SchedulerRequestKey schedulerKey,
SchedulerNode schedulerNode, SchedulingMode schedulingMode,
Optional<DiagnosticsCollector> dcOpt) {
this.readLock.lock();
try {
AppPlacementAllocator ap =
schedulerKeyToAppPlacementAllocator.get(schedulerKey);
return (ap != null) && (ap.getPlacementAttempt() < retryAttempts) &&
ap.precheckNode(schedulerNode, schedulingMode, dcOpt);
} finally {
this.readLock.unlock();
}
}
{code}
> Improve logic of multi-node allocation
> --------------------------------------
>
> Key: YARN-10589
> URL: https://issues.apache.org/jira/browse/YARN-10589
> Project: Hadoop YARN
> Issue Type: Task
> Affects Versions: 3.3.0
> Reporter: Tanu Ajmera
> Assignee: Tanu Ajmera
> Priority: Major
> Attachments: YARN-10589-001.patch, YARN-10589-002.patch,
> YARN-10589-003.patch
>
>
> {code:java}
> for (String partititon : partitions) {
> if (current++ > start) {
> break;
> }
> CandidateNodeSet<FiCaSchedulerNode> candidates =
> cs.getCandidateNodeSet(partititon);
> if (candidates == null) {
> continue;
> }
> cs.allocateContainersToNode(candidates, false);
> }{code}
> In above logic, if we have thousands of node in one partition, we will still
> repeatedly access all nodes of the partition thousands of times. There is no
> break point where if the partition is not same for the first node, it should
> stop checking other nodes in that partition.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
