[
https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16851660#comment-16851660
]
Juanjuan Tian commented on YARN-7494:
--------------------------------------
Thanks Weiwei for your reply. here seems there is another issue in
RegularContainerAllocator#allocate,
there it iterates though all nodes, but the reservedContainer doesn't change
with the iterated node, for muti-node policy, the reservedContainer and the
iterated node will be inconsistent, and may procude incorrect
ContainerAllocation(even though this ContainerAllocation will be abondoned at
last, but it seems really wastes opportunity). [~cheersyang] what's your
thought about this situation
while (iter.hasNext()) {
FiCaSchedulerNode node = iter.next();
if (reservedContainer == null) {
result = preCheckForNodeCandidateSet(clusterResource, node,
schedulingMode, resourceLimits, schedulerKey);
if (null != result) {
continue;
}
} else {
// pre-check when allocating reserved container
if (application.getOutstandingAsksCount(schedulerKey) == 0) {
// Release
result = new ContainerAllocation(reservedContainer, null,
AllocationState.QUEUE_SKIPPED);
continue;
}
}
result = tryAllocateOnNode(clusterResource, node, schedulingMode,
resourceLimits, schedulerKey, reservedContainer);
if (AllocationState.ALLOCATED == result.getAllocationState()
|| AllocationState.RESERVED == result.getAllocationState()) {
result = doAllocation(result, node, schedulerKey, reservedContainer);
break;
}
}
> Add muti-node lookup mechanism and pluggable nodes sorting policies to
> optimize placement decision
> --------------------------------------------------------------------------------------------------
>
> Key: YARN-7494
> URL: https://issues.apache.org/jira/browse/YARN-7494
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacity scheduler
> Reporter: Sunil Govindan
> Assignee: Sunil Govindan
> Priority: Major
> Fix For: 3.2.0
>
> Attachments: YARN-7494.001.patch, YARN-7494.002.patch,
> YARN-7494.003.patch, YARN-7494.004.patch, YARN-7494.005.patch,
> YARN-7494.006.patch, YARN-7494.007.patch, YARN-7494.008.patch,
> YARN-7494.009.patch, YARN-7494.010.patch, YARN-7494.11.patch,
> YARN-7494.12.patch, YARN-7494.13.patch, YARN-7494.14.patch,
> YARN-7494.15.patch, YARN-7494.16.patch, YARN-7494.17.patch,
> YARN-7494.18.patch, YARN-7494.19.patch, YARN-7494.20.patch,
> YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup
> based on partition to start with.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
