Andras Gyori created YARN-10622:
-----------------------------------
Summary: Fix preemption policy to exclude childless ParentQueus
Key: YARN-10622
URL: https://issues.apache.org/jira/browse/YARN-10622
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Andras Gyori
Assignee: Andras Gyori
ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be
preempted by this logic:
{code:java}
private Set<String> getLeafQueueNames(TempQueuePerPartition q) {
// If its a ManagedParentQueue, it might not have any children
if ((q.children == null || q.children.isEmpty())
&& !(q.parentQueue instanceof ManagedParentQueue)) {
return ImmutableSet.of(q.queueName);
}
Set<String> leafQueueNames = new HashSet<>();
for (TempQueuePerPartition child : q.children) {
leafQueueNames.addAll(getLeafQueueNames(child));
}
return leafQueueNames;
}
{code}
This, however does not take childless ParentQueues (which was introduced in
YARN-10596) into account.
A childless ParentQueue will throw a NPE in
FifoCandidatesSelector#selectCandidates:
{code:java}
LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
RMNodeLabelsManager.NO_LABEL).leafQueue;
{code}
TempQueuePerPartition has a leafQueue member variable, which is null, if the
queue is not a LeafQueue. In case of childless ParentQueue, it is null, but its
name is present in the leafQueueNames as stated before.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
